MRO-W Final Reports 2008-2009

The increased use of technology in the field of education has raised questions about the effectiveness of educational software, in particular, education games. This research project’s goal was to determine how effective reading and math software games are for children in the PK-12 age group. The research team consisted of two faculty members, one from the department of education and one from the information technology department, and three student researchers. The project was conducted at two sites, one a pre-school for children ages 3-5 and the other a Boys and Girls Club in an after school program for children in 4th-6th grades. The children at the pre-school were separated into two groups with each playing a different computer game intended to teach early literacy skills. The children at the Boys and Girls Club, also separated into two groups, played games intended to increase math skills.
We began the project by gathering articles for the literature review portion. The goal was to find articles that could give us insight into projects that were similar to ours as well as to discover what the results of those studies were for comparison. We met as a group each week to discuss our progress and detailed our work in weekly online journals. We would do this throughout the project. One student started a website for documentation of our project and another took over design of the website and have updated it with our journals each week during the project.
When we felt we had gathered a sufficient amount of articles for our literature review, we divided the concepts related to both education and computer games into sections and each of us wrote a four-page entry for our literature review. We looked at the features of educational computer games and w how certain features of these games such as, color, animation, sound, and instant feedback, were used to enhance learning. In some cases they were effective and in other cases they were not.
The next step in the process was to decide on a site for our research and to determine which games would be used. We discussed the features of computer games that from our literature review were shown to be most effective. After an exhaustive search, we decided on two early literacy games that were available online, Starfall (www.starfall.com) and AOL’s Learn Your Letters (kids.aol.com/kol-jr/games/learn-your-letters ) for the pre-school research. For the older children, we decided on two math software games, Math Detective and Math Essentials. The student researchers documented the features that each game had and used this list to create surveys that would be used for teachers and the older students to fill out. The sites were chosen based on proximity to campus and the age range of the children that were available to meet our needs for the study. The pre-school site was the CSUMB Child Development Center. The children had been taught some of their letters by the staff and would be good representatives for this part of our study. The Boys and Girls Club of Salinas was chosen because the children could participate after school and the children could be recruited from the club.
We worked on our Human Subjects application as well as the consent forms for parents of the children to sign before participation in the project. Each of us went through an online test on working with human subjects and was certified to work with the children at both sites. Unfortunately, both data collections were delayed due to unforeseen circumstances including a staff change at the pre-school and slow recruitment at the boys and girls club. The pre-school data collection began in mid to late spring and the boys and girls club data collection began in early summer.
The pre-school data collection began with the pre-testing of the children on initial sound fluency and letter naming fluency. During the following six weeks, each child spent ten minutes on each computer every day Monday-Friday. The first two weeks a student researcher was there each day to ensure the fidelity of the study. At this point, the staff of the center asked us to not come each day because of the disruption they felt it caused in the children’s day. We followed up at the center to do fidelity checks twice a week. The staff marked down on a signup sheet each time a child played their assigned game. Post testing was conducted to determine the effects of the computer use on the children’s reading skills. The same tests were used in the post-test that were used in the pre-test.
The Boys and Girls Club data collection began with pre-testing of the children in math skills appropriate to grades 4th-6th. We spent several weeks recruiting club members to participate in the project. We had an outpouring of interest and support, but the informal atmosphere of the club made it difficult to track down club members who were interested and had simply forgotten about the project, or who had stopped attending the club. Once we had enough members join our project the research began. The student researchers split the time up amongst the three of us and each of us monitored the children playing the games. The students were signed up with parental consent forms. Each child spent two hours a week on the game they were assigned. One group attended on Monday and Wednesday and the other on Tuesday and Thursday. This pattern continued for 6 weeks and was supplemented with rewards, such as “club dollars” and an ice cream party for finishing. After six weeks, the students were post-tested using the same math tests that were used for the pre-test.
Because of the delays in data collection, the boys and girls club data will need more time to be analyzed. We have some conclusions from the pre-school data,. The statistics showed that the group who played the AOL Learn Your Letters game had higher post-test scores in the area of letter naming fluency than the children who played the Starfall game. This was a surprising result since all of the teachers rated that the children did not seem to enjoy the game or to be engaged by it. Our preliminary interpretation is that the increased scores are due to the fact that the children playing the AOL game saw more letters in the ten minutes that they played than the children on the other computer. The Starfall game took more time per letter than the AOL game. The letters were repeated more often, then for the children on the AOL game.
A course management system was used to coordinate our project and a website was created for this project where we posted the progress of the group through our journal entries, biographies and research we used in the bibliography. We plan on going to the AECT conference in October by funding from our universities UROC program, which funds undergraduate research.

Individual Reflections:
Cassidy Bennett: Individual results I have concluded from this research project is to always be open and optimistic to new ideas and remember that research does not always go as planned. Sometimes the road will be straight ahead and other times there are bumps and hills that need to be overcome. I will always be open to future research in my life, especially ones involving education and technology. I hope there will be many more opportunities like this one.
Amelia Weigant: The data is still in the process of being reviewed and summarized. This project presented plenty of challenges to overcome and was a fascinating, enlightening view into the world of academic research.
Teresa Sundholm: My involvement in the project was from the very beginning. I had never been involved in a research project before, but as I was in my senior year of study with plans to attend graduate school in the field of instructional technology, I felt this project would be an opportunity to explore some of the issues related to computers and education. I have been working on the final report for the pre-school and writing about how we conducted our research. I have recently graduated from my undergraduate program and started my master’s degree program. This project has been very helpful in my master’s studies. I have an appreciation for how research projects are conducted and how important computers have become in education. The work I have done on the papers, literature review, and onsite data collection has been invaluable to my education and future goals.

Project website: http://myspace.csumb.edu/~mro_w/index.html
Presentation: Here is the proposal accepted to present during the 2009 AECT International Convention, Oct 27-31, Louisville, Kentucky. The paper for this conference proceeding will be done in a few weeks. We plan to publish at least one article based on the other research site data.
Research Title: Educational Games in the PK-12 Environment
Abstract: The purpose of this research is to determine the effect of learning games on the literacy of preschool children in terms of learning outcomes, level of engagement and satisfaction. The presenters will discuss the preliminary findings of this research as well as the practical implications of using learning games with young learners. Additionally, the presenters will discuss the teacher satisfaction with this intervention. The last purpose of this presentation is to discuss future research opportunities as well as recommendations for future research.

Proposal:
In recent years, the computer game and simulation industry has grown fast and become a part of our culture. They are not only welcomed by many educators as a way to motivate learners, but it also serve as a catalyst for learning. Prensky (2001, 2006), Aldrich (2005), Gee (2003, 2005), and Shaffer (2006) have written extensively about the potential benefits of using computer games for learning. According to Mitchell & Savill-Smith (2004), games “accommodate different leaning styles” through the combinations of “video, audio, and text, thereby promoting confidence and encouraging multimodal literacy.” Game players, especially children, are more interested in games with good audio and video. Kirriemuir & McFarlane (2004) identifies that the experience of game play seems to be affecting learners’ expectations of learning activities. Therefore, the idea of integrating games and simulations into curriculum is becoming an important research topic in education.
The use of computers has been prevalent in education even at the preschool level. Much of the research that has been conducted in this setting has looked at the impact on the social skills of preschool children (Chen & Chang, 2006). Using computers in the preschool setting, has also been found to increase the motivation of young children (Mioduser, Tur-Kaspa, & Leitner, 2000). However, there is a lack of understanding whether game-integrated technology tools indeed can enhance student learning outcomes in the area of literacy as compared to technology tools without games and simulations. There is a growing need to know empirically, how well the learning with simulations and games really works especially in the preschool setting. Limited research has been completed in this setting as this is a relatively new line of research (Chen & Chang, 2006; Clements & Sarama, 2003). As a result, a key goal of this study is to investigate how computer games may affect student learning outcomes in the preschool setting.
This project is being funded by the National Science Foundation. The focus of this funding course was Multidisciplinary Research Opportunities for Women (MRO-W). The funds from this grant as used to mentor three undergraduate females in the research process. The students identified for this project are being funded to complete research in the area of computer science. Currently, we have been collaborating with a local Child Development Center, and it is our research site. This research focuses on the benefits of educational games on literacy skills of preschool children. This presentation will discuss the preliminary findings of this student-led research. The purpose of this research is to determine the effect of learning games on the literacy of preschool children. Preschool students, ages 3 and 4, who attend this preschool will be selected to participate in this study. The students will be assigned two groups randomly. Students in the first group will receive 10 minutes daily engaged with a literacy based learning game. Students in the comparison group will receive an equal amount of time at the computer center. These students will be engaged with math-based learning games. Pre and post-test data will be collected using the Kaufman Survey of Early Academic Skills(K-SEALS) and the Bracken School Readiness Assessment-3. Teacher interviews will be conducted after the intervention ends to ascertain the teacher satisfaction with the intervention.
The presenters will discuss the preliminary findings of this research as well as the practical implications of using learning games with this very young population. Additionally, the presenters will discuss the teacher satisfaction with this intervention. The last purpose of this presentation is to discuss future research opportunities as well as recommendations for future research.
The overall research questions:
1. Is there a significant difference in literacy skills between the two games?
2. What features of the learning games may enhance student engagement?
3. What features of the learning games enhanced teacher perceptions of student satisfaction?
References
Aldrich, C. (2005). Learning by doing: A comprehensive guide to simulations, computer games, and pedagogy in e-learning and other educational experiences. San Francisco: CA: Pfeiffer.
Chen, J. Q., & Chang, C. (2006). Using computers in early childhood classrooms: Teachers’ attitudes, skills and practices. Journal of Early Childhood Research, 4(2), 169-188.
Clements, D.H., & Sarama, J. (2003). Young children and technology: What does the research say? Young Children, 58(6), 34-40.
Gee, J. P. (2003). What video games have to teach us about learning and literacy? New York: Palgrave Macmillan.
Gee, J. P. (2005). Why video games are good