Category: Research

Crowdfunding for Research on Educational Computer Games

I’ve looked at the crowdfunding of games before, and I’ve highlighted some interesting educational games via Twitter (this one in particular) that are going through the Kickstarter process.

Now an effort to study the lasting effects of educational games is starting up on Experiment.com, led by Jane Hornickel of Data Sense LLC. She seeks $3,400 to study the lasting benefits of educational videogames in students with learning difficulties.

Educational computer games can help students make big gains in school, particularly those who have learning difficulties. But it’s unclear how long these benefits last after finishing the games. I will look at test scores for children up to two years after they play educational computer games to see if children maintain their gains.

Crowdfunding for scientific purposes is an interesting phenomenon. Stay tuned to see if Dr. Hornickel successfully raises the needed funds.

-

Latest Research on Violent Videogames

Putting another log on the old fire of violent videogame controversies, Douglas Gentile’s latest study has been released online in JAMA Pediatrics.

As played out in the media, the question always is, Do violent videogames make children violent? For researchers, of course, the question is much more nuanced. For one thing, while measuring “violent” content in videogames is relatively easy, how do you measure “violence” in children? One approach that researchers like Gentile used in the past was to measure something called “violent arousal” in subjects. This is a physiological reaction to stimuli that can be easily measured in subjects. The problem with that approach, when trying to paint violent videogames with a negative brush, is that all “violent” media can gin up “violent arousal” in subjects. Physiological results are similar with “violent” music, movies, television shows, literature, etc.

Much more difficult to measure are long term attitudes and actions which may be influenced by videogames or other media. Gentile’s newest study tackles that issue, with a three year longitudinal study of 3,034 students in Singapore. Here is the abstract:

Importance Although several longitudinal studies have demonstrated an effect of violent video game play on later aggressive behavior, little is known about the psychological mediators and moderators of the effect.

Objective To determine whether cognitive and/or emotional variables mediate the effect of violent video game play on aggression and whether the effect is moderated by age, sex, prior aggressiveness, or parental monitoring.

Design, Setting, and Participants Three-year longitudinal panel study. A total of 3034 children and adolescents from 6 primary and 6 secondary schools in Singapore (73% male) were surveyed annually. Children were eligible for inclusion if they attended one of the 12 selected schools, 3 of which were boys’ schools. At the beginning of the study, participants were in third, fourth, seventh, and eighth grades, with a mean (SD) age of 11.2 (2.1) years (range, 8-17 years). Study participation was 99% in year 1.

Main Outcomes and Measures The final outcome measure was aggressive behavior, with aggressive cognitions (normative beliefs about aggression, hostile attribution bias, aggressive fantasizing) and empathy as potential mediators.

Results Longitudinal latent growth curve modeling demonstrated that the effects of violent video game play are mediated primarily by aggressive cognitions. This effect is not moderated by sex, prior aggressiveness, or parental monitoring and is only slightly moderated by age, as younger children had a larger increase in initial aggressive cognition related to initial violent game play at the beginning of the study than older children. Model fit was excellent for all models.

Conclusions and Relevance Given that more than 90% of youths play video games, understanding the psychological mechanisms by which they can influence behaviors is important for parents and pediatricians and for designing interventions to enhance or mitigate the effects.

The study has not generated a ton of reaction in the media so far, maybe because this is a horse that has been beaten so many times, to borrow a phrase. WTOP, a news radio station for the Washington, D.C. area, noted the study is “controversial,” and reporter Paula Wolfson went to the trouble of interviewing Angela Fletcher, with the Children’s National Health Network for some additional opinions on the matter. Wolfson’s report on the study is the best so far, in my opinion.

Bottom line, the issue is too complex to place in a convenient box. Exposure to any kind of “violent” media may cause a person to engage in violence at some point in the future. Or not. But, the research continues.

-

Meta-Study: Virtual Reality Based Instruction is Effective

An interesting meta-study from researchers at Texas A&M looked at instruction delivered through videogames, simulations, and virtual worlds. While all were found to be effective, educational videogames were found to be the most effective.

Here is the abstract:

The purpose of this meta-analysis is to examine overall effect as well as the impact of selected instructional design principles in the context of virtual reality technology-based instruction (i.e. games, simulation, virtual worlds) in K-12 or higher education settings. A total of 13 studies (N = 3081) in the category of games, 29 studies (N = 2553) in the category of games, and 27 studies (N = 2798) in the category of virtual worlds were meta-analyzed. The key inclusion criteria were that the study came from K-12 or higher education settings, used experimental or quasi-experimental research designs, and used a learning outcome measure to evaluate the effects of the virtual reality-based instruction.

Results suggest games (FEM = 0.77; REM = 0.51), simulations (FEM = 0.38; REM = 0.41), and virtual worlds (FEM = 0.36; REM = 0.41) were effective in improving learning outcome gains. The homogeneity analysis of the effect sizes was statistically significant, indicating that the studies were different from each other. Therefore, we conducted moderator analysis using 13 variables used to code the studies. Key findings included that: games show higher learning gains than simulations and virtual worlds. For simulation studies, elaborate explanation type feedback is more suitable for declarative tasks whereas knowledge of correct response is more appropriate for procedural tasks. Students performance is enhanced when they conduct the game play individually than in a group. In addition, we found an inverse relationship between number of treatment sessions learning gains for games.

With regards to the virtual world, we found that if students were repeatedly measured it deteriorates their learning outcome gains. We discuss results to highlight the importance of considering instructional design principles when designing virtual reality-based instruction.

Reference:

Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education, 70, 29-40.

-

-

#Gamestudy Roundup for April 2013

Here’s a roundup of some of the research on videogames I’ve been posting on Twitter. Follow me @EduGamRes, or search for the hashtag #Gamestudy to see more.

Diet techniques learned in Second Life transferred to weight loss in real life

Researchers at University of Kansas Medical School followed 20 subjects in a weight loss program, either visiting a clinic in real life or nutritionists in a virtual clinic in Second Life. Then all the subjects followed an additional six months of training in the game environment. The completely virtual program was found to be more effective.

http://www.jneb.org/article/S1499-4046%2812%2900667-7/abstract

-==-

Kids diagnosed with autism may be subject to greater likelihood of videogame overuse

Micah Mazurek at University of Missouri followed 202 children diagnosed as falling in the autism spectrum disorder, and 179 typically developing children. Those with ASD were found to engage in videogames more, and in social media less.

According to the MU News Bureau:

The study, “Television, Video Game and Social Media Use among Children with ASD and Typically Developing Siblings,” will be published in an upcoming issue of the Journal of Autism and Developmental Disorders. “Video Game Use and Problem Behaviors in Boys with Autism Spectrum Disorders,” was published in Research in Autism Spectrum Disorders.

-==-

Concept paper at SigBovik 2013 shows computer programs playing videogames

LearnFun and PlayFun are two computer programs. The first watches a human player engage in a videogame. The second one then plays and wins the game in a more efficient manner. The video shows author Tom Murphy play the first level of Super Mario Bros., then the computer program plays the level.

-==-

Study shows videogames don’t affect behavior

University of Glasgow researchers followed 13,587 children in the UK, finding that television screen time showed a minor effect on children, while videogame screen time showed none. The study contrasts with previous studies suggesting a link to violent behavior with increased screen time.

http://adc.bmj.com/content/98/5/341.full.pdf+html

-==-

Action videogames may help students with dyslexia

Researchers at University of Padua in Italy found that regularly playing action videogames increased the reading abilities of children diagnosed with dyslexia. Non action videogame did not produce the same results. The increase was observed without additional direct phonological training.

http://www.sciencedirect.com/science/article/pii/S0960982213000791

-

Research on Benefits of Children Playing Videogames

Some interesting new studies and projects have come out recently showing beneficial links to videogame playing for children. Linda A. Jackson, professor of psychology over at Michigan State University, led a study finding that videogame play was a strong predictor of creativity in children. Here is the abstract:

This research examined relationships between children’s information technology (IT) use and their creativity. Four types of information technology were considered: computer use, Internet use, videogame playing and cell phone use. A multidimensional measure of creativity was developed based on Torrance’s (1987, 1995) test of creative thinking. Participants were 491 12-year olds; 53% were female, 34% were African American and 66% were Caucasian American. Results indicated that videogame playing predicted of all measures of creativity. Regardless of gender or race, greater videogame playing was associated with greater creativity. Type of videogame (e.g., violent, interpersonal) was unrelated to videogame effects on creativity. Gender but not race differences were obtained in the amount and type of videogame playing, but not in creativity. Implications of the findings for future research to test the causal relationship between videogame playing and creativity and to identify mediator and moderator variables are discussed.

The paper can be downloaded here. The MSU press release is here. The paper is in press, and will be published in an upcoming issue of Computers in Human Behavior.

-+-

Meanwhile, A. Scott Cunningham, an assistant professor of economics over at Baylor, along with Benjamin Engelstätter at the Zentrum für Europäische Wirtschaftsforschung (Center for European Economic Research) and Michael R. Ward at University of Texas Arlington, released a working paper on the Social Science Research Network entitled “Understanding the Effects of Violent Video Games on Violent Crime.”

Researchers have long been able to measure physiological arousal in participants engaging in violent media. This physiological measurement is seen regardless of the media. Violent TV shows, movies, music, and videogames will elicit the measured arousal as study after study has shown. But, more tenuous are assertions this arousal leads to violence elsewhere once participants are away from the media. This study seeks to empirically link violent videogame sales with decreases in reports of violence. Here is the abstract:

Psychological studies invariably find a positive relationship between violent video game play and aggression. However, these studies cannot account for either aggressive effects of alternative activities video game playing substitutes for or the possible selection of relatively violent people into playing violent video games. That is, they lack external validity. We investigate the relationship between the prevalence of violent video games and violent crimes. Our results are consistent with two opposing effects. First, they support the behavioral effects as in the psychological studies. Second, they suggest a larger voluntary incapacitation effect in which playing either violent or non-violent games decrease crimes. Overall, violent video games lead to decreases in violent crime.

The paper can be accessed here. Some good articles discussing it in the media are here and here.

-+-

Finally, work on videogames to assist children in coping with medical problems continues in earnest. A recent example involves the University of Utah’s Engineering Arts and Entertainment (EAE) program, which brings in students from the school’s Dept. of Film and Media Arts and School of Computing to design interactive entertainment. Together with physical therapists and councilors, EAE students created a series of videogames designed to help children stricken with cancer. The unnamed minigames written for the PlayStation3 are currently being beta tested by patients in the pediatric ward at the Primary Children’s Medical Center in Salt Lake City, with possible retail release in the near future. Articles on the games can be found here and here.


What Can Angry Birds Teach us About … ? At the Forefront of Angry Birds Research

With over a billion downloads, Angry Birds is the most popular casual gaming app of all time, so it’s only natural for social scientists to investigate it. Here’s the results of some recent items I found while searching for what educators and others have been researching about the game.

David Kelly, blogging at Misadventures in Learning, notes design elements in Angry Birds spark positive influences for skill acquisition. Players can jump right in with little to no learning curve, follow multiple paths to success, and are offered incentives toward productivity. Its initial platform design assists in simple productivity as well:

One of the reasons Angry Birds is as successful as it is is its accessibility.  Unlike console video games, Angry Birds was designed for mobile devices. It has no tether restricting where it can be played and was in fact designed for mobile phones, a device many people have with them throughout the day.

In addition, the level structure of Angry Birds is packaged in small chunks.  An attempt at a level can be completed in less than 30 seconds.  It’s the perfect design for mobility.

Pertti Saariluoma, Editor-in-Chief of Human Technology, noted the games’ designers professed they have no idea why the game is successful. Indeed, Saariluoma notes, good game and software design often is intuitive rather than proscribed.

Market research firm AYTM.com offered up a handy infographic showing demographics and other data from the game. Interesting nuggets include: a total of 53% of players use the free version with the majority occasionally feeling “addicted” while playing. The firm noted Michael Chorost’s article in Psychology Today listing the “addictive” elements of the game. These include simplicity, reward, and realistically simulated physics. Dr. Chorost speculates a dopamine burst may be released, making the gaming experience a pleasurable one for players. As far as using Angry Birds in the classroom, Dan MacIsaac over at SUNY-Buffalo State notes that Google returns over a million hits for “physics teaching Angry Birds.”

Mobile apps in general are receiving scrutiny from researchers, and Angry Birds is often mentioned since it’s the most popular game. Matthias Böhmer over at the German Research Center for Artificial Intelligence, Brent Hecht, a PhD. student at Northwestern, and their colleagues released a large scale study of mobile app use at Mobile HCI 2011 in Stockholm. They found users spend about an hour a day on their phones, but only about a minute at a time with mobile apps. News apps were found to be more popular in the morning, while gaming apps are more popular in the evening:

Weather checking is, not surprisingly, largely a morning activity, as is the checking of one’s calendar. On the other hand, users’ desire to fling Angry Birds at pigs is absent in the morning, and only picks up in the early afternoon and into the evening. Kindle usage behavior is even more focused in the late evening.

Angry Birds and other popular mobile games will probably continue receiving attention from researchers, with efforts likely to include discerning design details that can be adapted to more educational endeavors, as well as a continued commitment to incorporating the game itself into academics. Research always lags pop culture. By the time several thorough studies of Angry Birds are published, if any ever are, the game will likely have faded in popularity and been replaced by the next new thing.


Using Video Games to Solve Complex Problems

The blogosphere and the Twitterverse were buzzing today with news about the latest crowdsourcing coup, where a video game was used to unravel the molecular structure of viral enzymes that cause AIDS in monkeys.

Such tedious work often requires human cognitive abilities, and combined efforts seem to flourish within a gaming environment. The online game used is called Foldit, and Firas Khatib and Frank DiMaio over at University of Washington’s Dept. of Biochemistry along with several others published a paper in Nature detailing the effort, entitled Crystal structure of a monomeric retroviral protease solved by protein folding game players. Here is their abstract:

Following the failure of a wide range of attempts to solve the crystal structure of M-PMV retroviral protease by molecular replacement, we challenged players of the protein folding game Foldit to produce accurate models of the protein. Remarkably, Foldit players were able to generate models of sufficient quality for successful molecular replacement and subsequent structure determination. The refined structure provides new insights for the design of antiretroviral drugs.

The game looks quite interesting, and by playing you might help make a significant contribution to science.

-

Study: Predicting Player Behavior and How Zynga Profits from Data Analysis

An interesting front page story in The Wall Street Journal today by journalist Nick Wingfield discusses how casual gaming giant Zynga cashes in on their millions of players. After developing Fishville, following in the footsteps of highly successful titles like Farmville, managers noted players spending in-game currency on one type of fish more so than others. The “translucent angler fish” was being purchased more than 6 times the rate of other virtual fish. So the company quickly developed a whole line of translucent sea creatures, charging as much as $4 (this time, in real world money) for more exotic varieties.

This formula has been very successful for the company. Although only about five percent of Zynga’s player base spends serious money in their games, so many millions of people play that the company rakes in millions. They rake in even more by figuring out what the players want through data analysis.

Zynga is transforming the game industry. Traditional videogame companies create games they think players will like, then sell them. Zynga offers free games through Facebook Inc.’s social network, then studies data on how its audience plays them. It uses its findings to fiddle with the games to get people to play longer, tell more Facebook friends about them and buy more “virtual goods.” At the heart of the whole process is Zynga’s ability to analyze reams of data on how players are reacting to its games.

“We’re an analytics company masquerading as a games company,” said Ken Rudin, a Zynga vice president in charge of its data-analysis team, in one of a series of interviews with Zynga executives prior to the company’s July filing for an initial public offering.

This formula for financial success has other companies following Zynga’s lead. Rather than spending millions developing a title with a short shelf life, companies are turning to free games with extras that cost money. The primitive graphics Zynga uses are generally derided by serious gamers, but Zynga aims for the mass market, much the way American beer brewers produce bland beverages that appeal to the most palates.

All of Zynga’s games go through what amounts to a giant ongoing lab experiment involving players. Zynga conducts hundreds of “A-B tests” within its games, in which two sets of players see virtual goods on sale with, say, subtle color differences to see which color sells better…

Sizhao Yang, a former Zynga executive who helped create its virtual farming hit “FarmVille,” says his development team figured out by analyzing virtual-goods-sales data that “people buy animals a lot more than tractors and other inanimate objects.” The findings led the “FarmVille” team to more prominently feature animals in its online store, he says.

Interestingly, Wingfield reports there is considerable tension in the company between the data jockeys and the game designers. The game designers have a certain idea of how a game should look and function. The analysts drive the direction of game development based on the data, leading to tension. Some designers have quit the company in protest. Still, data remains the keystone in Zynga’s game plan for the foreseeable future.

The Zynga story on data analysis comes on the heels of the recent International Conference on the Foundations of Digital Games in Bordeaux this summer (fdg2011.org). There, Brent Harrison and David L. Roberts over at North Carolina State delivered an interesting paper, Using sequential observations to model and predict player behavior. Here’s their abstract:

In this paper, we present a data-driven technique for designing models of user behavior. Previously, player models were designed using user surveys, small-scale observation experiments, or knowledge engineering. These methods generally produced semantically meaningful models that were limited in their applicability. To address this, we have developed a purely data-driven methodology for generating player models based on past observations of other players. Our underlying assumption is that we can accurately predict what a player will do in a given situation if we examine enough data from former players that were in similar situations. We have chosen to test our method on achievement data from the MMORPG World of Warcraft. Experiments show that our method greatly outperforms a baseline algorithm in both precision and recall, proving that this method can create accurate player models based solely on observation data.

While not fixating on the profit motives that Zynga has in mind, Harrison and Roberts offer clues to game designers in guiding player behavior in-game. Educational games could become more engaging:

The ability to accurately predict a player’s behavior in a game has a number of applications. While these applications are beyond the scope of this paper, we discuss two of them briefly here to better situate and motivate our approach. With a model of player behavior, we can create an experience that is unique to a user’s tendencies or preferences. For example, if we predict that the user will choose to fight a certain non-player character (NPC) rather than talk to it, that NPC can be made more willing to fight. Another application involves guiding players to parts of games that they may enjoy. Modern games often take place in large, sandbox worlds where the player is given total freedom. It’s quite possible that players may never see content that they would like because the sandbox is just so big. Predictions about a player’s behavior can be used to guide her to the parts of the game that she would enjoy.

Eschewing surveys, the authors recommend a purely data-driven approach (as does Zynga):

We feel that a purely data-driven approach has significant promise for creating accurate predictive models of player behavior in games without the difficulties associated with earlier modeling techniques. Very little research has been done in this area to date.

Read the entire paper for further discussion of the algorithm they developed. Very interesting.

References:

Harrison, B & Roberts, D. L. (2011). Using sequential observations to model and predict player behavior. In Proceedings of the 2011 Foundations of Digital Games Conference. (FDG 2011), Bordeaux, France.

Wingfield, N. (2011, September 9). Virtual products, real profits. The Wall Street Journal, p.A1.
-

Study: Tough Times in RL Lead to Greater Second Life Satisfaction

An interesting study by Edward Castronova over at Indiana and Gert G. Wagner at Berlin University of Technology came out this summer in the social sciences journal Kyklos. Castronova and Wagner examined life satisfaction ratings from the 2005 World Values Survey and another survey of life satisfaction among Second Life players. Subjecting both sets of data to regression analysis showed correlations between difficult problems in real life leading to a more intense time online. Here’s their abstract:

We study life satisfaction data from the 2005 World Values Survey and a 2009 survey of users of the virtual world Second Life. Among Second Life users, satisfaction with their virtual life is higher than satisfaction with their real life. Regression analysis indicates that people in certain life situations, such as unemployment, gain more life satisfaction from “switching” to the virtual world than from changing their real-life circumstances. Thus, an unemployed person can become happier by visiting Second Life rather than finding a job. Correspondingly, problems in real life are positive predictors of intense use of virtual life.

It’s one of those “I could have told you that,” studies. The importance of the study is, now you can say “Research shows that people with real life difficulties tend to gain greater satisfaction in virtual worlds.” Castronova sums it nicely on the Terra Nova blog:

We are not finding any causal effects here, just correlations. What’s noteworthy is the magnitude of the correlations. Second Life is providing a big chunk of life satisfaction, just as big as the factors that previous researchers on life satisfaction have found were the “biggies,” like health, employment, and family relationships. (By the way, in case you didn’t know, money does not make you happy.)

He has a link to his copy of the study here. The official journal link is here.

References:

Castronova, E. & Wagner, G. E. (2011, August). Virtual life satisfaction. Kyklos  64(3). 313-328.
-

Center for Children and Technology Reports on DS Games at AERA

I’m honored to be invited to participate in a discussion group this fall put together by the Education Development Center’s Center for Children and Technology (EDC/CCT). The research this group is involved with in the field of classroom gaming is impressive.

The list of projects EDC/CCT is working on is extensive. Among many, one project with the U.S. Dept. of Education involves the design of educational game modules for the Nintendo DSi handheld, aimed at middle school science and literacy.

A paper by Marion Goldstein, Marian Pasquale, and Katie McMillan Culp, members of the Possible Worlds team at CCT, was presented recently at AERA 2011. Here is the abstract for the paper, entitled Using Students’ Naïve Theories to Design Games for Middle-Grades Science:

This paper reports on one phase of a long-term research and development project that is creating video game modules for middle-school science classrooms. The games are intended to help teachers address common scientific misconceptions by providing students with opportunities to interact with visualizations of otherwise abstract or inaccessible concepts or phenomena that are the source of those misconceptions. The visualizations serve as metaphors for natural phenomena, and linking activities help teachers build connections between the visualizations and the targeted concepts. Findings presented here are derived from formative research conducted to inform the development of a game and associated classroom materials that address genetics and heredity. The paper discusses how teachers in our sample typically teach this material in seventh grade, student expressions of common misconceptions about genetics and heredity, and how an initial design for the game responds to and addresses those misconceptions. Students’ misconceptions were associated with the concepts of randomness of inheritance, gene expression, and natural selection.

Perhaps the most impressive aspect of the team’s approach to instructional handheld gaming design is the commitment to research-based efforts. Through direct research with middle school students, the team uncovered several misconceptions held by the students through a series of experiments. When showed a mixed race couple, students’ assumptions regarding the physical makeup of the couple’s children were based on misconceptions. Other experiments uncovered faulty assumptions based on genetic adaptations of beetles and the random characteristics of lotteries. With this research in hand, the team set out to tackle common misconceptions among students at this age and grade level. The remainder of the paper discusses results with prototypes of the resulting game modules.

It’s an excellent report of a work in progress. Research and design such as this will ultimately result in stronger and more effective educational video games.