Flaws found in video game studies

September 20, 2011 by Lisa Zyga, Medical Xpress weblog

(Medical Xpress) -- Over the past several years, many studies have found that people who regularly play action video games outperform people who don’t on tasks that involve perception and cognition. However, a new study has found that most, if not all, of these studies suffer from common pitfalls in experimental design, so that the results of the studies are undermined by methodological shortcomings.

The new study notes that, although game training could prove very useful if it works, it is actually in conflict with the typical findings of cognitive training research, which shows that training in one task rarely improves performance in other tasks. In this light, the possibility that video game training causes “broad transfer” of skills beyond gaming itself would be unusual, but potentially very constructive.

In their review article published in Frontiers of Psychology, Walter Boot and Daniel Blakely from Florida State University and Daniel Simons from the University of Illinois have discussed the methodological shortcomings of previous video game studies and offer guidelines for improved testing methods.

First, one of the biggest problems with video game studies was the recruiting process. In many studies, participants knew they were being recruited due to their gaming expertise, which could make them expect to perform well, and the expectation could in turn influence their performance. In contrast, non-gamer participants who were selected without gaming expertise would not experience the same motivation. This recruiting process violates a core principle of experimental design, which Boot, Blakely, and Simons suggest could be improved by covert recruiting. For instance, participants could be asked about their video game experience only at the end of the study or in a prescreening that is not linked to the particular experiment. Still, the review authors note that comparing gamers and non-gamers does not account for the possibility that people may become action gamers because they have the abilities required to excel at these games, or that a third factor might influence both gaming and cognitive abilities.

In other studies, all the participants are non-gamers and they are assigned to either action game training (the experimental group) or non-action game training (the control group). Participants’ performance on tasks involving perception and cognition is measured before and after their training. The problem with this set-up is that participants know which training they’re receiving and whether or not it should improve their performance on the cognitive tasks. Similar to the recruiting problem, participants trained on action video games may expect their performance to improve as a result of their training, while participants in the control group may have no such expectation. Boot, Blakely, and Simons suggest that participants should be asked whether they perceived a connection between their training and the tasks, and also whether they’re aware of past studies that have shown the benefits of gaming on cognitive performance.

The authors of the review also note that the results of some of these studies seem to conflict with one of the most fundamental principles of learning, which is that performance improves with practice. One would expect that who trained on non-action video games should improve at least a bit the second time they perform the cognitive tests, but many studies have found a lack of improvement. This unexpected lack of improvement may give the appearance of a greater benefit of action video game training in comparison. Interestingly, studies that do find an improvement in the control group often find no significant effect of video game training on cognitive performance. As the review authors suggest, in order to make claims about the benefits of video game training, the should perform as expected.

Finally, because video game training studies are costly and time-consuming, the results are often split up and published in multiple journal articles, even though they use the same training groups. Boot, Blakely, and Simons recommend that, when multiple results of a single experiment are reported in different articles, the connection should be clearly stated to avoid confusion and publication bias.

Overall, the review authors emphasize that their criticism does not mean that the results of all previous studies are wrong, just that their experimental methods are not compelling enough to draw strong conclusions. They emphasize that adopting a set of clinical trial best practices could illuminate the true benefits of gaming for cognitive performance. If future studies with improved methods can confirm that action video games do improve perception and cognition, then game training holds great promise for people with vision and attention disorder, as well as for remediating the effects of cognitive aging.

Explore further: Put down that Xbox remote: Researcher suggests video games may not boost cognition

More information: Walter R. Boot, et al. "Do action video games improve perception and cognition?" Frontiers in Psychology. DOI:10.3389/fpsyg.2011.00226

Related Stories

Put down that Xbox remote: Researcher suggests video games may not boost cognition

September 15, 2011
Wouldn't it be nice if all those hours kids spent glued to their PlayStation 3, Xbox 360 or Nintendo DS video games actually resulted in something tangible? Better grades, perhaps? Improved concentration? Superior driving ...

Recommended for you

Study of learning and memory problems in OCD helps young people unlock potential at school

January 22, 2018
Adolescents with obsessive-compulsive disorder (OCD) have widespread learning and memory problems, according to research published today. The findings have already been used to assist adolescents with OCD obtain the help ...

Intensive behavior therapy no better than conventional support in treating teenagers with antisocial behavior

January 19, 2018
Research led by UCL has found that intensive and costly multisystemic therapy is no better than conventional therapy in treating teenagers with moderate to severe antisocial behaviour.

Babies' babbling betters brains, language

January 18, 2018
Babies are adept at getting what they need - including an education. New research shows that babies organize mothers' verbal responses, which promotes more effective language instruction, and infant babbling is the key.

College branding makes beer more salient to underage students

January 18, 2018
In recent years, major beer companies have tried to capitalize on the salience of students' university affiliations, unveiling marketing campaigns and products—such as "fan cans," store displays, and billboard ads—that ...

Inherited IQ can increase in early childhood

January 18, 2018
When it comes to intelligence, environment and education matter – more than we think.

Modulating molecules: Study shows oxytocin helps the brain to modulate social signals

January 17, 2018
Between sights, sounds, smells and other senses, the brain is flooded with stimuli on a moment-to-moment basis. How can it sort through the flood of information to decide what is important and what can be relegated to the ...


Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Sep 20, 2011
Hrmph, I think the most interesting point raised by this article is the question of whether cognitive skills increase as a result of gaming or do people with better than average cognitive skills gravitate towards gaming more?, I'm a fan of twitch games because I find them engaging after a day of cognitively demanding programming work.
not rated yet Sep 20, 2011
" First, one of the biggest problems with video game studies was the recruiting process. In many studies, participants knew they were being recruited due to their gaming expertise, which could make them expect to perform well, and the expectation could in turn influence their performance. "

They should have taken a clue from the movie " The Last Starfighter ".

Enough from me, I'm off to fight the Ko-Dan Armada..
4 / 5 (1) Sep 21, 2011
I would suspect a common cause for both proficiencies (video games and perception/cognition tasks), probably with a strong neurological basis. An inherent aptitude would undoubtedly encourage the development of such proficiencies.
not rated yet Sep 24, 2011
Kudos Lisa.
Not all is lost - here is your comfort:


Helps establish what is to be understood under 'learning'.
The physical and theoretical basis is well established under a rubric one can label as:
The Nature of Sound.

not rated yet Sep 26, 2011
I also find there is an issue I find when the results of a study is released to the public. With the case of "action video games", you are looking at a very broad range of play styles, interaction, and thinking which is required. Some people adapt to particular playstyles better then others. I have a feeling they are only looking at first person shooter type games, but I canoot tell if they are also looking at real-time stratagies which are totally different.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.