Flaws found in video game studies

by Lisa Zyga weblog

(Medical Xpress) -- Over the past several years, many studies have found that people who regularly play action video games outperform people who don’t on tasks that involve perception and cognition. However, a new study has found that most, if not all, of these studies suffer from common pitfalls in experimental design, so that the results of the studies are undermined by methodological shortcomings.

The new study notes that, although game training could prove very useful if it works, it is actually in conflict with the typical findings of cognitive training research, which shows that training in one task rarely improves performance in other tasks. In this light, the possibility that video game training causes “broad transfer” of skills beyond gaming itself would be unusual, but potentially very constructive.

In their review article published in Frontiers of Psychology, Walter Boot and Daniel Blakely from Florida State University and Daniel Simons from the University of Illinois have discussed the methodological shortcomings of previous video game studies and offer guidelines for improved testing methods.

First, one of the biggest problems with video game studies was the recruiting process. In many studies, participants knew they were being recruited due to their gaming expertise, which could make them expect to perform well, and the expectation could in turn influence their performance. In contrast, non-gamer participants who were selected without gaming expertise would not experience the same motivation. This recruiting process violates a core principle of experimental design, which Boot, Blakely, and Simons suggest could be improved by covert recruiting. For instance, participants could be asked about their video game experience only at the end of the study or in a prescreening that is not linked to the particular experiment. Still, the review authors note that comparing gamers and non-gamers does not account for the possibility that people may become action gamers because they have the abilities required to excel at these games, or that a third factor might influence both gaming and cognitive abilities.

In other studies, all the participants are non-gamers and they are assigned to either action game training (the experimental group) or non-action game training (the control group). Participants’ performance on tasks involving perception and cognition is measured before and after their training. The problem with this set-up is that participants know which training they’re receiving and whether or not it should improve their performance on the cognitive tasks. Similar to the recruiting problem, participants trained on action video games may expect their performance to improve as a result of their training, while participants in the control group may have no such expectation. Boot, Blakely, and Simons suggest that participants should be asked whether they perceived a connection between their training and the tasks, and also whether they’re aware of past studies that have shown the benefits of gaming on cognitive performance.

The authors of the review also note that the results of some of these studies seem to conflict with one of the most fundamental principles of learning, which is that performance improves with practice. One would expect that who trained on non-action video games should improve at least a bit the second time they perform the cognitive tests, but many studies have found a lack of improvement. This unexpected lack of improvement may give the appearance of a greater benefit of action video game training in comparison. Interestingly, studies that do find an improvement in the control group often find no significant effect of video game training on cognitive performance. As the review authors suggest, in order to make claims about the benefits of video game training, the should perform as expected.

Finally, because video game training studies are costly and time-consuming, the results are often split up and published in multiple journal articles, even though they use the same training groups. Boot, Blakely, and Simons recommend that, when multiple results of a single experiment are reported in different articles, the connection should be clearly stated to avoid confusion and publication bias.

Overall, the review authors emphasize that their criticism does not mean that the results of all previous studies are wrong, just that their experimental methods are not compelling enough to draw strong conclusions. They emphasize that adopting a set of clinical trial best practices could illuminate the true benefits of gaming for cognitive performance. If future studies with improved methods can confirm that action video games do improve perception and cognition, then game training holds great promise for people with vision and attention disorder, as well as for remediating the effects of cognitive aging.

More information: Walter R. Boot, et al. "Do action video games improve perception and cognition?" Frontiers in Psychology. DOI:10.3389/fpsyg.2011.00226

Related Stories

Video games shown to improve vision

Mar 15, 2007

According to a new study from the University of Rochester, playing action video games sharpens vision. In tests of visual acuity that assess the ability to see objects accurately in a cluttered space, game players scored ...

How video games stretch the limits of our visual attention

Nov 18, 2010

They are often accused of being distracting, but recent research has found that action packed video games like Halo and Call of Duty can enhance visual attention, the ability that allows us to focus on relevant visual information. ...

Racing, shooting and zapping your way to better visual skills

Dec 17, 2009

Do your kids want a Wii, a PlayStation or an Xbox 360 this year? This holiday gift season is packed with popular gaming systems and adrenaline-pumping, sharpshooting games. What's a parent to do? Is there any redeeming value ...

Action video games improve vision

Mar 29, 2009

Video games that involve high levels of action, such as first-person-shooter games, increase a player's real-world vision, according to research in today's Nature Neuroscience.

Recommended for you

Report advocates improved police training

Aug 29, 2014

A new report released yesterday by the Mental Health Commission of Canada identifies ways to improve the mental health training and education that police personnel receive.

Meaningful relationships can help you thrive

Aug 29, 2014

Deep and meaningful relationships play a vital role in overall well-being. Past research has shown that individuals with supportive and rewarding relationships have better mental health, higher levels of subjective well-being ...

Learning to read involves tricking the brain

Aug 29, 2014

While reading, children and adults alike must avoid confusing mirror-image letters (like b/d or p/q). Why is it difficult to differentiate these letters? When learning to read, our brain must be able to inhibit ...

User comments

Adjust slider to filter visible comments by rank

Display comments: newest first

gmurphy
not rated yet Sep 20, 2011
Hrmph, I think the most interesting point raised by this article is the question of whether cognitive skills increase as a result of gaming or do people with better than average cognitive skills gravitate towards gaming more?, I'm a fan of twitch games because I find them engaging after a day of cognitively demanding programming work.
Isaacsname
not rated yet Sep 20, 2011
" First, one of the biggest problems with video game studies was the recruiting process. In many studies, participants knew they were being recruited due to their gaming expertise, which could make them expect to perform well, and the expectation could in turn influence their performance. "

They should have taken a clue from the movie " The Last Starfighter ".

Enough from me, I'm off to fight the Ko-Dan Armada..
tadchem
4 / 5 (1) Sep 21, 2011
I would suspect a common cause for both proficiencies (video games and perception/cognition tasks), probably with a strong neurological basis. An inherent aptitude would undoubtedly encourage the development of such proficiencies.
hush1
not rated yet Sep 24, 2011
Kudos Lisa.
Not all is lost - here is your comfort:

http://www.physor...ode.html

Helps establish what is to be understood under 'learning'.
The physical and theoretical basis is well established under a rubric one can label as:
The Nature of Sound.

JackAdler
not rated yet Sep 26, 2011
I also find there is an issue I find when the results of a study is released to the public. With the case of "action video games", you are looking at a very broad range of play styles, interaction, and thinking which is required. Some people adapt to particular playstyles better then others. I have a feeling they are only looking at first person shooter type games, but I canoot tell if they are also looking at real-time stratagies which are totally different.