If videogames do affect pupils' education and skills, it's time we found out how

A study published recently examined the effects of video games on maths, reading and science skills and makes some interesting claims about the positive influence of the teenage participants' online gaming habits.

The study is remarkable for a number of reasons. It focuses on data drawn from the popular PISA tests, which examine school pupils' learning in real-world situations, rather than traditional educational attainment measured through exams and more abstract tests. Author Alberto Posso, of the Royal Melbourne Institute of Technology in Australia, uses the 2012 PISA data drawn from 12,018 pupils aged 15 from 772 schools – far larger than most studies. The PISA data also records details about the students' internet use, including the proportion used for study or for social networking and chat, and details such as parents' wealth, occupation and education.

Posso makes a distinction between the use of online social networks and online gaming habits, and takes into account factors from the family environment. This sets the study apart from most, which tend to conflate different forms of online activity, are based on small, scarcely representative groups, and neglect the role of these contextual factors.

Assessing the probability that the PISA scores were influenced by participation in social networks and playing online games, surprisingly the results showed that spending time on Facebook or chatting online with friends was associated with lower performance in maths, reading, and science, while those playing online games regularly achieved higher scores in the same subject areas.

The study is robust, and adds weight to the idea that video games can be beneficial to learning – although a good amount of caution is needed when drawing implications from the findings. To be fair, the author himself invites caution, concluding that the priority for politicians and teachers should remain to address the socio-economic factors that affect poor educational performance. For example, the educational gap between children from ethnic minorities and their peers, or school truancy in less privileged communities.

Nonetheless, it can't be ignored that there is mounting evidence to support the positive effects of playing games on a wide range of outcomes, such as learning, developing practical skills, and increased attention and motivation. Perhaps it's time to move beyond studies that only find associations, for example in this case between game playing and educational performance. Instead, we need to start examining the specifics of the games played in more detail, and explore what features may support or hinder learning.

Time to get detailed

Whether studies show that videogames improve or degrade performance, part of the problem is that we still don't know how or why. Most evidence from studies displays the familiar weaknesses of correlational research: that the presence or absence of a relation cannot be satisfactorily explained because there is no detailed causal model that explains how the games might cause the effects recorded.

This means that claiming that online gaming habits correlate positively with skills in maths, reading and science tells us very little. It provides little insight into how different types of video games, such as strategy games as opposed to shooters or role-playing games, require different types of game-play, and how these different styles of play are supposed to influence the educational, affective or behavioural outcomes claimed for them.

During almost 50 years of continuous innovation, video games include a wide variety of game types, mechanics and game-play styles – even among the more recent class of games that can be played online. With such a variety, there are bound to be games that deliver different, even mutually incompatible or antagonistic influences.

For example, a rough list of game-play mechanics in Overwatch, a 3-D first-person shooter that is played competitively online, includes shooting, melee combat, looting, healing during cooperative play, special ability management, and so on. Each of these is governed within the game by its own set of algorithmic rules. Each requires a learning curve for the player to use it properly, and each is tied to the others in a complex procedural system that balances difficulty and rewards to keep players playing and coming back for more.

Any analysis of whether playing Overwatch has a positive or negative effect on behaviour and skills should take into account the role of these different mechanics and styles of play, evaluating the possibility that the way in which a game is played (more aggressively or more cooperatively, for example) rather than the game in itself may result in different effects on the player.

Similarly, it's hard to ignore how the mindless repetition of simple tasks, the grind required by many games to build up the "experience points" that upgrade a player character's strengths or abilities, relies on exploitative compulsion loops. These aim to keep players hooked, and may not encourage any form of meaningful learning – although it may be effective as a rather cynical form of behavioural control.

The same logic applies to most games. Without taking a more close-up, granular look at the game mechanics and their relationships to players' behaviours and skills, we're left making largely "black box" assumptions: we see only what goes in and what comes out, but have no understanding of why or how. Inevitably, this means that we may be underestimating the impact of other factors that have nothing to do with games themselves, but are an expression of the social and cultural environments in which a particular game is designed and played.

This story is published courtesy of The Conversation (under Creative Commons-Attribution/No derivatives).