New research on the neural details of 'swiping'

While observing fellow train passengers eagerly swiping their touch screens, neuroscientist Sara Fabbri got the idea to explore the neurocomputational basis of the swiping, tapping and pinching movements that people make when interacting with touch screen devices. For the next two years, she will work at the Donders Institute of Radboud University in the framework of the Radboud Excellence Initiative.

Sara Fabbri's research expertise is how the brain plans and executes reaching movements that allow us to interact with objects. At the Donders Institute for Brain, Cognition and Behaviour in Nijmegen, she will be the first to explore a completely new area of research: studying how the brain copes with movements people make while operating a touch screen.

'Most of the time, swiping movements don't have a clear goal', Fabbri explains regarding the special nature of these movements. 'You might be swiping towards something that is not visible on your screen yet, for instance when using a city map. Or you could be rotating a virtual 3D object like a globe by making movements on your touch screen in 2D. In the real world, there is a direct match between your perception of a target and the action that you take to reach that target. When using a touch screen, there sometimes is a mismatch between those pieces of information. The space on your screen is arbitrary, it does not necessarily relate to the real world. But your brain solves this mismatch automatically most of the time: working with touch screens is often intuitive. During my time at the Donders Institute, I want to find out how our brains represent the space we perceive on a touch screen and how we make plans to act upon that perception.'

Touch screens in the fMRI scanner

Touch screen devices are increasingly becoming part of our everyday life. However, Fabbri's research proposal constitutes the first attempt to find out how our brain deals with using them. One of her first projects in Nijmegen will be to compare how test subjects use city maps in real life and on touch screen devices. At this point in time, Fabbri's biggest challenge is a technical one: 'There are no commercial touch screens available yet that we can use inside the fMRI scanner, since they are all partly made out of metal. So first we have to find other ways to let people interact with touch screens while they are inside the scanner.'

Future collaborations

Fabbri aims to set up several collaborations with researchers at Radboud University, for instance to develop computational models and to investigate a wide range of test subjects: from babies to the elderly. 'You can imagine that the brain of a teenager who is growing up surrounded by touch screens has a different way of handling these devices than the brain of somebody that only starting using them when they were fifty years old', she explains. With the knowledge she will gain, more user-friendly touch screens can be made. 'I'm very excited to get started and most of all very curious about the results. It's very exciting to work on something that nobody has ever investigated before!'