Virtual reality at the service of psychology

December 7, 2017, Goethe University Frankfurt am Main
A picture from the SCEGRAM database which shows objects in unusual places. Melissa Vo's research group is investigating how the brain reacts. Credit: Goethe University Frankfurt am Main

Our environment is composed according to certain rules and characteristics that are so obvious that people are scarcely aware of them. Professor Melissa Le-Hoa Vo, psychologist at Goethe University Frankfurt, is studying this "scene knowledge" and other topics of visual cognition in a virtual reality laboratory. In the current issue of the Forschung Frankfurt research journal, journalist Jessica Klapp reports on her virtual trip to Italy and explains why people don't look for the milk under the bed or for our pillow in the bathtub.

"When we search for a specific object in a scene, we seem to have developed a precise idea of where to look for and find certain things," explains Melissa Vo. What she finds particularly interesting in her investigation of these naturalistic scenes is how people perceive the environment. In which circumstances are people especially attentive? And what will they remember later? To find this out, Professor Võ uses eye tracking and virtual reality scenarios in her laboratory, alongside measurement of brain potentials.

"We use eye tracking to measure which parts of a picture the observer finds interesting or important, how fast her gaze settles on specific objects in a scene and how long it dwells there," explains Dr. Dejan Draschkow, a member of Professor Vo's research group.

There is a particularly close relationship between eye movement and cognitive processes. The video-based systems used by the researchers register eye movements with the help of a camera. Head-mounted systems that resemble a pair of spectacles are paired with remote eye trackers installed in a computer monitor, along with a camera and infrared LEDs. With the mobile system, the test participants can move around in the room, search for objects and interact with them.

With the headset, the computer simulates a virtual 3-D world in which the test subject moves about. With simulated settings, such as an Italian piazza in the centre of which brown boxes are unexpectedly floating, the researchers check whether the results identified on two-dimensional monitors are also valid in a realistic, three-dimensional environment. They want to understand which rules people use to compose their environment and interact with the objects in it.

Investigating scene knowledge in children is one of the topics that the research group is tackling in particular depth. The aim of the project is the early detection and treatment of possible cognitive disorders such as dyslexia. Tests are conducted in a mobile laboratory right outside the nursery school: The researchers show children "ungrammatical" pictures, for example, a shoe on the cooker instead of a saucepan, and watch their reactions with the help of an eye-tracking camera. If one of the children in a large group behaves differently, they are interested to see whether there is a correlation between language development and attention behaviour.

Other fields, such as medicine, also benefit from the results. For example, the researchers measured the eye movements of radiologists when studying X-rays and analysed which strategies they use to recognize tumours and how successful these strategies are. The research results are also important for the security control of hand baggage at airports. How do staff decide which baggage needs a more thorough check? Why wasn't a dangerous found?

And finally, people with dementia could also profit from scene knowledge research, since Vo and her team have discovered that the ability to recall pictures in a scene increases if the test subjects have previously looked for and found individual objects. In a surprising memory test, they performed far better than participants who were supposed to memorize objects explicitly. "What this means for us is that in the case of a visual search, there is more interaction with the , and the participants commit objects better to memory," explains Professor Vo.

Explore further: Virtual reality users must learn to use what they see

Related Stories

Virtual reality users must learn to use what they see

December 4, 2017
Anyone with normal vision knows that a ball that seems to quickly be growing larger is probably going to hit them on the nose.

Researchers uncover our brain's filing system for storing experiences

September 27, 2017
A team of neuroscientists has uncovered how our brains organize, over time, our experiences: that is, according to their similarities.

Researchers compare the performance of human subjects versus deep neural networks in visual searches

September 27, 2017
Before you read on, look for toothbrushes in the photo above. Find them? Both of them? If you're like the vast majority of people, you honed in on the one near the sink, but probably took a moment or two before seeing the ...

Overturning widely held ideas: Visual attention drawn to meaning, not what stands out

September 25, 2017
Our visual attention is drawn to parts of a scene that have meaning, rather than to those that are salient or "stick out," according to new research from the Center for Mind and Brain at the University of California, Davis. ...

Recommended for you

Study: Tinder loving cheaters—dating app facilitates infidelity

February 23, 2018
The popular dating app Tinder is all about helping people form new relationships. But for many college-aged people, it's also helping those in relationships cheat on their romantic partners.

The 'loudness' of our thoughts affects how we judge external sounds

February 23, 2018
The "loudness" of our thoughts—or how we imagine saying something—influences how we judge the loudness of real, external sounds, a team of researchers from NYU Shanghai and NYU has found.

Looking for the origins of schizophrenia

February 23, 2018
Schizophrenia may be related to neurodevelopmental changes, including brain's inability to generate an appropriate vascular system, according to new study resulted from a partnership between the D"Or Institute for Research ...

Color of judo uniform has no effect on winning

February 22, 2018
New research on competitive judo data finds a winning bias for the athlete who is first called, regardless of the colour of their uniform. This unique study, published in Frontiers in Psychology, puts to rest the debate on ...

Infants are able to learn abstract rules visually

February 22, 2018
Three-month-old babies cannot sit up or roll over, yet they are already capable of learning patterns from simply looking at the world around them, according to a recent Northwestern University study published in PLOS One.

Antidepressants are more effective than placebo at treating acute depression in adults, concludes study

February 22, 2018
Meta-analysis of 522 trials includes the largest amount of unpublished data to date, and finds that antidepressants are more effective than placebo for short-term treatment of acute depression in adults.

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

RobertKarlStonjek
not rated yet Dec 07, 2017
...explains why people don't look for the milk under the bed or for our pillow in the bathtub.


The author has obviously never seen a college student's dorm or apartment ...

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.