Virtual reality users must learn to use what they see

December 4, 2017 by Chris Barncard, University of Wisconsin-Madison
Credit: CC0 Public Domain

Anyone with normal vision knows that a ball that seems to quickly be growing larger is probably going to hit them on the nose.

But strap them into a headset, and they still may need to take a few lumps before they pay attention to the visual cues that work so well in the real world, according to a new study from University of Wisconsin-Madison psychologists.

"The companies leading the virtual reality revolution have solved major engineering challenges—how do you build a small headset that does a good job presenting images of a ," says Bas Rokers, UW-Madison psychology professor. "But they have not thought as much about how the brain processes these images. How do people perceive a virtual world?"

Turns out, they don't perceive it like the real world—at least not without training, according a study Rokers and postdoctoral psychology researcher Jacqueline Fulvio published recently in the journal Nature Scientific Reports.

In 2015, Fulvio found that people were flunking her simple test of three-dimensional perception using a flat screen and standard 3D movie glasses. They were not good at discerning which direction a target was moving.

"Most importantly, they confused whether the object was coming toward them or going away from them," she says. "It was a surprising finding. Nobody believed it, because it's not something that happens often in the real world. You'd get hurt."

The researchers decided to move the test to virtual reality to provide more realistic indications of motion in three dimensions—such as binocular cues, in which slightly different views from the left and right eye reveal depth, and parallax, where closer objects appear to be moving faster than those farther away.

"We thought it was as easy as taking the same object-tracking task, putting it in the virtual environment, and having people do it the same way," Fulvio says. "And they did do it the same way. They made the same mistakes."

Given a one-second snippet of the movement of a small, round target across a plane that stretched away from the viewer at roughly eye level, study participants correctly moved a virtual paddle to intercept the target's course less than a quarter of the time.

What Fulvio and Rokers found was that when most people put on a , they still treat what they see like it's happening on any run-of-the-mill TV screen.

"There's no depth to a computer screen. There are no binocular cues. Close one eye, close the other eye, nothing changes," says Rokers, whose work was funded by Google. "If you take that expectation into a VR headset, where you do have binocular cues, you somehow just don't use them."

Unless you're trained to use those cues.

Fulvio began giving study subjects visual and audible feedback. Once they'd watched the one-second flight and set their virtual paddle to catch the target, the game would reveal the full path of the target and a cowbell noise for success or swish for a miss.

The visual feedback nearly doubled success rates. (The cowbell improved scores, too, but less so.)

"They were getting better, but how were they getting better? What were they doing differently?" Fulvio asked.

When she turned off the VR system's head tracking, taking away the effects of players' head movements and making them passive viewers, they were bad again. When she gave a little of that freedom back—restoring the system's response to head movements, but making the virtual world shifts lag behind players by as much as half a second—they were still bad.

Interestingly, even players who reported keeping their heads stock still showed improvements when the virtual reality system was incorporating the smallest wobbles of their heads into the scene they were seeing.

"These are head motions people make, tiny jitters, that are not planned movements," Rokers says. "When you think you're sitting still, your head is moving a little bit. And, it turns out, people actually use that information to improve depth perception. It's tiny. It's almost involuntary. But the visual system actually exploits that."

The results—that tiny movements and typical binocular cues of motion are there for the taking in virtual reality, but that most people only use them if they are actively shown how VR differs from a flat computer screen—should help virtual reality creators improve uptake of their products.

"Google packages a virtual reality YouTube viewer with their headset. That's a passive experience, and not the best thing to do," Rokers says. "What they should be doing is packaging action games with their headset, something that forces users to interact with the environment. That teaches them to use the information available in virtual reality, and treat it more like the and less like a computer screen."

"Otherwise you just have a really fancy TV, really close to your face," says Fulvio, who has moved on to testing the extent to which people's expectations influence their perception of flat versus virtual depth by having her study subjects watch TV inside virtual reality.

Rokers says showing the effects of teaching people to use cues to three-dimensional motion that they are otherwise ignoring may ultimately help refine treatment for vision disorders such as blind spots or amblyopia ("lazy eye") in which the brain can be trained to compensate for perceptual limitations.

Explore further: Virtual reality makes its best users the most queasy

Related Stories

Virtual reality makes its best users the most queasy

January 29, 2016
In a twist of virtual fate, people with the best 3-D vision are also the people most likely to suffer from motion sickness while using virtual reality displays.

Virtual reality training may be as effective as regular therapy after stroke

November 15, 2017
Using virtual reality therapy to improve arm and hand movement after a stroke is equally as effective as regular therapy, according to a study published in the November 15, 2017, online issue of Neurology, the medical journal ...

Some brains are blind to moving objects

September 28, 2016
As many as half of people are blind to motion in some part of their field of vision, but the deficit doesn't have anything to do with the eyes.

Recommended for you

Neuroimaging study reveals 'hot spot' for cue-reactivity in substance-dependent population

November 20, 2018
When patients with dependence on alcohol, cocaine or nicotine are shown drug cues, or images related to the substance, an area of their brain known as the medial prefrontal cortex (mPFC) shows increased activity, report investigators ...

Being fair: The benefits of early child education

November 20, 2018
Children from low-income families who got intensive education early in life treat others with high levels of fairness in midlife, more than 40 years later, even when being fair comes at a high personal cost, according to ...

Does Netflix's '13 Reasons Why' influence teen suicide? Survey asks at-risk youths

November 20, 2018
A significant proportion of suicidal teens treated in a psychiatric emergency department said that watching the Netflix series "13 Reasons Why" had increased their suicide risk, a University of Michigan study finds.

MDMA makes people cooperative, but not gullible

November 19, 2018
New research from King's College London has found that MDMA, the main ingredient in ecstasy, causes people to cooperate better—but only with trustworthy people. In the first study to look in detail at how MDMA impacts cooperative ...

Exploring the genetic contribution to suicide risk

November 19, 2018
Researchers at University of Utah Health identified four gene changes that occur more frequently in people who died by suicide that may point to increased risk in vulnerable individuals.

Response to daily stressors could affect brain health in older adults

November 19, 2018
Taking typical daily annoyances such as a long wait at the doctor's office or a traffic jam on the freeway in stride may help preserve brain health in older adults, while emotional reactions could contribute to declines in ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.