Study shows audio experience in virtual reality can be 'authentic'

Experts, as part of the University of York's XR Stories project, have proposed a new sound strategy to developing virtual reality (VR) environments, based on how people receive images and sounds from the real world, ultimately improving user experience and reducing the risk of VR motion sickness effects.

Virtual reality applications—experienced in the home environment—use a variety of techniques to convey sound; some use a 'movie' approach, where the sound is outside of the interactions in the VR world, others use a single object focus for music, such as a radio, and some match sound with environmental interactions, such as the sound of an object falling on the floor.

Most often, however, VR uses a mixture of audio methods—some sounds originating from an object within the world and other sounds appearing to be imagined or outside of the virtual world.

Disconnected

This approach comes with some issues, however, which can contribute to the feeling of being disconnected to the virtual world that has been created, or in some cases can create a motion sickness feeling, whereby the cues sent from the user's brain to their eyes and ears don't quite match-up.

If sound does not mirror the everyday audio experience- a bird singing should get louder when the VR user gets closer to it for example—or if music is overlaid with no apparent source or reason for the sound, it can result in an inauthentic, confusing experience.

Research at the University of York proposed a new sound design strategy based on objects that would naturally produce sound in the 'real world'; this means the whole audio environment in VR would be exclusively built around objects that produce sound in a specific position in space, mirroring how people experience sound in real time.

Responsive

Constantin Popp, Research Associate on the XR Stories project at the University of York's AudioLab, says that "the difference from previous methods is that we aim to apply this thinking to all sounds and music that are part of an experience, not just some parts."

"This thinking allows us to make each sound producing object interactive and responsive to the user, which in doing so improves the user's experience. In this way we can also re-use data existing in the VR game, such as an object's velocity, and apply it to sound."

"For example, when a user drops an object, the game would play a matching sound indicative of how fast and where the object fell. This strategy improves believability and narrative depth."

Everyday world

The researchers added, however, that this methodology requires more computer processing power in the VR headset than exists with many current approaches, and would also increase the developmental phase of VR and the overall cost, so more work may be needed to make this process quicker and less expensive.

Professor Damain Murphy, from the University of York and Director of XR Stories, says that they "believe that in applying this method we can make VR experiences feel more 'real' as it increases the responsiveness of the environment to be more in line with that of our everyday world."

"Better audio-visual design also reduces the risk of the user feeling 'spaced out' or suffering the effects of motion sickness—a common issue for some people in VR—by allowing the eyes and ears to sync-up with the images that the brain is receiving."

"This approach to audio in VR can provide a more unified, natural, and authentic experience."

The research was published in Applied Sciences.