Movies and music play with our minds by bending time
October 25, 2017
by Shaoni Bhattacharya, From Horizon Magazine, Horizon: The EU Research & Innovation Magazine
Movie directors regularly manipulate the passage of time in films to entertain their audiences, but researchers are trying to unravel the effect this can have on our brains.
Little is known about how the human brain processes time or how slow motion, time lapse and even music can alter our perception of how quickly it passes, changing our emotions and cognitive abilities.
Professor Clemens Wöllner, a cognitive musicology researcher at the University of Hamburg in Germany, describes an iconic scene in the 1990s blockbuster film Forrest Gump as an example.
As the kind-hearted but hapless hero, then a child, is being chased by bullies on bikes, he runs jerkily before picking up speed and the leg braces that constrain him pop off.
"It's a very emotional scene—and of course it's in slow motion," said Prof. Wöllner, who is the principal investigator on an EU-funded project called SloMo.
By using slow motion, usually in combination with emotionally charged music, movie directors intuitively tap into how our perception of time affects our cognition, according to Prof. Wöllner.
The way the mind perceives time is something that researchers are only beginning to study at a fundamental level, though one day the findings from such work could feed into other research on human diseases and wellbeing.
The SloMo project is aiming to explore the theory that emotional processing by the mind is linked to the perception of time.
"The theory is that during highly emotional moments we are highly susceptible to information and take in more information," said Prof. Wöllner. "Our brain is more alert and so we have the feeling that time passes more slowly."
The team is examining the use of slow motion in music, dance, performing arts and audiovisual media like film or sports footage.
The project, which started in April and is funded for five years through the EU's European Research Council (ERC), comprises six main studies and an initial total of 15 experiments.
The researchers have already presented initial findings at the European Society for the Cognitive Sciences of Music Conference, which was held at Ghent University, Belgium, during the summer. The team showed healthy volunteers slow-motion scenes taken from movies, dance and sports footage, recording their emotional, physiological and eye-movement responses. They then recorded the same parameters in the viewers while speeding up the footage to a real-life pace.
The iconic Forrest Gump scene was one of the pieces shown. Prof. Wöllner said two key emotional dimensions, which psychologists call arousal and valence, were markedly different when people watched the original slow-motion version compared with the normal-speed version.
Arousal refers to the emotional state connected to alertness and internal activity, including physiological parameters like heart rate, while valence refers to the value of an emotion, often either as positive for feelings like happiness, or negative for feelings like anger or fear.
When subjects were shown the Forrest Gump scene in real time, their arousal state was higher, while valence was higher in the slow-motion version when music was played. The team also found that people's eye movements differed.
"In slow motion, the gaze behaviour of the eye was more dispersed," said Prof. Wöllner. "They looked at other things (in the scene)." He added that in real time, viewers tended to focus more, although not exclusively, on the main character.
In other words, slow motion enabled viewers to perceive details they otherwise wouldn't have noticed.
But other experiments within the project are also probing how music can be used to influence our perception of motion and time. Music compositions are often multi-layered with different levels of structure in both the melody and rhythm, to which listeners can synchronise.
In one experiment, the researchers are manipulating musical excerpts with different structural levels and will examine the volunteers' judgements of duration, along with physical responses like the speed the participants tapped their fingers to the perceived pulse of the music.
From a learning point of view, slow speed is important in music practice and dance rehearsal, so the project may have implications for education. The theory is that, in slowing down, the brain's cognitive load—or mental capacity—is reduced, making it easier to concentrate on new notes or information.
Prof. Wöllner said the project will also examine 'time dilation'—a phenomenon that music can also be used to alter. The idea is that repetition makes time appear to speed up, which is why time seems to go faster as we age, or why the first time we travel to a new place the journey seems long but the return leg seems faster.
Repetition is central to most musical genres and affects listeners' expectations. Prof. Wöllner says it is possible to alter some of the musical features and study how this affects people's perceptions of duration and behavioural responses to music.
He believes this ability to use music to change people's perception of time could also bring benefits to their wellbeing, although he stresses that SloMo is focusing on fundamental, basic research.
"One field certainly is how music may help in diseases with a very strong temporal component, for instance Parkinson's," he said.
But it may be important to answer some of the basic questions about how the brain is able to encode time before looking at how this can be applied to helping treat diseases.
The EU-funded Brain in Time (BiT) project, is looking at how our brains map duration. A lot of research in recent years has focused on where in the brain time is encoded, says Professor Domenica Bueti, principal investigator on the project and head of the time perception lab at the International School for Advanced Studies (SISSA) in Trieste, Italy.
"What I proposed is, to go from "where" to "how" and "when,"" she said. "How do brain regions encode time? What are the connections between these regions? How do they talk?"
Her project aims to test the hypothesis that there is a topography or map of time within the brain, something she refers to as a 'chronotopy." It will look at how the brain encodes time in the short term, at split-second level.
Prof. Bueti believes they will find that specific groups of brain cells respond selectively with different millisecond time durations.
She and her colleagues will use neuroimaging techniques, such as functional magnetic resonance imaging, to study the brains of healthy volunteers as they perform different tasks.
The team will also use a technique called transcranial magnetic stimulation to create virtual lesions within the brain while subjects are performing tasks. This can temporarily interfere with signals in the brain, allowing the researchers to study what different parts involved in time cognition do.
The project started in October 2016 with a five-year grant from the ERC.
Prof. Bueti hopes that chronomaps could provide a neurological signature of whether the brain's time cognition system is working properly or not. This could be used to help people with conditions that have time perception problems.
People with damage to specific brain regions like the basal ganglia or cerebellum, or those with certain degenerative diseases like Parkinson's can experience difficulties in judging time on the short scale. Understanding how the brain deals with time cognition could help develop new diagnostic tools for temporal deficiencies like these.
"Time is so abstract," said Prof. Bueti. "Yet, it's a fundamental dimension of everything."
Provided by Horizon: The EU Research & Innovation Magazine