Brain scanner that senses when you're going into information overload

February 5, 2014 by Michael Blanding

Picture an air-traffic controller tracking 10 planes approaching an airport. Now imagine he's having trouble focusing on all 10 aircraft, perhaps because he's been up all night or just has a lot on his mind. What would happen if his computer sensed his mental fatigue, removed one plane from his oversight and reassigned it to a controller who just started her shift?

The scenario might seem like science fiction, but with new technology being developed by Tufts researchers Robert Jacob and Sergio Fantini, it could be quite real someday. Jacob and Fantini have developed a brain-scanning device that allows a computer to sense the level of mental exertion of its user and adjust tasks accordingly to achieve the correct balance between boredom and overload.

"Humans and computers are two powerful information processors connected by this miserably narrow bandwidth—a mouse and a keyboard," says Jacob, a professor of computer science in the School of Engineering. Jacob's challenge is to find ways to create a more direct connection between machine and human brain to make both more efficient.

Eight years ago, he walked into the lab of Fantini, a professor of biomedical engineering at Tufts, and saw him experimenting with a technique called diffuse optical spectroscopy, which uses light to peer inside the body. Initially employed in the 1920s to examine cancerous tissue, it more recently has been used to look at brain function. "We illuminate the tissue, in this case brain tissue, with a laser diode," says Fantini. "Because the light diffuses, some of it will travel to the brain cortex and make it back."

In the video above, Rob Jacob and Sergio Fantini talk about their work on the mind-reading computer. Video: Steffan Hacker

The amount of light that is absorbed depends on how much oxygen is circulating through the arteries and veins bringing blood to and from the brain. When a particular part of the brain works hard, it consumes oxygen more rapidly, sending a signal to the body to dilate the arteries and bring more oxygen-rich blood to that area. At the same time, the light that returns to the scanner is red or infrared in color, since that is the only wavelength that can penetrate the skull. By measuring the change in color of the diffused light, researchers can get a sense of how much mental effort a person is applying to a task.

"When I found out Sergio had a new technique for measuring what is happening in your brain, I said, yes! We can use this as an additional channel of communicating information" between humans and computers, says Jacob. While many researchers studying brain-computer interaction focus on creating devices to help disabled people, Jacob and Fantini want to develop new interfaces for the average user.

It's an important distinction. In the case of disabled people, researchers work with direct control, in which users consciously think of a motion or command in order to perform a particular task. Jacob and Fantini, however, are working with passive control, in which a computer senses incidental brain activity and automatically adjusts its function, like lightening the load of the exhausted air traffic controller. No explicit user action required.

Brain Activity

To achieve that passive control, the two researchers created a device—a sports headband that holds a collection of tiny fiber optic cables tight against a user's forehead—that sends beams of near-infrared light through the skull and about three centimeters into the brain. Fantini compares it to the way your fingers take on a reddish glow when you hold a flashlight to them. (For a later iteration of the sensing device, Jacob and Fantini collaborated with costume designers in Tufts' drama department to design a custom headband.)

It's a far cry from other brain-scanning techniques, such as functional magnetic resonance imaging (fMRI), which requires users to lie perfectly still inside a giant machine. With the headband scanner, you can sit and use a computer in an ordinary office setting.

Jacob hastens to add that the device does not read the content of thoughts—only the level of brain activity. "Of course we might be able to measure that in the future, who knows," he adds. "Then another argument comes into play, which is the question of whether it's your personal device and it's helping you, or whether your employer is using it to monitor you."

Such questions are far in the future, however. At this point, the researchers are conceiving of the device only as one that will help users maximize their brain energy and productivity.

Never Idle

To test their invention, Jacob and Fantini initially experimented with simple memory tasks, measuring the increase in blood flow to the part of the cerebral cortex dealing with attention and short-term memory. That turned out to be quite difficult. Blood flow to the brain changes with every motion of the body—and even fluctuates when you're sitting still. "Even when you sleep, the brain is not idle, so it's never just a flat signal," says Fantini.

In order to account for this, the researchers let the computer program do the work—relying on a technique called "machine learning," in which the computer examines incoming data and makes new calculations based on past experience. "We start with known examples of high and low workload, and have it come up with an answer," says Jacob.

After calibrating the device, they developed several simulations that allowed the computer to directly communicate with the brain, and set up studies along with graduate students to test them. The first was an airplane simulation similar to the air-traffic controller juggling too many incoming flights. Study participants wearing the headband were told to guide several planes to their proper destinations. When the computer recognized that a person's attention was slipping, it would surreptitiously remove one of the planes. When it saw that a person was getting bored, it would add an airplane to keep him or her engaged.

By optimizing the number of airplanes, the researchers found that they could improve participants' performance in correctly guiding planes to their destinations. "It's the kind of thing that if we had a knob that said, 'Now I'm feeling busy; now I'm feeling not so busy,' we might do ourselves, but that would be very distracting," says Jacob. It would also be less effective.

The brain scanner makes the computer a better judge of when we are getting tired or distracted. In fact, Jacob and Fantini worked hard to make the changes in their simulation so subtle that participants hardly noticed when a plane appeared or disappeared on the screen.

They are developing additional applications for similar situations in which a person might have varying levels of attention for a task. One is a special cursor that grows larger or smaller in order to select items as you use a mouse to drag it across a screen. By measuring the attention span of the user, it can better anticipate what item he or she is trying to select.

Prioritizing Our Work

"When we are overworked, we overemphasize the high-priority topics, making the low-priority topics harder to get to," says Jacob. The same principle, he says, could be adapted to web browsing or word processing, to give you more information when you have more mental capacity to process it, and less when your brain needs a rest.

The researchers have also been working with Google to create a more streamlined version of the device. In the future, it could possibly be integrated into Google Glass—a small wearable computer that fits onto a pair of eyeglasses—in order to tweak the screen display that viewers see, giving them more or less information depending on their brain activity.

Currently, the monitoring unit that the headband is hooked up to is the size of a stereo receiver. It would require some significant engineering to shrink the device to a size that would make it truly transportable, but the researchers say it is possible. "The brain scanner is just a little light and a sensor," says Jacob. "It's easy to imagine a relatively cheap device that we can stick up against your forehead." Graduate student Chirag Sthalekar is working on a new interface for the device in the form of a circuit board only a couple of inches on each side.

At the same time, the current device is limited to measuring only through skin on the forehead. The lab is developing a sensor that could work around hair more easily to measure parts of the controlling other functions and experimenting with other body sensors to measure attributes such as heart rate and galvanic skin response (small electrical charges in the skin due to stress or emotion). One day, our computers may know more about our state of mind than we do—and help us be more productive.

Explore further: Doctoral dissertation studies the use of light in measuring cerebral circulation

Related Stories

Recommended for you

Study identifies how brain connects memories across time

May 23, 2016

Using a miniature microscope that opens a window into the brain, UCLA neuroscientists have identified in mice how the brain links different memories over time. While aging weakens these connections, the team devised a way ...

The brain needs cleaning to stay healthy

May 26, 2016

Research led by the Achucarro Basque Center for Neuroscience, the University of the Basque Country (UPV/EHU), and the Ikerbasque Foundation has revealed the mechanisms that keep the brain clean during neurodegenerative diseases.

Neuroscientists illuminate role of autism-linked gene

May 25, 2016

A new study from MIT neuroscientists reveals that a gene mutation associated with autism plays a critical role in the formation and maturation of synapses—the connections that allow neurons to communicate with each other.

Teen brains facilitate recovery from traumatic memories

May 25, 2016

Unique connections in the adolescent brain make it possible to easily diminish fear memories and avoid anxiety later in life, according to a new study by Weill Cornell Medicine researchers. The findings may have important ...

Mimicking deep sleep brain activity improves memory

May 26, 2016

It is not surprising that a good night's sleep improves our ability to remember what we learned during the day. Now, researchers at the RIKEN Brain Science Institute in Japan have discovered a brain circuit that governs how ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.