Brain scanner that senses when you're going into information overload

February 5, 2014 by Michael Blanding, Tufts University

Picture an air-traffic controller tracking 10 planes approaching an airport. Now imagine he's having trouble focusing on all 10 aircraft, perhaps because he's been up all night or just has a lot on his mind. What would happen if his computer sensed his mental fatigue, removed one plane from his oversight and reassigned it to a controller who just started her shift?

The scenario might seem like science fiction, but with new technology being developed by Tufts researchers Robert Jacob and Sergio Fantini, it could be quite real someday. Jacob and Fantini have developed a brain-scanning device that allows a computer to sense the level of mental exertion of its user and adjust tasks accordingly to achieve the correct balance between boredom and overload.

"Humans and computers are two powerful information processors connected by this miserably narrow bandwidth—a mouse and a keyboard," says Jacob, a professor of computer science in the School of Engineering. Jacob's challenge is to find ways to create a more direct connection between machine and human brain to make both more efficient.

Eight years ago, he walked into the lab of Fantini, a professor of biomedical engineering at Tufts, and saw him experimenting with a technique called diffuse optical spectroscopy, which uses light to peer inside the body. Initially employed in the 1920s to examine cancerous tissue, it more recently has been used to look at brain function. "We illuminate the tissue, in this case brain tissue, with a laser diode," says Fantini. "Because the light diffuses, some of it will travel to the brain cortex and make it back."

In the video above, Rob Jacob and Sergio Fantini talk about their work on the mind-reading computer. Video: Steffan Hacker

The amount of light that is absorbed depends on how much oxygen is circulating through the arteries and veins bringing blood to and from the brain. When a particular part of the brain works hard, it consumes oxygen more rapidly, sending a signal to the body to dilate the arteries and bring more oxygen-rich blood to that area. At the same time, the light that returns to the scanner is red or infrared in color, since that is the only wavelength that can penetrate the skull. By measuring the change in color of the diffused light, researchers can get a sense of how much mental effort a person is applying to a task.

"When I found out Sergio had a new technique for measuring what is happening in your brain, I said, yes! We can use this as an additional channel of communicating information" between humans and computers, says Jacob. While many researchers studying brain-computer interaction focus on creating devices to help disabled people, Jacob and Fantini want to develop new interfaces for the average user.

It's an important distinction. In the case of disabled people, researchers work with direct control, in which users consciously think of a motion or command in order to perform a particular task. Jacob and Fantini, however, are working with passive control, in which a computer senses incidental brain activity and automatically adjusts its function, like lightening the load of the exhausted air traffic controller. No explicit user action required.

Brain Activity

To achieve that passive control, the two researchers created a device—a sports headband that holds a collection of tiny fiber optic cables tight against a user's forehead—that sends beams of near-infrared light through the skull and about three centimeters into the brain. Fantini compares it to the way your fingers take on a reddish glow when you hold a flashlight to them. (For a later iteration of the sensing device, Jacob and Fantini collaborated with costume designers in Tufts' drama department to design a custom headband.)

It's a far cry from other brain-scanning techniques, such as functional magnetic resonance imaging (fMRI), which requires users to lie perfectly still inside a giant machine. With the headband scanner, you can sit and use a computer in an ordinary office setting.

Jacob hastens to add that the device does not read the content of thoughts—only the level of brain activity. "Of course we might be able to measure that in the future, who knows," he adds. "Then another argument comes into play, which is the question of whether it's your personal device and it's helping you, or whether your employer is using it to monitor you."

Such questions are far in the future, however. At this point, the researchers are conceiving of the device only as one that will help users maximize their brain energy and productivity.

Never Idle

To test their invention, Jacob and Fantini initially experimented with simple memory tasks, measuring the increase in blood flow to the part of the cerebral cortex dealing with attention and short-term memory. That turned out to be quite difficult. Blood flow to the brain changes with every motion of the body—and even fluctuates when you're sitting still. "Even when you sleep, the brain is not idle, so it's never just a flat signal," says Fantini.

In order to account for this, the researchers let the computer program do the work—relying on a technique called "machine learning," in which the computer examines incoming data and makes new calculations based on past experience. "We start with known examples of high and low workload, and have it come up with an answer," says Jacob.

After calibrating the device, they developed several simulations that allowed the computer to directly communicate with the brain, and set up studies along with graduate students to test them. The first was an airplane simulation similar to the air-traffic controller juggling too many incoming flights. Study participants wearing the headband were told to guide several planes to their proper destinations. When the computer recognized that a person's attention was slipping, it would surreptitiously remove one of the planes. When it saw that a person was getting bored, it would add an airplane to keep him or her engaged.

By optimizing the number of airplanes, the researchers found that they could improve participants' performance in correctly guiding planes to their destinations. "It's the kind of thing that if we had a knob that said, 'Now I'm feeling busy; now I'm feeling not so busy,' we might do ourselves, but that would be very distracting," says Jacob. It would also be less effective.

The brain scanner makes the computer a better judge of when we are getting tired or distracted. In fact, Jacob and Fantini worked hard to make the changes in their simulation so subtle that participants hardly noticed when a plane appeared or disappeared on the screen.

They are developing additional applications for similar situations in which a person might have varying levels of attention for a task. One is a special cursor that grows larger or smaller in order to select items as you use a mouse to drag it across a screen. By measuring the attention span of the user, it can better anticipate what item he or she is trying to select.

Prioritizing Our Work

"When we are overworked, we overemphasize the high-priority topics, making the low-priority topics harder to get to," says Jacob. The same principle, he says, could be adapted to web browsing or word processing, to give you more information when you have more mental capacity to process it, and less when your brain needs a rest.

The researchers have also been working with Google to create a more streamlined version of the device. In the future, it could possibly be integrated into Google Glass—a small wearable computer that fits onto a pair of eyeglasses—in order to tweak the screen display that viewers see, giving them more or less information depending on their brain activity.

Currently, the monitoring unit that the headband is hooked up to is the size of a stereo receiver. It would require some significant engineering to shrink the device to a size that would make it truly transportable, but the researchers say it is possible. "The brain scanner is just a little light and a sensor," says Jacob. "It's easy to imagine a relatively cheap device that we can stick up against your forehead." Graduate student Chirag Sthalekar is working on a new interface for the device in the form of a circuit board only a couple of inches on each side.

At the same time, the current device is limited to measuring only through skin on the forehead. The lab is developing a sensor that could work around hair more easily to measure parts of the controlling other functions and experimenting with other body sensors to measure attributes such as heart rate and galvanic skin response (small electrical charges in the skin due to stress or emotion). One day, our computers may know more about our state of mind than we do—and help us be more productive.

Explore further: Tech gets in your head, literally, to ease stress

Related Stories

Tech gets in your head, literally, to ease stress

January 10, 2014
Yes, technology can get into your head, says the maker of a new brain-sensing headband that promises to reduced stress.

New brain-scanning technique allows scientists to see when and where the brain processes visual information

January 27, 2014
Every time you open your eyes, visual information flows into your brain, which interprets what you're seeing. Now, for the first time, MIT neuroscientists have noninvasively mapped this flow of information in the human brain ...

Doctoral dissertation studies the use of light in measuring cerebral circulation

April 12, 2013
Tiina Näsi, a researcher of biomedical engineering at Aalto University, studied in her doctoral thesis the use of light in measuring the brain's blood circulation. This optical measurement may in the future help discover ...

Neural prosthesis restores brain function after injury (w/ Video)

December 9, 2013
Scientists from Case Western Reserve University and University of Kansas Medical Center have restored behavior—in this case, the ability to reach through a narrow opening and grasp food—using a neural prosthesis in a ...

Recommended for you

Miles Davis is not Mozart: The brains of jazz and classical pianists work differently

January 16, 2018
Keith Jarret, world-famous jazz pianist, once answered in an interview when asked if he would ever be interested in doing a concert where he would play both jazz and classical music: "No, that's hilarious. [...] It's like ...

Researchers identify protein involved in cocaine addiction

January 16, 2018
Mount Sinai researchers have identified a protein produced by the immune system—granulocyte-colony stimulating factor (G-CSF)—that could be responsible for the development of cocaine addiction.

New study reveals why some people are more creative than others

January 16, 2018
Creativity is often defined as the ability to come up with new and useful ideas. Like intelligence, it can be considered a trait that everyone – not just creative "geniuses" like Picasso and Steve Jobs – possesses in ...

Neuroscientists suggest a model for how we gain volitional control of what we hold in our minds

January 16, 2018
Working memory is a sort of "mental sketchpad" that allows you to accomplish everyday tasks such as calling in your hungry family's takeout order and finding the bathroom you were just told "will be the third door on the ...

Brain imaging predicts language learning in deaf children

January 15, 2018
In a new international collaborative study between The Chinese University of Hong Kong and Ann & Robert H. Lurie Children's Hospital of Chicago, researchers created a machine learning algorithm that uses brain scans to predict ...

Preterm babies may suffer setbacks in auditory brain development, speech

January 15, 2018
Preterm babies born early in the third trimester of pregnancy are likely to experience delays in the development of the auditory cortex, a brain region essential to hearing and understanding sound, a new study reveals. Such ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.