Can a brain-computer interface convert your thoughts to text?

Ever wonder what it would be like if a device could decode your thoughts into actual speech or written words? While this might enhance the capabilities of already existing speech interfaces with devices, it could be a potential game-changer for those with speech pathologies, and even more so for "locked-in" patients who lack any speech or motor function.

"So instead of saying 'Siri, what is the weather like today' or 'Ok Google, where can I go for lunch?' I just imagine saying these things," explains Christian Herff, author of a review recently published in the journal Frontiers in Human Neuroscience.

While reading one's thoughts might still belong to the realms of science fiction, scientists are already decoding speech from signals generated in our brains when we speak or listen to speech.

In their review, Herff and co-author, Dr. Tanja Schultz, compare the pros and cons of using various brain imaging techniques to capture neural signals from the brain and then decode them to text.

The technologies include functional MRI and near infrared imaging that can detect neural signals based on of neurons, to methods such as EEG and magnetoencephalography (MEG) that can detect electromagnetic activity of neurons responding to speech. One method in particular, called electrocorticography or ECoG, showed promise in Herff's study.

This study presents the Brain-to-text system in which epilepsy patients who already had electrode grids implanted for treatment of their condition participated. They read out texts presented on a screen in front of them while their was recorded. This formed the basis of a database of patterns of neural signals that could now be matched to speech elements or "phones".

When the researchers also included language and dictionary models in their algorithms, they were able to decode to text with a high degree of accuracy. "For the first time, we could show that brain activity can be decoded specifically enough to use ASR technology on ," says Herff. "However, the current need for implanted electrodes renders it far from usable in day-to-day life."

So, where does the field go from here to a functioning thought detection device? "A first milestone would be to actually decode imagined phrases from brain activity, but a lot of technical issues need to be solved for that," concedes Herff.

Their study results, while exciting, are still only a preliminary step towards this type of .

Explore further

Speech recognition from brain activity

More information: Christian Herff et al, Automatic Speech Recognition from Neural Signals: A Focused Review, Frontiers in Neuroscience (2016). DOI: 10.3389/fnins.2016.00429
Provided by Frontiers
Citation: Can a brain-computer interface convert your thoughts to text? (2016, October 25) retrieved 22 August 2019 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Feedback to editors

User comments

Oct 26, 2016
This comment has been removed by a moderator.

Oct 26, 2016
Only the very last step in the process, when we are actually forming words in preparation for speech, and the very first step, when we hear or see words, are thoughts purely word-like.

So if the disabled or locked in person tries to speak and stimulates that part of the brain then there is hope that this could work, but as for thoughts, forget it...we are not even close :)

Oct 26, 2016
Considering how hard it is for a person to translate their inner thoughts to words it seems highly unlikely that an interface could do it. It has been widely reported by writers, philosophers and others that one can have a perfectly formed thought in the mind and yet find it almost impossible or very difficult to write down the thought that was clear in the mind.

The most probable reason is that thoughts do not include all the words of a sentence but a combination of feelings and emotion with some words, so when we come to write it down we find the thought was not in a clear sentences as it seemed to be.

When you read text you have a reaction to it. You have all kinds of feelings and emotions in reaction to it. Composition is the reverse process. So the feeling can be there *as if* the stimulating words were there but they are not.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more