Decoder uses fMRI brain scans to reconstruct human thoughts

Researchers at the University of Texas at Austin have developed a decoder that uses information from fMRI scans to reconstruct human thoughts. Jerry Tang, Amanda LeBel, Shailee Jain and Alexander Huth have published a paper describing their work on the preprint server bioRxiv.

Prior efforts to create technology that can monitor brain waves and decode them to reconstruct a person's thoughts have all consisted of probes placed in the brains of willing patients. And while such technology has proven useful for research efforts, it is not practical for use in other applications such as helping people who have lost the ability to speak. In this new effort, the researchers have expanded on work from prior studies by applying findings about reading and interpreting brain waves to data obtained from fMRI scans.

Recognizing that attempting to reconstruct brainwaves into individual words using fMRI was impractical, the researchers designed a decoding device that sought to gain an overall understanding of what was going on in the mind rather than a word-for-word decoding. The decoder they built was a computer algorithm that accepted fMRI data and returned paragraphs describing general thoughts. To train their algorithm, the researchers asked two men and one woman to lie in an fMRI machine while they listened to podcasts and recordings of people telling stories.

Over time, each volunteer listened to 16 hours of voice recordings that covered a wide range of topics. The decoder algorithm analyzed data from the fMRI machines, crunched what it was given and used what it learned to generate predictors that represented textual ideas. The team then used the predictors (guesses) made by the decoder to try to reconstruct what was going on in the mind of the subject based on new sets of fMRI data. To create an accuracy measure, they developed a scoring system and used it to measure the accuracy of text generated by the decoder compared to the fMRI data that was generated originally. The researchers describe their decoder as working "pretty well" in matching brainwave activity and the content of the original story.

The researchers also note that their technology could not be used to read a person's mind without their knowledge or consent because training of the decoder must be done first using fMRI data from the people whose thoughts are to be decoded.