Google Glass helps kids with autism read facial expressions, study finds

August 2, 2018, Stanford University Medical Center

Children with autism were able to improve their social skills by using a smartphone app paired with Google Glass to help them understand the emotions conveyed in people's facial expressions, according to a pilot study by researchers at the Stanford University School of Medicine.

Prior to participating in the study, Alex, 9, found it overwhelming to look people in the eye. Gentle encouragement from his mother, Donji Cullenbine, hadn't helped. "I would smile and say things like, 'You looked at me three times today!' But it didn't really move the bar," she said. Using Google Glass transformed how Alex felt about looking at faces, Cullenbine said. "It was a game environment in which he wanted to win—he wanted to guess right—and he got an instant reward when he did."

The therapy, described in findings to be published online Aug. 2 in npj Digital Medicine, uses a Stanford-designed app that provides real-time cues about other people's facial expressions to a child wearing Google Glass. The device, which was linked with a smartphone through a local wireless network, consists of a glasses-like frame equipped with a camera to record the wearer's field of view, as well as a small screen and a speaker to give the wearer visual and audio information. As the child interacts with others, the app identifies and names their emotions through the Google Glass speaker or screen. After one to three months of regular use, parents reported that children with autism made more eye contact and related better to others.

The treatment could help fill a major gap in autism care: Right now, because of a shortage of trained therapists, children may wait as long as 18 months after an autism diagnosis to begin receiving treatment.

'Really important unmet need'

"We have too few autism practitioners," said the study's senior author, Dennis Wall, Ph.D., associate professor of pediatrics and of biomedical data science. Early autism therapy has been shown to be particularly effective, but many children aren't treated quickly enough to get the maximum benefit, he said. "The only way to break through the problem is to create reliable, home-based treatment systems. It's a really important unmet need."

Autism is a developmental disorder that affects 1 in 59 children in the United States, with a higher prevalence in boys. It is characterized by social and communication deficits and repetitive behaviors.

The researchers named the new therapy "Superpower Glass" to help make it appealing to children. The therapy is based on applied behavior analysis, a well-studied autism treatment in which a clinician teaches emotion recognition using structured exercises such as flash cards depicting faces with different emotions. Although traditional applied behavior analysis helps children with autism, it has limitations: It must be delivered one-on-one by trained therapists, flash cards can't always capture the full range of human emotion and children may struggle to transfer what they learn to their daily lives.

Eight core facial expressions

Wall's team decided to try using applied behavior analysis principles in a way that would bring parents and everyday situations into the treatment process. They built a that uses machine learning to recognize eight core : happiness, sadness, anger, disgust, surprise, fear, neutral and contempt. The app was trained with hundreds of thousands of photos of faces showing the eight expressions, and also had a mechanism to allow people involved in the study to calibrate it to their own "neutral" faces if necessary.

Typically developing children learn to recognize emotions by engaging with people around them. For children with autism, it's different. "They don't pick those things up without focused treatment," Wall said.

In the study, 14 families tested the Superpower Glass setup at home for an average of 10 weeks each. Each family had a child between the ages of 3 and 17 with a clinically confirmed .

The families used the therapy for at least three 20-minute sessions per week. At the start and end of the study, parents completed questionnaires to provide detailed information about their child's . In interviews, parents and children also gave feedback about how the program worked for their families.

The researchers designed three ways to use the face-recognizing program: In "free play," children wear Google Glass while interacting or playing with their families, and the software provides the wearer with a visual or auditory cue each time it recognizes an emotion on the face of someone in the field of view. There are also two game modes. In "guess my emotion," a parent acts out a facial expression corresponding to one of the eight core emotions, and the child tries to identify it. The game helps families and researchers track children's improvement at identifying emotions. In "capture the smile," children give another person clues about the emotion they want to elicit, until the other person acts it out, which helps the researchers gauge the children's understanding of different emotions.

Good reviews from families

Families told the researchers that the system was engaging, useful and fun. Kids were willing to wear the Google Glass, and the devices withstood the wear and tear of being used by children.

Twelve of the 14 families, including Alex's, said their children made more eye contact after receiving the treatment. A few weeks into the trial, Alex began to realize that people's faces hold clues to their feelings.

"He told me, 'Mommy, I can read minds!'" Cullenbine said. "My heart sang. I'd like other parents to have the same experience."

Families whose children had more severe autism were more likely to choose the game modes rather than free play, the researchers reported.

The children's mean score on the SRS-2, a questionnaire completed by parents to evaluate children's social skills, decreased by 7.38 points during the study, indicating less severe symptoms of autism. None of the participants' SRS-2 scores increased during the study, meaning nobody's autism symptoms worsened. Six of the 14 participants had large enough declines in their scores to move down one step in the severity of their autism classification: four from "severe" to "moderate," one from "moderate" to "mild" and one from "mild" to "normal."

The results should be interpreted with caution since the study did not have a control arm, Wall said. However, the findings are promising, he added.

Parents' comments in interviews helped illustrate the improvements, he said. "Parents said things like 'A switch has been flipped; my child is looking at me.' Or 'Suddenly the teacher is telling me that my child is engaging in the classroom.' It was really heartwarming and super-encouraging for us to hear," Wall said.

His team is currently completing a larger, randomized trial of the therapy. In addition, they also plan to test the therapy in who have just been diagnosed with and are on a waiting list for treatment. Stanford University has filed a patent application for the technology.

Explore further: Study shows cognitive behavioural therapy can improve emotion regulation in children with autism

More information: Jena Daniels et al. Exploratory study examining the at-home feasibility of a wearable tool for social-affective learning in children with autism, npj Digital Medicine (2018). DOI: 10.1038/s41746-018-0035-3 , www.nature.com/articles/s41746-018-0035-3

Related Stories

Study shows cognitive behavioural therapy can improve emotion regulation in children with autism

April 24, 2018
New research from York University's Faculty of Health shows cognitive behavioural therapy (CBT) can help children with autism manage not only anxiety but other emotional challenges, such as sadness and anger.

Meet Nao, the robot that helps treat kids with autism

May 9, 2018
(HealthDay)—It may seem counterintuitive, but a robot might help kids with autism interact better with humans.

Criticism from parents affects how children's brains respond to emotional information

June 11, 2018
Children of highly critical parents show less attention to emotional facial expressions, according to new research from Binghamton University, State University at New York.

Therapy for kids with autism pays off for moms, dads

August 11, 2017
(HealthDay)—Behavioral therapy for children with autism also benefits their parents, a new study finds.

Mobile app for autism screening yields useful data

June 1, 2018
A Duke study of an iPhone app to screen young children for signs of autism has found that the app is easy to use, welcomed by caregivers and good at producing reliable scientific data.

Recommended for you

Autism linked to egg cells' difficulty creating large proteins

August 16, 2018
New work from Carnegie's Ethan Greenblatt and Allan Spradling reveals that the genetic factors underlying fragile X syndrome, and potentially other autism-related disorders, stem from defects in the cell's ability to create ...

First biomarker evidence of DDT-autism link

August 16, 2018
A study of more than 1 million pregnancies in Finland reports that elevated levels of a metabolite of the banned insecticide DDT in the blood of pregnant women are linked to increased risk for autism in the offspring. An ...

Discovery of a key protein involved in the development of autism

August 16, 2018
Most individuals with autism spectrum disorder cannot be distinguished by physical traits or by severe neurological symptoms. In fact, these cases can be identified only on the basis of certain behaviours, namely their obsessive ...

People with autism may not have trouble focusing on people in photos

August 16, 2018
While people with autism may avoid eye contact in one-on-one conversations, they may not avoid looking at people in photos, according to Penn State researchers.

Mizzou program significantly reduces delay in autism diagnosis

August 15, 2018
When Katie New first suspected her son had autism, she had to wait 18 months for a diagnosis. She also had to travel nearly 100 miles from her hometown of Poplar Bluff, Missouri, to see an autism specialist in Cape Girardeau. ...

Ouija board study highlights ineffective treatment for autism

August 7, 2018
A new study into Ouija board practices revealed how the planchette (or a glass, as is often used) seemingly moves without any of the individual players being aware that they are doing it.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.