App will teach infants with cochlear implants to speak

A team of Swinburne researchers is developing an app to teach infants with cochlear implants how to speak.

GetTalking is designed for infants who are born deaf and receive cochlear implants. These infants must learn how to associate the sounds they can newly hear with the sounds they are able to make with their mouths.

The project was started by Swinburne's Dr Belinda Barnet after her own experience raising her hearing impaired daughter.

"With my own daughter – she had an implant at 11 months old – I could afford to take a year off to teach her to talk. This involved lots of repetitive exercises and time. Now that she can talk I'd like to help other families who may not have that time," Dr Barnet says.

Rewarding speech

Being developed to run on iPads or other tablets, GetTalking gives infants a bright visual reward for speaking.

"When a child has not heard any sound, they don't understand that a noise has an effect on the environment. So the first thing has to be a visual reward for an articulation," Dr Barnet says.

"At 12 months children respond well to visual rewards – and even an 'ahhh' or 'ohhh' should get a response from the app."

After the app recognises any kind of speech from the infant it then has to recognise what word the child was trying to say, rewarding them for speaking words and approximations of words.

As understanding a baby that doesn't know how to form words is exceptionally complicated, complex speech recognition software and artificial intelligence is needed.

"That's quite difficult. The speech needs to be cross-matched with thousands of articulations from normally-speaking babies," Dr Barnet says.

Swinburne's BabyLab is supporting GetTalking in this area with a large collection of speech samples.

GetTalking responds to the infant's word-approximation by re-articulating the correct word and showing a picture of what they are saying.

Building a team

Swinburne Department Chair of Health and Medical Sciences, Associate Professor Rachael McDonald, became involved with GetTalking as Dr Barnet was seeking someone with experience in occupational therapy and child development.

"I had just started at Swinburne and was very excited about the prospect of developing real technology that had the opportunity to really change people's lives," Associate Professor McDonald says.

"Dr Barnet and I gathered a team of experts to work on this project, in order to not only develop the app but to ensure that it is engaging for children and families and that it has an evidence base, so that it can be used in clinical settings."

Associate Professor McDonald hopes that the app will improve the lives of children and families who need to do repetitive speech activities.

"Rather than replacing, we are augmenting the experience, which will improve adherence to activities that often children and families struggle to do."