Findings could lead to improved lip-reading training for the deaf and hard-of-hearing

September 10, 2009
A new study by the University of East Anglia suggests computers are now better at lip-reading than humans. Credit: Sarah Hilder/UEA

A new study by the University of East Anglia suggests computers are now better at lip-reading than humans.

The peer-reviewed findings will be presented for the first time at the eighth International Conference on Auditory-Visual Speech Processing (AVSP) 2009, held at the University of East Anglia from September 10-13.

A research team from the School of Computing Sciences at UEA compared the performance of a machine-based lip-reading system with that of 19 human lip-readers. They found that the automated system significantly outperformed the human lip-readers - scoring a recognition rate of 80 per cent, compared with only 32 per cent for human viewers on the same task.

Furthermore, they found that machines are able to exploit very simplistic features that represent only the shape of the face, whereas human lip-readers require full video of people speaking.

The study also showed that rather than the traditional approach to lip-reading training, in which viewers are taught to spot key lip-shapes from static (often drawn) images, the dynamics and the full appearance of speech gestures are very important.

Using a new video-based training system, viewers with very limited training significantly improved their ability to lip-read monosyllabic words, which in itself is a very difficult task. It is hoped this research might lead to novel methods of lip-reading training for the deaf and hard of hearing.

"This pilot study is the first time an automated lip-reading system has been benchmarked against human lip-readers and the results are perhaps surprising," said the study's lead author Sarah Hilder.

"With just four hours of training it helped them improve their lip-reading skills markedly. We hope this research will represent a real technological advance for the deaf community."

Agnes Hoctor, campaigns manager at the RNID, said: "This research confirms how difficult the vital skill of lip-reading is to learn and why RNID is campaigning for people who are deaf or hard of hearing to have improved access to classes. We would welcome the development of video-based or online training resources to supplement the teaching of lip-reading. Hearing loss affects 55 per cent of people over 60 so, with the ageing population, demand to learn lip-reading is only going to increase."

The AVSP conference is being held in the UK for the first time since its inception in 1998. The University of East Anglia will host cutting edge researchers including psychologists, engineers, scientists and linguists from as far afield as Australia, Canada and Japan.

Source: University of East Anglia

Related Stories

Recommended for you

Sugar industry withheld evidence of sucrose's health effects nearly 50 years ago

November 21, 2017
A U.S. sugar industry trade group appears to have pulled the plug on a study that was producing animal evidence linking sucrose to disease nearly 50 years ago, researchers argue in a paper publishing on November 21 in the ...

Female researchers pay more attention to sex and gender in medicine

November 7, 2017
When women participate in a medical research paper, that research is more likely to take into account the differences between the way men and women react to diseases and treatments, according to a new study by Stanford researchers.

Drug therapy from lethal bacteria could reduce kidney transplant rejection

August 3, 2017
An experimental treatment derived from a potentially deadly microorganism may provide lifesaving help for kidney transplant patients, according to an international study led by investigators at Cedars-Sinai.

Exploring the potential of human echolocation

June 25, 2017
People who are visually impaired will often use a cane to feel out their surroundings. With training and practice, people can learn to use the pitch, loudness and timbre of echoes from the cane or other sounds to navigate ...

Team eradicates hepatitis C in 10 patients following lifesaving transplants from infected donors

April 30, 2017
Ten patients at Penn Medicine have been cured of the Hepatitis C virus (HCV) following lifesaving kidney transplants from deceased donors who were infected with the disease. The findings point to new strategies for increasing ...

'bench to bedside to bench': Scientists call for closer basic-clinical collaborations

March 24, 2017
In the era of genome sequencing, it's time to update the old "bench-to-bedside" shorthand for how basic research discoveries inform clinical practice, researchers from The Jackson Laboratory (JAX), National Human Genome Research ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

RayCherry
not rated yet Sep 10, 2009
Finally. :-)

Soon we will have a pair of glasses with stereo cameras with an iPod like device for image processing and builtin head phones, permitting the hard-of-hearing to receive a real assistance - filtering the desired voice from the background.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.