Scientists help explain visual system's remarkable ability to recognize complex objects

July 2, 2013, Salk Institute
From left: This is John Reynolds, Anirvan Nandy and Tatyana Sharpee. Credit: Salk Institute for Biological Studies

How is it possible for a human eye to figure out letters that are twisted and looped in crazy directions, like those in the little security test internet users are often given on websites?

It seems easy to us—the just does it. But the apparent simplicity of this task is an illusion. The task is actually so complex, no one has been able to write computer code that translates these distorted letters the same way that neural networks can. That's why this test, called a CAPTCHA, is used to distinguish a human response from computer bots that try to steal sensitive information.

Now, a team of at the Salk Institute for Biological Studies has taken on the challenge of exploring how the brain accomplishes this remarkable task. Two studies published within days of each other demonstrate how complex a visual task decoding a CAPTCHA, or any image made of simple and intricate elements, actually is to the brain.

The findings of the two studies, published June 19 in Neuron and June 24 in the Proceedings of the National Academy of Sciences (PNAS), take two important steps forward in understanding vision, and rewrite what was believed to be established science. The results show that what neuroscientists thought they knew about one piece of the puzzle was too simple to be true.

Their deep and detailed research—involving recordings from hundreds of neurons—may also have future clinical and practical implications, says the study's senior co-authors, Salk neuroscientists Tatyana Sharpee and John Reynolds.

"Understanding how the brain creates a can help humans whose brains are malfunctioning in various different ways—such as people who have lost the ability to see," says Sharpee, an associate professor in the Computational Neurobiology Laboratory. "One way of solving that problem is to figure out how the brain—not the eye, but the cortex— processes information about the world. If you have that code then you can directly stimulate neurons in the cortex and allow people to see."

Reynolds, a professor in the Systems Neurobiology Laboratory, says an indirect benefit of understanding the way the brain works is the possibility of building computer systems that can act like humans.

"The reason that machines are limited in their capacity to recognize things in the world around us is that we don't really understand how the brain does it as well as it does," he says.

The scientists emphasize that these are long-term goals that they are striving to reach, a step at a time.

Integrating parts into wholes

In these studies, Salk neurobiologists sought to figure out how a part of the visual cortex known as area V4 is able to distinguish between different visual stimuli even as the stimuli move around in space. V4 is responsible for an intermediate step in neural processing of images.

"Neurons in the visual system are sensitive to regions of space— they are like little windows into the world," says Reynolds. "In the earliest stages of processing, these windows —known as receptive fields—are small. They only have access to information within a restricted region of space. Each of these neurons sends brain signals that encode the contents of a little region of space—they respond to tiny, simple elements of an object such as edge oriented in space, or a little patch of color."

Neurons in V4 have a larger that can also compute more complex shapes such as contours. They accomplishes this by integrating inputs from earlier visual areas in the cortex—that is, areas nearer the retina, which provides the input to the visual system, which have small receptive fields, and sends on that information for higher level processing that allow us to see complex images, such as faces, he says.

Both new studies investigated the issue of translation invariance— the ability of a neuron to recognize the same stimulus within its receptive field no matter where it is in space, where it happens to fall within the receptive field.

The Neuron paper looked at translation invariance by analyzing the response of 93 individual neurons in V4 to images of lines and shapes like curves, while the PNAS study looked at responses of V4 neurons to natural scenes full of complex contours.

Dogma in the field is that V4 neurons all exhibit translation invariance.

"The accepted understanding is that individuals neurons are tuned to recognize the same stimulus no matter where it was in their receptive field," says Sharpee.

For example, a neuron might respond to a bit of the curve in the number 5 in a CAPTCHA image, no matter how the 5 is situated within its receptive field. Researchers believed that neuronal translation invariance—the ability to recognize any stimulus, no matter where it is in space—increases as an image moves up through the visual processing hierarchy.

"But what both studies show is that there is more to the story," she says. "There is a trade off between the complexity of the stimulus and the degree to which the cell can recognize it as it moves from place to place."

A deeper mystery to be solved

The Salk researchers found that neurons that respond to more complicated shapes—like the curve in 5 or in a rock— demonstrated decreased translation invariance. "They need that complicated curve to be in a more restricted range for them to detect it and understand its meaning," Reynolds says. "Cells that prefer that complex shape don't yet have the capacity to recognize that shape everywhere."

On the other hand, neurons in V4 tuned to recognize simpler shapes, like a straight line in the number 5, have increased translation invariance. "They don't care where the stimuli they are tuned to is, as long as it is within their receptive field," Sharpee says.

"Previous studies of object recognition have assumed that neuronal responses at later stages in visual processing remain the same regardless of basic visual transformations to the object's image. Our study highlights where this assumption breaks down, and suggests simple mechanisms that could give rise to object selectivity," says Jude Mitchell, a Salk research scientist who was the senior author on the Neuron paper.

"It is important that results from the two studies are quite compatible with one another, that what we find studying just lines and curves in one first experiment matches what we see when the brain experiences the real world," says Sharpee, who is well known for developing a computational method to extract neural responses from natural images.

"What this tells us is that there is a deeper mystery here to be solved," Reynolds says. "We have not figured out how translation invariance is achieved. What we have done is unpacked part of the machinery for achieving integration of parts into wholes."

Explore further: Warning! Collision imminent! The brain's quick interceptions help you navigate the world

Related Stories

Warning! Collision imminent! The brain's quick interceptions help you navigate the world

February 7, 2012
Researchers at The Neuro and the University of Maryland have figured out the mathematical calculations that specific neurons employ in order to inform us of our distance from an object and the 3-D velocities of moving objects ...

Neurons that can multitask greatly enhance the brain's computational power, study finds

May 20, 2013
Over the past few decades, neuroscientists have made much progress in mapping the brain by deciphering the functions of individual neurons that perform very specific tasks, such as recognizing the location or color of an ...

Scientists discover previously unknown requirement for brain development

June 21, 2013
Scientists at the Salk Institute for Biological Studies have demonstrated that sensory regions in the brain develop in a fundamentally different way than previously thought, a finding that may yield new insights into visual ...

How rats see things

April 4, 2013
The image of an object, when projected into the eyes, may take on the most diverse shapes depending on the chosen point of view, as this can change its distance, perspective and so on, yet generally we have no difficulty ...

Recommended for you

New neuron-like cells allow investigation into synthesis of vital cellular components

January 22, 2018
Neuron-like cells created from a readily available cell line have allowed researchers to investigate how the human brain makes a metabolic building block essential for the survival of all living organisms. A team led by researchers ...

Finding unravels nature of cognitive inflexibility in fragile X syndrome

January 22, 2018
Mice with the genetic defect that causes fragile X syndrome (FXS) learn and remember normally, but show an inability to learn new information that contradicts what they initially learned, shows a new study by a team of neuroscientists. ...

Epilepsy linked to brain volume and thickness differences

January 22, 2018
Epilepsy is associated with thickness and volume differences in the grey matter of several brain regions, according to new research led by UCL and the Keck School of Medicine of USC.

Research reveals atomic-level changes in ALS-linked protein

January 18, 2018
For the first time, researchers have described atom-by-atom changes in a family of proteins linked to amyotrophic lateral sclerosis (ALS), a group of brain disorders known as frontotemporal dementia and degenerative diseases ...

Fragile X finding shows normal neurons that interact poorly

January 18, 2018
Neurons in mice afflicted with the genetic defect that causes Fragile X syndrome (FXS) appear similar to those in healthy mice, but these neurons fail to interact normally, resulting in the long-known cognitive impairments, ...

How your brain remembers what you had for dinner last night

January 17, 2018
Confirming earlier computational models, researchers at University of California San Diego and UC San Diego School of Medicine, with colleagues in Arizona and Louisiana, report that episodic memories are encoded in the hippocampus ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Jul 03, 2013
no link to original study.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.