What's in a simple line drawing? Quite a lot, our brains say

May 16, 2011 by Jeff Grabmeier

A new study using sophisticated brain scans shows how simple line drawings can capture the essence of a beach or a mountain for viewers just as well as a photograph would.

Researchers found that viewing a "beach" scene depicted in a line drawing activated nearly the same patterns of brain activity in study participants as did viewing an actual color photograph of a beach. The same was true when people viewed line drawings and photographs of other natural scenes including city streets, forests, highways, mountains and offices.

Even when researchers removed up to 75 percent of the pixels in a line drawing, people still did better than chance at determining what the lines represented -- as long as the remaining lines showed the broad contours of the scene.

"Our results suggest that our brains can recreate whole detailed scenes from just a few lines, said Dirk Bernhardt-Walther, lead author of the study and assistant professor of at Ohio State University.

"The representations in our brain for categorizing these scenes seem to be a bit more abstract than some may have thought – we don't need features such as texture and color to tell a beach from a street scene," he said.

Walther conducted the study with Barry Chai and Li Fei-Fei of Stanford University and Eamon Caddigan and Diane Beck with the University of Illinois. Their results appear in the online early edition of the Proceedings of the National Academy of Sciences.

For the study, 10 participants viewed color photographs and line drawings of six categories of scenes -- beaches, city streets, forests, highways, mountains and offices -- while their brains were scanned using functional magnetic resonance imaging (fMRI).

The fMRI images showed the researchers what was going on in several areas of the participants' brains when they viewed the photos and line drawings. The most significant results occurred in the parahippocomal place area (PPA), an area of the brain that scientists know plays an important role in the encoding and recognition of scenes (rather than faces or objects).

Using the data from when participants viewed the color photos, the researchers trained a software-based decoder to tell what type of scene the participants viewed -- a beach, mountain, etc., -- based on the patterns of in the PPA revealed in the fMRI.

The decoder was far from perfect, but it did better than chance at predicting what scene a person was viewing in a particular fMRI image.

Most importantly, the decoder could do just as well at predicting which scene a person viewed when it was focused on line drawings as it was on photographs. In fact, the decoder did slightly better -- although not significantly so -- at predicting line drawings compared to photographs in the primary visual cortex.

"We expected that line drawings would be good enough to allow some decoding, but it was surprising that there was no benefit to photographs -- the decoder was no better when it was used on photos than it was on line drawings," Walther said.

Findings showed that when the decoder was trained on photographs, it still did equally well at predicting which scenes people were viewing in line drawings, and vice versa.

"That suggests the brain uses the same information to decode which scene it is viewing when it is presented with line drawings or photos," he said.

In addition, results showed that when the decoder did make errors, it made similar errors in both photographs and line drawings. For example, if the decoder thought people were looking at a photo of a mountain when they were really looking at a photo of a forest, it would make the same mistake when it was analyzing line drawings.

"The patterns of error match incredibly well, so that's an additional piece of evidence that the representations for photos and line drawings are very similar in the brain," Bernhardt-Walther said.

But what is it about line drawings that allow people to recognize what they represent? As part of the study, the researchers removed some of the lines in the line drawings and asked participants if they could still tell what scene was depicted. In some cases, they removed up to 75 percent of the pixels in the drawing.

If the researchers left the long contours in the drawings, which represented global structure -- such as sky, water or sand -- participants could still correctly predict what kind of scene was depicted about 60 percent of the time.

However, when researchers took out these long contours and left only short ones -- representing details like leaves, windows in buildings or individual ridges in a mountainside -- the accuracy of participants went way down.

These findings cast doubt on some models of human visual perception which argue that people need specific information that is found in photographs -- such as color, shading and texture -- to classify a scene.

"Of course, we use the rich sources of information found in a photograph when it is available, but the is an opportunist -- it uses what is available," Walther said. "We can get a lot of information from a line drawing."

The results also suggest why line drawings have played such an important role in human history, both as an art form and a way of presenting information simply.

"Imagine the astonishment of early man when he discovered he could draw shapes on a rock wall and it resembled the actual animal he had just killed. Line drawings have been with us since prehistoric times," Walther said.

Related Stories

Recommended for you

The neural codes for body movements

July 21, 2017
A small patch of neurons in the brain can encode the movements of many body parts, according to researchers in the laboratory of Caltech's Richard Andersen, James G. Boswell Professor of Neuroscience, Tianqiao and Chrissy ...

Faulty support cells disrupt communication in brains of people with schizophrenia

July 20, 2017
New research has identified the culprit behind the wiring problems in the brains of people with schizophrenia. When researchers transplanted human brain cells generated from individuals diagnosed with childhood-onset schizophrenia ...

Scientists reveal how patterns of brain activity direct specific body movements

July 20, 2017
New research by Columbia scientists offers fresh insight into how the brain tells the body to move, from simple behaviors like walking, to trained movements that may take years to master. The discovery in mice advances knowledge ...

Scientists discover combined sensory map for heat, humidity in fly brain

July 20, 2017
Northwestern University neuroscientists now can visualize how fruit flies sense and process humidity and temperature together through a "sensory map" within their brains, according to new research.

Team traces masculinization in mice to estrogen receptor in inhibitory neurons

July 20, 2017
Researchers at Cold Spring Harbor Laboratory (CSHL) have opened a black box in the brain whose contents explain one of the remarkable yet mysterious facts of life.

Speech language therapy delivered through the Internet leads to similar improvements as in-person treatment

July 20, 2017
Telerehabilitation helps healthcare professionals reach more patients in need, but some worry it doesn't offer the same quality of care as in-person treatment. This isn't the case, according to recent research by Baycrest.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.