Study reveals how the brain categorizes thousands of objects and actions

December 19, 2012
This figure shows the semantic space projected onto the cortical surface for subjects SN, AH, and JG. (A,C,E) Category model weights for each cortical voxel are projected onto PCs 2-4 of the group semantic space, and then assigned a color. These colors are projected onto cortical flat maps constructed separately for each subject. Each location on these maps represents a single voxel. Locations with similar colors have similar semantic selectivity. Brain areas identified using functional localizers are outlined in white and labeled (see Table S1 for details). Uncertain anatomical boundaries are shown as dashed white lines. Major sulci are denoted by dark blue lines and labeled. Some anatomical regions are labeled in light blue (Abbreviations: PrCu = precuneus; TPJ = temporoparietal junction). Cuts made to the cortical surface during flattening are indicated by dashed red lines and a red border. Blue borders show the edge of the corpus callosum and subcortical structures. Regions of fMRI signal dropout are shaded with black hatched lines. (B,D,F) Projection of voxel model weights onto the first PC. Voxels that are positively correlated with the first PC appear red, while negatively correlated voxels appear blue. Voxels orthogonal to the first PC appear gray. Credit: Huth et al., Neuron

Humans perceive numerous categories of objects and actions, but where are these categories represented spatially in the brain?

Researchers reporting in the December 20 issue of the Cell Press journal Neuron present their study that undertook the remarkable task of determining how the over a thousand object and action categories when subjects watched natural movie clips. The results demonstrate that the brain efficiently represents the diversity of categories in a compact space. Instead of having a distinct brain area devoted to each category, as previous work had identified, for some but not all types of stimuli, the researchers uncovered that brain activity is organized by the relationship between categories.

The video will load shortly
Humans perceive numerous categories of objects and actions, but where are these categories represented spatially in the brain? Researchers reporting in the Dec. 20 issue of the Cell Press journal Neuron present their study that undertook the remarkable task of determining how the brain maps over a thousand object and action categories when subjects watched natural movie clips. Credit: Huth et al., Neuron

"Humans can recognize thousands of categories. Given the limited size of the human brain, it seems unreasonable to expect that every category is represented in a distinct brain area," says first author Alex Huth, a graduate student working in Dr. Jack Gallant's laboratory at the University of California, Berkeley.

The authors proposed that perhaps a more efficient way for the brain to represent object and action categories would be to organize them into a continuous space that reflects the similarity between categories.

(A) Coefficients of all 1,705 categories in the first group PC, organized according to the graphical structure of WordNet. Links indicate "is a" relationships (e.g., an athlete is a person); some relationships used in the model have been omitted for clarity. Each marker represents a single noun (circle) or verb (square). Red markers indicate positive coefficients and blue indicates negative coefficients. The area of each marker indicates the magnitude of the coefficient. This PC distinguishes between categories with high stimulus energy (e.g., moving objects like "person" and "‘vehicle’") and those with low stimulus energy (e.g., stationary objects like "sky" and "city"). (B) The three-dimensional RGB colormap used to visualize PCs 2–4. The category coefficient in the second PC determined the value of the red channel, the third PC determined the green channel, and the fourth PC determined the blue channel. Under this scheme, categories that are represented similarly in the brain are assigned similar colors. Categories with zero coefficients appear neutral gray. (C) Coefficients of all 1,705 categories in group PCs 2–4, organized according to the WordNet graph. The color of each marker is determined by the RGB colormap in (B). Marker sizes reflect the magnitude of the three-dimensional coefficient vector for each category. This graph shows that categories thought to be semantically related (e.g., "athletes" and "walking") are represented similarly in the brain. Credit: Huth et al., Neuron

To test this hypothesis, they used -dependent (BOLD fMRI) to measure human brain activity evoked by natural movies in five people. They then mapped out how 1,705 distinct object and action categories are represented across the surface of the cortex of the brain. Their results show that categories are organized as smooth gradients that cover much of the surface of the visual as well as nonvisual cortex, such that similar categories are located next to each other, and notably, this organization was shared across the individuals imaged.

"Discovering the feature space that the brain uses to represent information helps us to recover functional maps across the cortical surface. The probably uses similar mechanisms to map other kinds of information across the cortical surface, so our approach should be widely applicable to other areas of cognitive neuroscience," says Dr. Gallant.

Explore further: Visual working memory not as specialized in the brain as visual encoding, study finds

More information: Huth et al.: "A continuous semantic space describes the representation of thousands of object and action categories across the human brain."

Related Stories

How the brain forms categories

October 20, 2012

Neurobiologists at the Research Institute of Molecular Pathology (IMP) in Vienna investigated how the brain is able to group external stimuli into stable categories. They found the answer in the discrete dynamics of neuronal ...

How the brain assigns objects to categories

July 28, 2011

The human brain is adept at recognizing similar items and placing them into categories — for example, dog versus cat, or chair versus table. In a new study, MIT neuroscientists have identified the brain activity that ...

Neural balls and strikes: Where categories live in the brain

January 15, 2012

Hundreds of times during a baseball game, the home plate umpire must instantaneously categorize a fast-moving pitch as a ball or a strike. In new research from the University of Chicago, scientists have pinpointed an area ...

Neuroscientists unlock shared brain codes

October 20, 2011

A team of neuroscientists at Dartmouth College has shown that different individuals' brains use the same, common neural code to recognize complex visual images.

Recommended for you

Skin stem cells used to generate new brain cells

April 25, 2017

Using human skin cells, University of California, Irvine neurobiologists and their colleagues have created a method to generate one of the principle cell types of the brain called microglia, which play a key role in preserving ...

How brains process facial expressions

April 25, 2017

Have you ever thought someone was angry at you, but it turned out you were just misreading their facial expression? Caltech researchers have now discovered that one specific region of the brain, called the amygdala, is involved ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.