Human brain to help computers analyze images

(PhysOrg.com) -- Brain-computer-interfaces (BCIs) are most often designed to help people (such as those with disabilities) do things they find difficult on their own, but a joint project by Columbia University and a spinoff company, Neuromatters, LLC, is trying to develop a BCI to help computers to rapidly identify images of interest.

The technology, called Cortically Coupled Computer Vision (C3 Vision) takes advantage of the fact that the human brain can react to images it finds interesting or relevant faster than its owner is consciously aware of, and couples this with the processing power of computers for rapidly identifying those images.

Using the human brain provides the subjective or abstract qualities that computers lack. Dr. Paul Sajda, Professor of Biomedical Engineering at Columbia University, said the brain is good at ranking images according to whether or not the person likes the picture or finds it interesting, which computers cannot do. But computers can process millions of images quickly, which would easily overwhelm the human brain.

The human subject is fitted with an electroencephalography (EEG) cap and shown a sample of images from a large database at a rate of up to 10 images per second. In one experiment the subjects were asked to look for surface to air missiles in satellite images. Every time an interesting or relevant image is shown the electrodes in the cap detect electrical activity in the brain, and the signals are captured and decoded by C3Vision. The person may be unaware, or just barely aware of having seen the image.

The computer then ranks the images according to the strength of the electrical activity, and looks for similarities in the images, such as the shapes, textures and colors. In this way the computer gradually learns which images the brain is deeming interesting, and is eventually able to identify them by itself.

Once trained the computer examines images in the much larger database from which the samples were taken. This could contain many millions of images, and the computer selects images similar in visual characteristics to those selected

Using the human brain provides the subjective or abstract qualities that computers lack. Dr. Paul Sajda, Professor of Biomedical Engineering at Columbia University, said the brain is good at ranking images according to whether or not the person likes the picture or finds it interesting, which computers cannot do. But computers can process millions of images quickly, which would easily overwhelm the human brain.

The human subject is fitted with an electroencephalography (EEG) cap and shown a sample of images from a large database at a rate of up to 10 images per second. In one experiment the subjects were asked to look for surface to air missiles in satellite images. Every time an interesting or relevant image is shown the electrodes in the cap detect electrical activity in the brain, and the signals are captured and decoded by C3Vision. The person may be unaware, or just barely aware of having seen the image.

The computer then ranks the images according to the strength of the electrical activity, and looks for similarities in the images, such as the shapes, textures and colors. In this way the computer gradually learns which images the brain is deeming interesting, and is eventually able to identify them by itself.

Once trained the computer examines images in the much larger database from which the samples were taken. This could contain many millions of images, and the computer selects images similar in visual characteristics to those selected by the human brain.

The project has received financial support from the Defense Advanced Research Projects Agency (DARPA) to develop and test the technology, which is intended for use as in applications such as helping government image analysts to evaluate millions of satellite images for specific characteristics. It may also find application in security applications such as video surveillance, where it would be looking for images of suspicious activity. Neuromatters CEO, Barbara Hanna, said the company aimed to "push the envelope" of computer-brain interfaces to help in areas where there is a problem of information overload.

Other applications envisaged for C3Vision include analysis of consumer preferences in online markets such as fashion, consumer goods, travel products, and even real estate, in a new field known as neuro-marketing. The project is overseen by the technology transfer office at Columbia University, "Columbia Technology Ventures," which acts as a gateway for commercial companies wishing to join forces with the University in developing new technologies.

© 2010 PhysOrg.com