Why visual perception is a decision process

A popular theory in neuroscience called predictive coding proposes that the brain produces all the time expectations that are compared with incoming information. Errors arising from differences between actual input and prediction are then iteratively minimized along a hierarchical processing scheme. It is assumed that such stepwise iteration leads to updating of brain predictions so that internal prediction errors are finally explained away. Neuroscientists at the Ruhr-Universität Bochum (RUB), together with colleagues at the Freiburg University, show that this is not strictly the case. Instead, they demonstrate that prediction errors can occasionally appear as visual illusion when viewing rapid image sequences. Thus, rather than being explained away, prediction errors remain accessible at the final processing stages forming perception. Previous theories of predictive coding therefore need to be revised. The study is reported in PLOS One on 4 May 2020.

Visual system starts making predictions within a few milliseconds

To fixate objects in the outside world, the eyes perform far more than 100,000 rapid movements per day called saccades. However, as soon as the eyes rest for about 100 milliseconds, the brain starts making predictions. Differences between previous and current image contents are then forwarded to subsequent processing stages as prediction errors. The advantage to dealing with differences instead of complete image information is obvious: Similarly to video compression techniques, the data volume is drastically reduced. Another advantage turns up only at second sight: Statistically, there is a high probability that the next saccade lands on locations where differences to previous image contents are largest. Thus, calculating potential changes of image content as the differences to previous content prepares the visual system early on for new input.

To test whether the brain uses such a strategy, the authors presented rapid sequences of two images to human volunteers. In the first image, two gratings were superimposed; in the second image, only one of the gratings was present. The task was to report the orientation of the last seen single grating. In most cases, the participants correctly reported the orientation of the present orientation, as expected. Surprisingly, however, in some cases, the observer perceived an orientation that was exactly orthogonal to the present orientation—that is, participants sometimes saw the difference between the previous superimposed gratings and the present single grating. "Seeing the difference instead of the real current input is here a visual illusion that can be interpreted as directly seeing the prediction error," says Robert Staadt from the Institute of Neural Computation of the RUB, first author of the study.

Avoiding the pigeonhole benefits flexibility

"Within the framework of the predictive coding theory, prediction errors are mostly conceived in the context of higher cognitive functions that are coupled to conscious expectations. However, we demonstrate that prediction errors also play a role in the context of highly dynamic perceptual events that take place within fractions of a second," explains Dr. Dirk Jancke, head of the Optical Imaging Group at the Institute of Neural Computation. The present study reveals that the visual system simultaneously keeps up information about past, current, and possible future image content. This strategy confers both stability and flexibility when viewing rapid image sequences. "Altogether, our results support hypotheses that consider perception as a result of a decision process," says Jancke. Hence, prediction errors should not be sorted out too early, as they might become relevant for following events.

Visual perception underlies decision making

In subsequent studies, the scientists will scrutinize the sets of parameters that drive the perceptual illusion most effectively. Besides straightforward physical parameters like stimulus duration, brightness and contrast, other more elusive factors that characterize psychological features might be involved. The authors' long-term perspective is the development of practical visual tests that can be used for an early diagnosis of cognitive disorders connected to rapid perceptual decision processes.