Study investigates the role of gaze in tackling new and complex tasks

Putting away items on a shelf is something we do without giving it much thought—right? In fact, various skills need to be coordinated to accomplish even simple tasks: Hand-eye coordination, memory, problem solving and attention are just some of them. To ensure that everything works smoothly and quickly in the end, the interplay of these cognitive requirements must also be planned.

In 2001, a study found that people prefer to use simple action patterns for easy and familiar tasks and are "lazy" planners. A team of researchers from the Institute of Cognitive Science at Osnabrück University has now discovered that we also pursue the so-called "just in time" strategy for new and difficult tasks.

This is the result of a study by cognitive scientist Ashima Keshava and the head of the Neurobiopsychology department, Prof. Dr. Peter König. The findings, posted to bioRxiv, can help to better understand and optimize learning processes.

"When we solve tasks in which we use objects—for example when driving a car or cooking—we can gain a lot of useful information by looking at the objects and adapting our movements accordingly: For example, we need to handle a knife differently from a ball and a hot object differently from a cold one," explains Ashima Keshava, Ph.D. student in cognitive science.

"The just-in-time strategy describes how we focus our gaze on the object just before we use it. It is only in the last 600 milliseconds before the action that we take in all the important information about it. Instead of looking at objects for a long time, memorizing their properties and then planning a course of action, our gaze only supports short-term actions. This saves mental effort in complex tasks and enables quick action, but often leads to less than optimal processes."

In the study, 55 participants were asked to sort objects into the compartments of a cupboard in a virtual environment. The objects differed in shape and color and had to be sorted according to certain specifications: In the simple difficulty level, for example, they were only supposed to sort by color. In the more complex level of difficulty, on the other hand, each color and shape had to appear only once per row.

In the more complex task, the participants had planned longer, but carried out the actions just as quickly and according to the "just-in-time" method. As a result, the action sequences were far from optimal despite longer planning.

"Our study shows a close connection between gaze and action. To be more precise, gaze only ever serves short-term planning: We look where the next object is, guide our hands in this direction and monitor the action," says Prof. Dr. Peter König, head of the Neurobiopsychology department. "Such a simple approach ensures that people prefer suboptimal processes to planning when faced with increased cognitive demands."

"Our study suggests that tasks involving objects have little to do with memory and that people refrain from planning actions for the long term. We always jump to the object we need next," says Keshava. "This knowledge could be used to guide actions and improve learning processes. For example, our industrial partners at Halocline.io are developing virtual reality solutions for training in the correct use of machines and tools.

"Training methods with an AI assistant could facilitate learning by highlighting parts of the machines that are relevant for an immediate action. Human gaze behavior, in turn, can give the AI system feedback on the cognitive state of the human. If the person has lost the thread and is looking for something, the assistant can guide the person back to the task-relevant object."

The results of the study can be categorized into the theory of fast and slow thinking. The system developed by the recently deceased Nobel Prize winner Daniel Kahneman distinguishes between two types of thinking: slow thinking happens consciously, is only activated in challenging situations and seems logical. Fast thinking, on the other hand, happens automatically, is always active and tends to be more emotional.

The capacity for slow thinking is quickly exhausted or unused as it is effortful. Therefore, we often fall back on the simple fast thinking process. In their study, Keshava and König were able to show that eye movements—the most frequently performed movement in everyday life—are closely linked to fast thinking. "People are only 'lazy' planners when things have to be done quickly and this also applies to our gaze," they state.