The eyes have a plan

Making a business plan, selecting a travel itinerary, or playing a game of chess have in common, that we need to plan our actions ahead of time in order to achieve our goals. A fundamental characteristic of planning is that sometimes we may need to get away from our goal temporarily: we need to invest by spending money before getting a return and earning, sometimes we need to sacrifice a chess piece to mate the king of our opponent, or we need to take a detour to get faster to our destination.

Artificial intelligence has a precise definition of planning, which says that deciding on the course of action by considering possible future situations before they are actually experienced constitutes planning. This makes planning tough, because many future consequences may need to be taken into account when planning ahead. It would be easier to only consider the consequences of a single next action, i.e. not to plan ahead. Unfortunately, in general this may have really detrimental consequences and may even lead to missing the goal altogether. Thus, how humans take future costs and benefits into account has considerable consequences.

Now researchers at the Centre for Cognitive Science have shown in a series of experiments that the way humans choose to direct their eyes must be planned ahead. To this end, the researchers measured fast eye movements of their participants, who were instructed to look for tiny targets within irregular shapes shown on a monitor. While in one condition participants had as little time as was sufficient to only carry out a single eye movement, in a second experiment enough time was available to carry out two eye movements.

The experiments revealed that if participants were given time for a single eye movement their gaze selected a target that allowed a good coverage of the shape to find the target. By contrast, when it was possible to carry out two eye movements, participants automatically directed their eyes to a different point of the shape, which did not allow to find the target as well. But together with the second eye movement, the overall sequence of gaze shift was much more successful at finding the target.

Remarkably, the gaze targets selected by the human participants were well predicted by a planning algorithm from artificial intelligence. By contrast, previously utilized models for the prediction of human gaze targets, which do not plan ahead where to look, deviated from the observed behavior.

The present study reveals the sophistication of the information processing of the human visual system: it takes future actions into account in order to plan where to look next. This result is not only relevant for the understanding and prediction of human gaze shifts, which we carry out about three times a second, but also informs researchers about the information processing going on in the brain. While conscious planning as in playing chess, selecting a travel itinerary, or investing your hard earned money require effort and may be riddled by bad choices, our eyes plan for us almost optimally.

David Hoppe & Constantin A. Rothkopf. Multi-step planning of eye movements in visual search. Scientific Reports, volume 9, Article number: 144 (2019) DOI: 10.1038/s41598-018-37536-0