A photo shows us only the bonnet of a car protruding from behind a house – and yet adult viewers instantly see that it is a car. This is because the human mind can normally grasp what objects are present in a scene, and also imagine the hidden parts of it. But how exactly does the mind recognise the meaning of scenes like this? This question will be investigated in the “SEGMENT” research project headed by , which is now funded by a renowned TU Psychology and Cognitive Science professor Thomas Wallis. The funding amount over a period of five years amounts to around 2.1 million euros. ERC Consolidator Grant
The first step in the visual understanding of scenes is known as segmentation. This is when the human brain attempts to recognise which parts of a scene belong to which objects. Adults can do this to a certain extent from photos – but this is likely because they learnt to see in a 3D world as toddlers rather than purely through photos. Information on our environment is provided by, for instance, the way scenes project into the human eye, how light is influenced by the optics of the eye, how photoreceptors in the eye react to light, and how the eyes move.
How do adults segment scenes
However, it is not known how people combine all this information from active 3D vision for segmentation. One reason is because traditional visual display devices are unable to accurately mimic these factors – and it would be unethical to manipulate these factors for a long time in young children. This is where Professor Wallis's project comes in. The aim of “SEGMENT” is to develop a new display device and experimental methods to investigate how adults segment scenes, and to create simulations of the visual system from early development to adulthood. Professor Wallis's team at Darmstadt wants to use groundbreaking new technologies based on advanced computer graphics and machine learning for this purpose.
There are also plans to conduct computer experiments with artificial neural networks to help in understanding segmentation learning. The researchers want to systematically restrict or manipulate various factors such as optics or eye movements. At the end, the learned behaviour of the artificial networks will be compared with the behaviour of adults during segmentation in the active exploration of 3D scenes. “We want to use similarities and differences to better understand a fundamental mystery in perception: how the mind makes sense of scenes,” explained Professor Wallis.
About the recipient
has been a Professor of Perception at the Institute of Psychology and Thomas Wallis at TU Darmstadt since 2021. His research focuses on visual perception in humans and machines, machine learning and cognitive modelling, and applications in visual perception. Centre for Cognitive Science
Australian-born Wallis received his doctorate from the University of Queensland in Australia in 2010. He then conducted research at Schepens Eye Research Institute and Harvard Medical School in the USA as well as at the Centre for Integrative Neurosciences and the Department of Computer Science at the Eberhard Karl University of Tübingen. Thomas Wallis was a researcher at Amazon Deutschland from 2019 until 2021.
are awarded by the ERC Consolidator Grants to researchers from all disciplines between seven and twelve years after completing their doctorates. The European Union does so to promote promising research: the Consolidator Grant is aimed at researchers who have already achieved excellent work and are now to be supported in groundbreaking research projects in order to achieve scientific consolidation. In the current round, 321 grants have been awarded and 2.222 applications submitted. European Research Council
Along with Professor Wallis, other recipients of an ERC Consolidator Grant at TU Darmstadt were and Professor Leopoldo Molina-Luna for the ELECTRON project . Professor Eva Kassens-Noor for the scAInce project