Author: Dominik Straub

Referees:
Prof. Constantin A. Rothkopf, Ph.D.
Prof. Dr. Joan López-Moliner

Defense: 06.03.2025

Abstract:

Normative models based on probability theory and decision theory are popular in many areas of cognitive science. One reason for this popularity is that they frame behavior as the result of rational decisions based on beliefs and goals and thereby supply explanations similar to our everyday intuitive understanding of behavior. Another reason is that by analyzing the optimal solution to a task, they constrain the space of hypotheses for possible process models and algorithmic-level explanations.

In perception research, normative models are especially prolific and have belonged to its standard toolkit for decades in the form of ideal observers and signal detection theory. More recently, Bayesian models of perception have given normative accounts for perceptual decisions across various tasks. Mathematically, Bayesian models represent beliefs as probability distributions, which constitute an internal model, and goals as cost functions. Behavior is modeled as the action that minimizes the expected cost under the current belief, given the internal model and sensory observations.

If the probability distributions in the internal model match the statistics of the actual generative model of the world, and the cost function matches the external reward structure of the actual task being performed, the behavior of a Bayesian model constitutes a theoretical optimum. However, there are many reasons why people might have internal models that differ from the statistics of an experiment or cost functions that differ from the task assigned to them by a researcher while still acting rationally. If we would like to understand their behavior with normative models in terms of beliefs and goals, we cannot simply assume their internal model and cost function a priori but must infer them from behavior. This approach can be called inverse normative modeling.

However, inverse normative modeling has so far largely been limited to relatively simple tasks with static stimuli and discrete choices. The reason for this limitation is that such tasks allow tractable computation of the probabilistic inference and decision-making problems implied by the normative model. If solving these problems already requires computationally intensive numerical methods, inferring the model’s parameters from behavioral data would be prohibitively expensive. For this reason, methods that extend inverse normative modeling to more naturalistic tasks with continuous behavior and a sequential structure are a significant and necessary advancement.

Developing such methods is the central contribution of this thesis.

First, we introduce a new method for approximating Bayesian models for tasks with continuous actions using neural networks. This approach allows us to efficiently compute optimal actions for combinations of cost functions and probability distributions that do not admit analytically tractable solutions. Using the trained neural network as a stand-in for the Bayesian model enables efficient Bayesian inference of the model’s parameters given observed behavior. We show on simulated data that the inferred posterior distributions are in close alignment with those obtained using analytical solutions where they exist. Once we turn to more complex cost functions, unidentifiability issues can arise, as it is not always possible to determine whether biases in behavior were due to prior beliefs or costs. With the new method, we show how to disentangle these different factors.

Second, we apply inverse normative modeling to continuous psychophysics, a recent methodological proposal in vision science that replaces the rigid structure of classical psychophysics experiments, which consist of independent trials and binary decisions, with a more naturalistic continuous tracking task. Formalizing the task in the normative framework of optimal feedback control incorporates not only perceptual uncertainty but also motor variability, behavioral costs, and subjective internal models. To infer these factors from observed behavior, we introduce a probabilistic method for inverse optimal control. Using this method, we not only recover perceptual uncertainty parameters in alignment with classical psychophysics but also test the additional motor and cognitive processes using principled model comparison, providing a robust validation of the new method. Third, we extend the inverse optimal control algorithm to account for a broader range of behaviors. While optimal feedback control with signal-dependent noise is one of the most influential modeling approaches for sensorimotor control, there has been no method for fitting such models to behavioral data. Integrating signal-dependent noise enables the application of inverse optimal control to behaviors like saccadic eye movements and manual reaching tasks. We further extend the algorithm to non-linear dynamics and non-quadratic costs, which broadens the domain of applications to more detailed biomechanical models and enables the investigation of information-seeking behavior.

Finally, we investigate the relationship between descriptive and normative models. In the context of interception tasks, in which people walk towards moving targets, the dominant approach has been to model the behavior using so-called online control strategies, which explain behavior without assuming internal models or goals. We show that both a model-based optimal control model of an interception task and an online control strategy equivalently capture average trajectories. However, when realistic levels of perceptual uncertainty, motor variability, and sensorimotor delays are included, the performance of online control without an internal model quickly deteriorates. Based on this analysis, we caution against inferring the absence of internal models simply from observing behavior in agreement with a model-free strategy. We also emphasize the importance of incorporating what is known about the sensorimotor system in models.

In summary, this thesis provides a new set of tools for analyzing behavior with continuous actions under perceptual uncertainty, which opens the possibility of fitting normative models to behavior in more naturalistic tasks. We discuss how the posited components of the models developed in this thesis could be validated using other measurements. We suggest potential avenues for extending the approach and connecting it to theories about neural implementations of Bayesian inference and decision-making.

Learn more (TUprints)