Enhancing Human-AI Interaction with Probabilistic Behavior Prediction for Situational Autonomy Adaptation

Author: Eric Frodl

Supervisors: Dr. Dorothea Koert, Dr. Dirk Balfanz

Submission: April 2023

Abstract:

Due to ongoing advances in robotics and artificial intelligence (AI), the role of robots in the work context evolves from a tool that serves only a strictly defined purpose to a teammate that can work with the human towards a shared goal. Designing robots for collaborative tasks imposes new challenges as their interaction with the human and the environment is more complex. The notion of dynamic adaptive autonomy (DAA) addresses these challenges by allowing the robot to act according to its capabilities, while being able to adapt to unknown situations that occur over time. Yet, there is a lack of actual frameworks of DAA that guide the design of use cases to examine its strengths and potential weaknesses. This thesis proposes a concept of DAA that should fill the gap between theoretical autonomy concepts and concrete implementations that lead to real experiments. Additionally, an experimental setup that implements the proposed concept is outlined, in which predictions of human actions are used by the robot to adapt to the human co-worker. For this purpose, different skeleton-based action recognition algorithms are implemented and analyzed. The results justify their use in the presented use-case, motivating the conduction of the present experiment and follow-up studies under the proposed notion of autonomy.

Figure 1: Snapshots of a video-based data collection for skeleton-based action recognition. (a) to (f) show different actions that should be recognized after training the action recognition algorithms.
Figure 1: Snapshots of a video-based data collection for skeleton-based action recognition. (a) to (f) show different actions that should be recognized after training the action recognition algorithms.
Figure 2: A visualization of the joints from a participant performing the actions sort (a), punch (b) and insert (c). It can be observed, that these actions contain very similar poses which can be easily confused.
Figure 2: A visualization of the joints from a participant performing the actions sort (a), punch (b) and insert (c). It can be observed, that these actions contain very similar poses which can be easily confused.