“Why Should I Trust in AI?" – Analysis of the Influence of Explanations on Trust in Artificial Intelligence

“Why Should I Trust in AI?" – Analysis of the Influence of Explanations on Trust in Artificial Intelligence

Author: Franziska Pia Herbert

Supervisors: Prof. Dr. Kristian Kersting, Prof. Dr. Frank Jäkel

Submission: March 2019

Abstract:

This study has the purpose to investigate the influence of explanations on trust in Artificial Intelligence (AI). Explainable AI is a resurging field of study, which aims to increase trust in AI and usage of AI by explaining its decision. However, most of the explanations are produced by AI developers and are not validated by users. Therefore, the target of this study is to investigate, whether users trust explained AI decisions more than AI decisions without explanations, as well as to investigate the quality of explanations.

An online survey with more than 140 subjects and two focus groups with nine participants were performed. The presented AI classified images with four variations of explanations (test conditions TC): no explanation (TC1), a correct explanation (TC2), nonsystematic false explanations (TC3), and systematic false explanations (TC4). The quantity of subjects and their occupation per test condition is given in Figure 1 (total number of valid participant tests: 144).

Figure 1: Quantity of subjects’ occupations per test condition (N = 144)
Figure 1: Quantity of subjects’ occupations per test condition (N = 144)

Concerning Statistical Methods, a missing value analysis was conducted. For all tests a significance level with alpha being 5% was used. For all tests with the same sample/samples the alpha level was corrected via the Bonferroni-Holm method. For testing the hypotheses one multi-factorial analysis of variances (MANOVA) and several one-factorial ANOVAs were conducted. The required normal distribution of data, independence of data as well as homogeneity of the variances were verified with a Levene-Test before every ANOVA and the MANOVA. If the analyses of variances were significant, post hoc tests were carried out, i.e. the Tukey-HSD-Test and the Pairwise-Test were performed.

The results show, that people trust significantly less in the AI classification with a false explanation, than in the AI classification with no explanation or with a correct explanation. No significant difference in trust was found for no explanation and a correct explanation of the AI classification. Consequently, correct explanations were not found to increase trust but false explanations were found to diminish trust in AI. The overall trust in AI was moderate. Users demanded easy visual explanations, information on the underlying model and information on the used features for the AI classification, especially for security relevant tasks, as, for example, autonomous driving. This demonstrates the users’ need for explainable AI and that explanation correctness should be a major goal of explainable AI.