HCOMP2022_ARCHIE (Analogy-based Explanation)

It is Like Finding a Polar Bear in the Savannah! Concept-level AI Explanations with Analogical Inference from Commonsense Knowledge. HCOMP 2022.

This is our code, collected analogies, and expert evaluation results. More info will be updated later.

Illustration for our motivation

Introduction

With recent advances in explainable artificial intelligence (XAI), researchers have started to pay attention to conceptlevel explanations, which explain model predictions with a high level of abstraction. However, such explanations may be difficult to digest for laypeople due to the potential knowledge gap and the concomitant cognitive load. Inspired by recent work, we argue that analogy-based explanations composed of commonsense knowledge may be a potential solution to tackle this issue. In this paper, we propose analogical inference as a bridge to help end-users leverage their commonsense knowledge to better understand the concept-level explanations. Specifically, we design an effective analogy-based explanation generation method and collect 600 analogy-based explanations from 100 crowd workers. Furthermore, we propose a set of structured dimensions for the qualitative assessment of analogy-based explanations and conduct an empirical evaluation of the generated analogies with experts. Our findings reveal significant positive correlations between the qualitative dimensions of analogies and the perceived helpfulness of analogy-based explanations. These insights can inform the design of future methods for the generation of effective analogy-based explanations. We also find that the understanding of commonsense explanations varies with the experience of the recipient user, which points out the need for further work on personalization when leveraging commonsense explanations.

Acknowledgement

Any scientific publications that use our codes and datasets should cite the following paper as the reference:

@inproceedings{He-HCOMP-2022,
    title = "It is Like Finding a Polar Bear in the Savannah! Concept-level AI Explanations with Analogical Inference from Commonsense Knowledge",
    author = {Gaole He and
              Agathe Balayn and
              Stefan Buijsman and
              Jie Yang and
              Ujwal Gadiraju},
    booktitle = {10th AAAI Conference on Human Computation and Crowdsourcing, {HCOMP} 2022},
    year = {2022},
}

Nobody guarantees the correctness of the data, its suitability for any particular purpose, or the validity of results based on the use of the data set. The data set may be used for any research purposes under the following conditions:

• The user must acknowledge the use of the data set in publications resulting from the use of the data set.
• The user may not redistribute the data without separate permission.
• The user may not try to deanonymise the data.
• The user may not use this information for any commercial or revenue-bearing purposes without first obtaining permission from us.