ISSN :2582-9793

Evaluation of Explanation Methods of AI - CNNs in Image Classification Tasks with Reference-based and No-reference Metrics

Original Research (Published On: 16-Jan-2023 )
Evaluation of Explanation Methods of AI - CNNs in Image Classification Tasks with Reference-based and No-reference Metrics
DOI : 10.54364/AAIML.2023.1143

Benois-Pineau, Jenny

Adv. Artif. Intell. Mach. Learn., 3 (1):620-646

Benois-Pineau, Jenny : University of Bordeaux, LABRI UMR 5800 UBX, CNRS, IPB

Download PDF Here

DOI: 10.54364/AAIML.2023.1143

Article History: Received on: 21-Dec-22, Accepted on: 11-Jan-23, Published on: 16-Jan-23

Corresponding Author: Benois-Pineau, Jenny

Email: jenny.benois-pineau@u-bordeaux.fr

Citation: Benois-Pineau, Jenny (2023). Evaluation of Explanation Methods of AI - CNNs in Image Classification Tasks with Reference-based and No-reference Metrics. Adv. Artif. Intell. Mach. Learn., 3 (1 ):620-646

          

Abstract

    

The most popular methods in AI-machine learning paradigm are mainly black boxes. This is
why explanation of AI decisions is of emergency. Although dedicated explanation tools have
been massively developed, the evaluation of their quality remains an open research question. In
this paper, we the generalize the methodologies of evaluation of post-hoc explainers of CNNs’
decisions in visual classification tasks with reference and no-reference based metrics. We apply
them on our previously developed explainers (FEM, MLFEM), and popular Grad-CAM. The
reference-based metrics are Pearson correlation coefficient and Similarity computed between
the explanation map and its ground truth represented by a Gaze Fixation Density Map obtained
with a psycho-visual experiment. As a no-reference metric we use stability metric, proposed by
Alvarez-Melis and Jaakkola. We study its behaviour, consensus with reference-based metrics
and show that in case of several kind of degradation on input images, this metric is in agreement
with reference-based ones. Therefore it can be used for evaluation of the quality of explainers
when the ground truth is not available.

Statistics

   Article View: 913
   PDF Downloaded: 71