EXPLAINABLE ARTIFICIAL INTELLIGENCEALEXANDRA NAU | TRENDS IN MACHINE LEARNING AND DATA ANALYTICS | WS 2019/20

CONTENT

¡ Background

¡ Introduction XAI

¡ Reasons for XAI

¡ Explainability methods

¡ LRP

¡ LIME

¡ Taxonomy of interpretability evaluation

¡ Summary

2

BACKGROUND

¡ Machine Learning & Artificial Intelligence => greatimpact on society

¡ Black Box => Not trustworthy

¡ Trust needs explainability

3

Image 1: Example DeepNeurional Network (https://www.darpa.mil/attachments/XAIProgramUpdate.pdf)

INTRODUCTION – XAI

¡ „eXplainable Artificial Intelligence“

¡ 2004 , Van Lent, Fisher und Mancuso

¡ No standard or general definition

¡ Countermeasure to lack of transparency

¡ Understanding and explainablility => trust

4

INTRODUCTION – TREND CHART

5

Image 2: Google trends result for research interest of „Explainable Artificial Intelligence“ and „Interpretable machine learning“ (https://trends.google.com/trends/explore?date=all&q=Explainable%20Artificial%20Intelligence,interpretable%20machine%20learning; 20/01/2020)

„Interpretable Artificial Intelligence“ „Explainable Artificial Intelligence“

REASONS FOR XAI I

6

Justifying Decisions

Controlling

Improving

Learning

REASONS FOR XAI II

7

Image 3: End to End ML Workflow, (Mathews; 2019)

INTERPRETABILITY VS. ACCURACY

8

Image 4: Trade-off between modelinterpretability and accuracy,(Arrieta, Del Ser et al; 2019)

EXPLAINABILITY METHODS

9

Image 5: A pseudo ontology of XAI methods taxonomy(Adadi, Berrada; 2018)

EXAMPLES OF EXPLAINABILITY METHODS

10

LAYER-WISE RELEVANCE PROPAGATION (LRP)

¡ Post-hoc, model-agnostic

¡ Which features of the input vector have a great impact on the output?

¡ Backward process of the neuronal network

11

LRP – EXAMPLE I

DEEP MACHINE LEARNING | EXPLAINABLE ARTIFICIAL INTELLIGENCE | ALEXANDRA NAU 12

Image 6: LRP Example (Samek, Müller et al.; 2015)

LRP – EXAMPLE II

13

Image 7: A female and male subject in happy (left) and sad (right); (Arbabzadah, Montavon , Müller & Samek; 2016)

LRP – EXAMPLE III

14

Image 8: Two old test subjects (Arbabzadah, Montavon , Müller & Samek; 2016)

LOCAL INTERPRETABLE MODEL-AGNOSTIC EXPLANATIONS (LIME)

¡ Model-agnostic

¡ local

¡ Transparency and interpretability of classificators

¡ Local approximation => interpretable model

¡ Text: Highlighting of important words

¡ Images: Highlighting of important superpixels

15

LOCAL INTERPRETABLE MODEL-AGNOSTIC EXPLANATIONS (LIME)

16

Image 9: Illustration of explanationprocess for LIME Framework. Original non-linearmodel’s decision function representedby blue/pink background. Bright redcross is theinstance being explained (X). Dashedline is linear model (Mathews; 2019)

LOCAL INTERPRETABLE MODEL-AGNOSTIC EXPLANATIONS (LIME)

1. Permute data

2. Calculate distance permutations and original observations

3. Make predictions on new data using complex model

4. Pick m features best describing the complex model outcome from the permuted data.

5. Fit a simple model to the permutated data with m features and similarity scores as weights.

6. Feature weights from the simple model make explanations for the complex models local behaviour

17

LIME – EXAMPLE I

18

Image 10: wolf and husky prediction (https://www.slideshare.net/0xdata/interpretable-machine-learning-using-lime-framework-kasia-kulma-phd-data-scientist)

LIME – EXAMPLE I

19

Image 11: wolf and huskyprediction - explanation(https://www.slideshare.net/0xdata/interpretable-machine-learning-using-lime-framework-kasia-kulma-phd-data-scientist)

LIME – EXAMPLE II

20

Image 12: Result of the first fiveobservations for interpretationof tweet classification(Mathews; 2019)

LIME – EXAMPLE II

21

Image 12: Results of the textexplanations which indicate thedirection but not strengthof the relationship of support orcontradiction of certain wordsfor a given label(Mathews; 2019)

TAXONOMY OF INTERPRETABILITY EVALUATION

APPLICATION-GROUNDED

¡ Humans & real tasks

¡ Experiment within application

¡ Human domain expert

FUNCTIONALLY-GROUNDED

¡ No humans, proxy task

¡ Formal definition ofinterpretability as proxy forexplanation quality

¡ Only use when model classesare already validated

22

¡ Humans & simplified tasks

¡ Maintain essence of targetapplication

¡ No domain expert

HUMAN-GROUNDED

SUMMARY

¡ Not every black-box model needs to be explainable

¡ Black-box models must not be used in critical domains

¡ Trade-off: Accuracy ó Interpretability

¡ XAI => trust in AI

¡ New findings through XAI

23

REFERENCES I

¡ Adadi & Berrada; Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI); 2018

¡ Samek, Wiegand & Müller; EXPLAINABLE ARTIFICIAL INTELLIGENCE: UNDERSTANDING,VISUALIZING AND INTERPRETING DEEP LEARNING MODELS; 2017

¡ Samek, Müller et al.; Explaining NonLinear Classification Decisions with DeepTaylor Decomposition; 2015

¡ Mathews; Explainable Artificial Intelligence Applications in NLP, Biomedical, and Malware Classification: A Literature Review; 2019

¡ Doshi-Velez & Kim; Towards A Rigorous Science of Interpretable Machine Learning; 2017

¡ Hoffmann; LIME ein vielseitiges Erklärermodell - auch für Machine-Learning-Laien; 2017/2018

¡ Arbabzadah, Montavon , Müller & Samek; Identifying individual facial expressions by deconstructing a neural network; 2016

¡ Arrieta, Del Ser et al.; Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges towardResponsible AI; 2019

24

REFERENCES II

¡ https://www.slideshare.net/0xdata/interpretable-machine-learning-using-lime-framework-kasia-kulma-phd-data-scientist, last access: 20/01/2020

¡ http://danshiebler.com/2017-04-16-deep-taylor-lrp/, last access : 10/01/2020

¡ https://www.darpa.mil/attachments/XAIProgramUpdate.pdf, last access : 10/01/2020

25