Papers on Explainable Artificial Intelligence

This is an on-going attempt to consolidate all interesting efforts in the area of understanding / interpreting / explaining / visualizing machine learning models.

---

GUI tools

• Deep Visualization Toolbox. Yosinski et al. 2015 code | pdf

Surveys

• Methods for Interpreting and Understanding Deep Neural Networks. Montavon et al. 2017 pdf
• The Mythos of Model Interpretability. Lipton 2016 pdf
• Towards A Rigorous Science of Interpretable Machine Learning Doshi-Velez & Kim. 2017 pdf
• Visualizations of Deep Neural Networks in Computer Vision: A Survey. Seifert et al. 2017 pdf
• How convolutional neural network see the world - A survey of convolutional neural network visualization methods. Qin et al. 2018 pdf
• A brief survey of visualization methods for deep learning models from the perspective of Explainable AI. Chalkiadakis 2018 pdf

Visualizing Preferred Stimuli

Activation Maximization

• Visualizing higher-layer features of a deep network. Erhan et al. 2009 pdf
• Understanding Neural Networks through Deep Visualization. Yosinski et al. 2015 pdf | url
• Multifaceted Feature Visualization: Uncovering the different types of features learned by each neuron in deep neural networks. Nguyen et al. 2016 pdf | code
• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. Nguyen et al. 2016 pdf | code
• Plug and Play Generative Networks. Nguyen et al. 2017 pdf | code
• Feature Visualization. Olah et al. 2017 url

Real images / Segmentation Masks

• Object Detectors Emerge in Deep Scene CNNs. Zhou et al. 2015 pdf
• Network Dissection: Quantifying Interpretability of Deep Visual Representations. Bau et al. 2017 url | pdf
• Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networks. Fong & Vedaldi 2018 pdf

Heatmaps

• Learning how to explain neural networks: PatternNet and PatternAttribution pdf
• A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations pdf
• A Taxonomy and Library for Visualizing Learned Features in Convolutional Neural Networks pdf

Layer-wise Backpropagation

• Beyond saliency: understanding convolutional neural networks from saliency prediction on layer-wise relevance propagation pdf
• Explaining NonLinear Classification Decisions With Deep Tayor Decomposition pdf

Inverting Neural Networks

• Understanding Deep Image Representations by Inverting Them pdf
• Inverting Visual Representations with Convolutional Networks pdf
• Neural network inversion beyond gradient descent pdf

Bayesian approaches

• Yang, S. C. H., & Shafto, P. Explainable Artificial Intelligence via Bayesian Teaching. NIPS 2017 pdf

Distilling DNNs into more interpretable models

• Interpreting CNNs via Decision Trees pdf
• Distilling a Neural Network Into a Soft Decision Tree pdf

DNNs that learn to explain

• Deep Learning for Case-Based Reasoning through Prototypes pdf
• Unsupervised Learning of Neural Networks to Explain Neural Networks pdf

Others

• Understanding Deep Architectures by Interpretable Visual Summaries pdf