Papers on Understanding and Explaining Neural Networks

This is an on-going attempt to consolidate all interesting efforts in the area of understanding / interpreting / explaining / visualizing neural networks.

---

1. GUI tools

• Deep Visualization Toolbox code | pdf

2. Feature Visualization / Activation Maximization

• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks code | pdf
• Plug and Play Generative Networks pdf | code

3. Heatmap / Attribution

• Learning how to explain neural networks: PatternNet and PatternAttribution (pdf)

Layer-wise Backpropagation

• Beyond saliency: understanding convolutional neural networks from saliency prediction on layer-wise relevance propagation pdf
• Explaining NonLinear Classification Decisions With Deep Tayor Decomposition pdf

4. Bayesian approaches

• Yang, S. C. H., & Shafto, P. Explainable Artificial Intelligence via Bayesian Teaching. NIPS 2017 pdf

5. Distilling DNNs into more interpretable models

• Interpreting CNNs via Decision Trees pdf
• Distilling a Neural Network Into a Soft Decision Tree pdf

6. DNNs that learn to explain

• Deep Learning for Case-Based Reasoning through Prototypes pdf

7. Others

• Understanding Deep Architectures by Interpretable Visual Summaries pdf