Holistically Interpreting Deep Neural Networks via Channel Ablation

Abstract

This paper aims to contribute to the field of convolutional neural network interpretability by analyzing the most significant channels within layers of AlexNet used in making image classifications. We perform various ablation (or "zero-ing out") methods on the layers of AlexNet for a wide variety of images to determine the crucial set of neurons needed to make a correct prediction. We introduce multiple ablation methodologies called top-down and bottom-up ablation that aim to delete the most and least activated channels in a particular model layer. We find that most layers, especially those later in the model, allow for a large amount of ablation of the least significant channels. This points to the fact that object recognition may be specialized to a subset of significant neurons. We also find that ablating this set of crucial neurons has great impact on the model’s prediction for an image, further solidifying the confidence that these channels are indeed highly class specific. In summary, we aim to add to the general knowledge of how the inner-workings of convolutional neural networks function for a particular object recognition task.