Persistent Homology of Feed Forward Neural Networks on Discrete Classification Tasks

Abstract

The prevailing of goal of Persistent Homology in Topological Data Analysis is to study the shape of complex datasets by seeing which homological features persist as some continuous parameter ε is changed. This is formalized by constructing simplicial complexes, which are triangulations of topological spaces, in which more simplices are added as ε increases. This then forms a nested collection of simplicial complexes in which homology classes may be born or die. The longer a class persists, the more likely it indicates some underlying shape of the data.

Persistent homology provides a promising path for neural network interpretability as one may understand how a neural network completes a task, or how well it may generalize, based on what its shape is. In this thesis, we first reproduce the results of Naitzat et al. (2020). to show that multilayer perceptrons on simple discrete classification tasks work by changing the topology of the input layer by layer. Specifically, higher dimensional homology groups become trivial while more connected components are formed. Secondly, we seek to analyze how the topology of the neural network itself adjusts as it learns. For the latter we propose a different way of forming a simplicial complex from a fully connected feed forward neural network by applying a semi metric on the nodes.