Signal Filtering and Classification with Hopfield Networks

Abstract

Hopfield networks as a mathematical model for associative memory have been well studied and used in a variety of applications since their inception by John Hopfield in his 1982 paper "Neural networks and physical systems with emergent collective computational abilities". In the past few years, significant improvements have been made to the storage capacity, retrieval accuracy, and efficiency of these Hopfield networks. This paper aims to highlight the recent developments in Hopfield networks by demonstrating their improved properties and some example use-cases for problems in signal filtering and classification. This was done by building a variety of Hopfield network implementations, testing and verifying their capabilities, and then applying them to settings including audio pitch denoising, MNIST handwritten digit classification, and NSynth musical instrument pitch classification. I was able to demonstrate the improved memory storage and retrieval capabilities of the continuous modern Hopfield network when compared to the simple binary Hopfield network, as well as the exponential in the number of neurons storage capacity of the new continuous Hopfield networks. Furthermore, I achieved practically usable classification accuracy results of 97.0% and 86.1% for the MNIST and NSynth datasets respectively, highlighting that Hopfield networks now present a useful framework to consider for other machine learning problems.