Encoding the Ising Model as a Sequence Generator using Recurrent Neural Networks

Abstract

Analysis of temporal sequences and their correlations are essential to almost all fields of science. Maximum entropy approaches from the field of statistical mechanics allow one to view such temporal sequences quite generally as being generated from the equilibrium statistical mechanics of lattice spin models such as the Ising model. Recent popularization of machine learning techniques in science and elsewhere have illustrated the power of artificial neural networks to model and generate temporal data with sophisticated structure, such as natural language. In this thesis, I conduct a theoretical and numerical study of a simple recurrent neu- ral network (RNN) and its ability to encode the structure of the Ising model viewed as a generator of sequence data in dimensions d = 1 and d = 2. One-dimensional sequences are taken from Ising model data and used to train a recurrent neural network, and the network is capable of successfully learning the distribution associated with the d = 1 model with k-th nearest neighbor couplings for k up to 3. Moreover, the RNN learns an efficient encoding wherein its parameters are tuned such that the state space of the network’s hidden vari- ables is discretized. Transitions between the discrete states in this space are used to encode relevant features of the input sequence, and numerical evidence is found suggesting that a small RNN with only three parameters can learn a rather sophisticated encoding capable of distinguishing k > 1 models from k = 1 models with large correlation lengths. The RNN struggles to encode the d = 2 Ising model, particularly at criticality. The solution it does RNN converges to suggests the discretization of state space does not suffice to capture the features of the critical model, suggesting that higher-dimensional RNNs are not just practically useful but in fact required.