Interpretable Machine Learning for the Physical Sciences

Abstract

Would Kepler have discovered his laws if machine learning had been around in 1609? Or would he have been satisfied with the accuracy of some black box regression model, leaving Newton without the inspiration to find the law of gravitation? In this thesis I will present a review of machine learning and its use cases in the physical sciences.I will emphasize a major issue facing their use in science: a lack of interpretability. Overparameterized black box models are susceptible to memorizing spurious correlations in training data. Not only does this threaten reported research advances made with machine learning, but it also deprives scientists of our most powerful toolbox: symbolic manipulation and logical reasoning. With this in mind, I will demonstrate a framework for interpretable machine learning, using physically-motivated inductive biases and a new technique ``symbolic distillation''. The combination of these methods allow a practitioner to translate a trained neural network model into an interpretable symbolic expression. I will first discuss the deep learning strategy for performing this distillation, and then review ``symbolic regression,'' an algorithm for optimizing symbolic expressions using evolutionary algorithms. In particular, I will describe my PySR/SymbolicRegression.jl software package, which is an easy-to-use high-performance symbolic regression package in Python and Julia. Tangential to this, I will discuss several physically-motivated inductive biases which make this technique more effective. In the second half of this thesis, I will review a variety of applications of this and other interpretable machine learning techniques, focusing on problems in astrophysics. These include: cosmology with cosmic voids, sub-grid scale modelling in computational fluid dynamics, optimal telescope time allocation, flexible modeling of population models in stellar and gravitational wave astronomy, and learning effective and probabilistically-rigorous models for planetary instability.