Please use this identifier to cite or link to this item:
http://arks.princeton.edu/ark:/88435/dsp011544bs46d
Title: | Maximum Entropy, Symmetry, and the Relational Bottleneck: Unraveling the Impact of Inductive Biases on Systematic Reasoning |
Authors: | Segert, Simon |
Advisors: | Cohen, Jonathan |
Contributors: | Neuroscience Department |
Subjects: | Neurosciences Cognitive psychology |
Issue Date: | 2024 |
Publisher: | Princeton, NJ : Princeton University |
Abstract: | A major outstanding challenge in cognitive science is to understand how people perform extrapolations generalizations that are far outside the context of their previous experiences. There has been longstanding controversy regarding the ability of connectionist (i.e., neural- network based) models to attain such behavior in the absence of explicit symbolic primitives. In this thesis, we argue for an alternative perspective that neural networks can learn to behave as if they were implementing a symbolic procedure, provided that they are endowed with certain general inductive biases that shape and constrain the learning process (crucially, these biases do not involve the explicit imposition of any symbolic structure). Specifically, we consider three inductive biases that are deeply motivated by previous work in psychology: The Maximum Entropy Principle, the Relational Bottleneck, and Symmetry. Said in brief, the Maximum Entropy principle prescribes that an agent ought to disperse its internal representations to the fullest possible extent, while Symmetry refers to propensity to attend to repetitive or self-similar structure within the environment, and the Relational Bottleneck refers to a certain architectural constraint which encourages a model to attend exclusively to higher-order relational information at the expense of lower-order perceptual details. In a series of studies, we characterize the effect of multiple combinations of these three learning biases across a wide range of problem domains (analogical reasoning, function learning, and arithmetic). Taken together, our results demonstrate that these biases in combination can effectively enhance the out-of-domain generalization abilities of neural network models, and consequently, due to the psychological plausibility of the inductive biases, point towards a deeper understanding of how such capabilities may be implemented in the human mind. |
URI: | http://arks.princeton.edu/ark:/88435/dsp011544bs46d |
Type of Material: | Academic dissertations (Ph.D.) |
Language: | en |
Appears in Collections: | Neuroscience |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
Segert_princeton_0181D_14980.pdf | 5.28 MB | Adobe PDF | View/Download |
Items in Dataspace are protected by copyright, with all rights reserved, unless otherwise indicated.