Skip navigation
Please use this identifier to cite or link to this item:
Title: Beyond simple model-free reinforcement learning in human decision making
Authors: Solway, Alec
Advisors: Botvinick, Matthew M
Contributors: Neuroscience Department
Subjects: Neurosciences
Issue Date: 2014
Publisher: Princeton, NJ : Princeton University
Abstract: Over the last two decades, there has been a large scale effort in cognitive neuroscience to understand learning and decision making from the perspective of simple model-free reinforcement learning algorithms. This interest was invigorated in the mid 1990's, when it was realized that the phasic activity of midbrain dopaminergic neurons resembles reward prediction errors. The algorithms studied formalize the notion of learning from past experiences through trial and error. Although important, there are many aspects of behavior they cannot explain. More recent work has begun to fill in some of these gaps by borrowing yet additional ideas from computational reinforcement learning. One line of inquiry has concentrated on aligning goal-directed behavior, which resembles the common sense notion of &ldquo;planning&rdquo;, with <italic>model-based</italic> reinforcement learning. This work has aimed to understand how the brain is able to learn the world model prescribed by the model-based framework, and to characterize the neural correlates of the value functions it predicts. This thesis adds to this work by offering two separate, but related, algorithmic accounts of how the brain may be able to actually map the world model into a decision. Existing data are examined and new experiments are performed. A second line of inquiry has concentrated on understanding behavior from the perspective of <italic>hierarchical</italic> reinforcement learning. The thesis makes two contributions to this area as well. First, it is shown that the brain codes pseudo-reward prediction errors, a prediction error in response to a faux reward signal that is used to train skills that are not in themselves useful, but that may be used to achieve other means. Second, an optimality framework is provided for understanding which skills are most beneficial to have when confronted with an ensemble of tasks.
Alternate format: The Mudd Manuscript Library retains one bound copy of each dissertation. Search for these copies in the library's main catalog
Type of Material: Academic dissertations (Ph.D.)
Language: en
Appears in Collections:Neuroscience

Files in This Item:
File Description SizeFormat 
Solway_princeton_0181D_10921.pdf17.23 MBAdobe PDFView/Download

Items in Dataspace are protected by copyright, with all rights reserved, unless otherwise indicated.