Skip navigation
Please use this identifier to cite or link to this item:
Title: Rethinking Prediction Errors in Dopaminergic Signals in the Brain
Authors: Lee, Rachel Stephanie
Advisors: Daw, Nathaniel D
Witten, Ilana B
Contributors: Neuroscience Department
Keywords: Computational Neuroscience
Reinforcement Learning
TD Learning
Subjects: Neurosciences
Issue Date: 2024
Publisher: Princeton, NJ : Princeton University
Abstract: The hypothesis that midbrain dopamine (DA) neurons broadcast an error for the prediction of reward (reward prediction error, RPE) is among the great successes of computational neuroscience (Houk, Adams, and Barto 1995; Montague, Dayan, and Sejnowski 1996; Schultz, Dayan, and Montague 1997). However, modern empirical results have challenged core aspects of this theory, and my dissertation looks in two such datasets: the first of which delved into how DA responses reflect movement correlates that cannot be explained by a RPE (Parker et al. 2016) and the second of which showed how DA might reflect heterogeneous features rather than convey a scalar, global signal (Engelhard et al. 2019). In my dissertation, I build new models and theories to update the classic RL algorithm TD learning in order to account for these new results. In Chapter 2, I investigate dorsal medial striatum (DMS) projecting DA neurons and how they do not reflect RPE with respect to contralateral movement, but contralateral movement directly and the value of the chosen action. In Chapter 3, I introduce a “feature-specific” PE model in order to explain heterogeneous DA responses in the VTA. I argue that the heterogeneity is a reflection of the input state features upstream of the DA neurons, and show how our model can recapitulate how patterns of heterogeneity might arise for reward prediction errors and movement responses. In Chapter 4, I revisit the dataset from Chapter 2 to determine if the movement correlates could be prediction errors with respect to movements rather than reward. While I could not find an action PE with respect to contralateral movement, I am able to show that DMS-projecting DA population at lever presentation does reflect an identity-free action PE, or an “action-surprise” signal. Overall, my dissertation aims to rethink the PE signal in TD learning, showing that classic RPE accounts of DA need to be updated to account for these new, puzzling data. Together, my models show that DA data may be better understood as a more generalized family of PE models, reflecting a richer signal than just a scalar RPE depending on the inputs they receive.
Type of Material: Academic dissertations (Ph.D.)
Language: en
Appears in Collections:Neuroscience

Files in This Item:
This content is embargoed until 2026-02-06. For questions about theses and dissertations, please contact the Mudd Manuscript Library. For questions about research datasets, as well as other inquiries, please contact the DataSpace curators.

Items in Dataspace are protected by copyright, with all rights reserved, unless otherwise indicated.