Skip navigation
Please use this identifier to cite or link to this item: http://arks.princeton.edu/ark:/88435/dsp01h415pd86c
Title: The factor structure of reinforcement learning behaviors
Authors: Zorowitz, Samuel
Advisors: NivDaw, YaelNathaniel
Contributors: Neuroscience Department
Keywords: computational psychiatry
psychometrics
reinforcement learning
Subjects: Neurosciences
Cognitive psychology
Quantitative psychology
Issue Date: 2023
Publisher: Princeton, NJ : Princeton University
Abstract: Reinforcement learning (RL) is a cognitive and algorithmic framework that specifies how agents (e.g., humans, animals, machines) can learn reward-maximizing policies through trial-and-error. The burgeoning field of computational psychiatry has demonstrated the relevance of RL for understanding psychiatric conditions including depression, obsessive- compulsive disorder, and substance use. However, the field is currently hampered by many theoretical and psychometric issues that limit the validity, interpretability, and utility of symptom-behavior correlations. In this dissertation, I develop a battery of reliable RL tasks — spanning multiple dimensions of reinforcement learning behaviors including exploration, risk sensitivity, Pavlovian-instrumental biases, and model-based planning — in order to characterize the factor structure of reinforcement learning behaviors. The ultimate aim of this research is to better understand the dimensionality of individual-differences in RL behaviors, and their relationship to cognitive ability and psychiatric symptoms. In Chapter 2, I introduce a novel task measure of differential sensitivity to signed prediction errors that is test-retest reliable and robust against confounding from choice autocorrelation. In Chapter 3, I introduce an improved measure of Pavlovian-instrumental biases that is test-retest reliable and less subject to practice effects. In Chapter 4, I validate the matrix reasoning item bank and use it to construct brief, reliable measures of fluid reasoning. In Chapter 5, I show how inattentive responding by participants can induce spurious symptom-behavior correlations. I then validate a screening method which prevents them. Finally, in Chapter 6, I investigate the factor structure of individual-differences in performance on the RL task battery in a sample of N=975 participants. I show that prior research has underestimated across-task behavioral correlations by neglecting measurement error. I also demonstrate that covariation in RL behaviors are well-described by a hierarchical factor model, in which behavior reflects both a general factor — capturing the overall acuity of participants’ decision-making — and a number of specific factors that reflect participants’ proclivities towards task-specific strategies. I also find that only the general factor correlates with participants’ cognitive ability and psychiatric symptoms. In the conclusion, I interpret the results with respect to the current state of computational psychiatry and discuss a number of challenges for the field moving forward.
URI: http://arks.princeton.edu/ark:/88435/dsp01h415pd86c
Type of Material: Academic dissertations (Ph.D.)
Language: en
Appears in Collections:Neuroscience

Files in This Item:
This content is embargoed until 2024-09-28. For questions about theses and dissertations, please contact the Mudd Manuscript Library. For questions about research datasets, as well as other inquiries, please contact the DataSpace curators.


Items in Dataspace are protected by copyright, with all rights reserved, unless otherwise indicated.