Skip navigation
Please use this identifier to cite or link to this item:
Title: Metacognition: toward a computational framework for improving our minds
Authors: Krueger, Paul
Advisors: Griffiths, Thomas L
Contributors: Computer Science Department
Keywords: Bounded Rationality
Reinforcement learning
Resource rationality
Risky choice
Subjects: Cognitive psychology
Artificial intelligence
Issue Date: 2023
Publisher: Princeton, NJ : Princeton University
Abstract: In this dissertation I will show how reinforcement learning (RL) can be applied to the inner workings of cognition. The usual application of RL is to understand human behavior or build intelligent machines interacting in the external world. The same RL formalism can be inverted onto cognitive processes themselves, resulting in a normative account of how to explore and select mental computations, referred to as metacognitive RL. This framework can 1) be used to generate observable behavioral predictions, 2) provide a resource-rational benchmark for both assessing and improving cognition, and 3) motivate cognitive process models based on interacting RL systems. The formalism of metacognitive RL rests on meta-level Markov Decision Processes (meta-MDPs), which provide a general-purpose computational framework that can also make task-specific predictions. The first study applies the resource-rational framework to risky choice resulting in the identification of heuristics and accurate predictions about how people adapt their use of heuristics. The second study uses the same metacognitive RL framework to predict which structures of the environment will enhance metacognitive learning in humans during a planning task. In the third study, rather than manipulate the decision environment---which is often infeasible to do in the real-world---the metacognitive RL framework is used to produce feedback in the form of \textit{pseudorewards}, resulting in faster metacognitive learning in a related planning task. Next, a new cooperative RL architecture is proposed. This approach also uses pseudorewards to promote learning, but rather than generate the pseudorewards from a computational model, it is proposed that they may be produced internally by a distinct RL system. The successful application of the metacognitive RL framework to understand and improve cognitive function depends critically on developing machine learning methods to solve these problems. In the final chapter, I briefly explore the application of a recently proposed machine learning method for solving meta-MDPs.
Type of Material: Academic dissertations (Ph.D.)
Language: en
Appears in Collections:Computer Science

Files in This Item:
File Description SizeFormat 
Krueger_princeton_0181D_14417.pdf16.36 MBAdobe PDFView/Download

Items in Dataspace are protected by copyright, with all rights reserved, unless otherwise indicated.