Structured Representations Underlying Efficient Decision-Making

Abstract

Optimal sequential decision-making requires considering a vast number of potential action sequences, growing exponentially with every step into the future. The theoretical difficulty of decision-making is compounded by the resource limitations faced by humans and other animals, such as finite time and memory. While these limitations are often cited as the reason for suboptimal behavior, the emerging framework of resource-rationality adopts the perspective that seemingly-suboptimal behavior is actually adaptively tuned to these limitations and makes rational use of limited resources. This dissertation focuses on two broad approaches humans and other animals take to adaptively simplify their decisions. The first is hierarchical representation, intuitively appealing because appropriate choice of hierarchy can decompose complex tasks into simpler subtasks. We develop a resource-rational framework where subgoals are chosen based on how they can simplify the costly process of planning. We identify novel connections between our framework and alternative accounts in simulations, and find that the predictions of our framework are consistent with human behavior in a large-scale behavioral experiment. In a separate study, we run a process-tracing experiment where participants create hierarchically-structured programs and identify a heuristic bias towards reuse that guides hierarchical representations. The second broad approach we consider are heuristic strategies for reinforcement learning tasks. We propose a framework for strategy inference, where strategies are formulated as programs and evaluated based on their task performance in addition to their complexity. Focusing on simulations of bandit problems, we examine how strategies vary based on the weight given to complexity and their relationship to behavioral signatures in previous research.