Multi-Agent Sequential Decision Modeling for Information Collection and Intervention in Epidemics and Wildfires

Abstract

This thesis develops a multi-agent extension to the unified framework for sequential decision problems. We introduce concepts such as interaction layers, and belief models, and tie them back into a unified modeling framework for sequential decision problems. The interaction layers break down the interactions between agents into physical processes and informational processes. The belief model captures the uncertainty about the state of the world and outlines how to structure a probabilistic mathematical model to capture the complexities associated with the subjective uncertainty of each agent. The multi-agent extension is structured in such a way to decompose the complex multi-agent models back into the original five components of the unified framework to converge back into a search over the four classes of policies for each agent. To illustrate the ability of this framework to scale to real-world problems we design two models for epidemics and wildfires. These problems produce natural multi-agent problems with partial observability, active learning, complex dynamics, and high dimensionality across the state, action, and observation spaces. The agents in these problems will learn about the environment by collecting valuable information, implementing decisions which directly change the environment, and interacting with each other. The interaction layers, complex belief models, and robust policies would be extremely challenging to implement with any existing multi-agent modeling frameworks from the literature including POMDPs, MARL, or game theory. After the formal mathematical modeling, we demonstrate results from python simulations using real data from case studies in each of the application domains. In both applications, the structure of the problems leads to classes of parameterized direct lookahead approximation policies which perform best.