Skip navigation
Please use this identifier to cite or link to this item:
Title: Exploring multi-armed bandit decision-making strategies in an underwater vehicle testbed
Authors: Valverde Lizano, Jonathan
Advisors: Leonard, Naomi
Department: Mechanical and Aerospace Engineering
Class Year: 2016
Abstract: The problem of field estimation, or finding the spatial distribution of a resource in space, has many applications in problems of robotic search. This thesis approaches the problem in the framework of a Gaussian Multi-Armed Bandit (MAB) task, a problem in which an agent must learn about an unknown environment while maximizing expected reward. The arms correspond to discretized points in space, and the smoothness of the field is modeled as spatial correlation between the arms. Up-per Confidence Limit (UCL), an algorithm developed by Reverdy et al. in 2014 for Gaussian MAB problems with correlated arms and prior knowledge, is then applied to this problem. The smoothness of the field is measured by a parameter known as the length scale. In real world applications, the agent can only have an estimate of this length scale. This thesis explores the performance of UCL with correlation in comparison to other algorithms when the estimate of the length scale is correct. The effect of overestimates and underestimates in the length scale is then explored for fields of different smoothness. The search task is finally implemented in a testbed with a robot, giving additional metrics of performance. The simulations showed that knowledge of the spatial correlation of the arms can result in improvements in performance using UCL when compared to other algorithms that do not account for correlation. In addition, it is shown that, in the cases studied, best performance is not actually obtained for the correct length scale estimate, but for some particular overestimate. This effect must be studied further, but the results suggest that an agent performing this search should use an overestimate and not an underestimate of the length scale that describes the field, provided this overestimate is not grossly inaccurate.
Extent: 109 pages
Type of Material: Princeton University Senior Theses
Language: en_US
Appears in Collections:Mechanical and Aerospace Engineering, 1924-2016

Files in This Item:
File SizeFormat 
ValverdeLizano_Thesis.pdf1.5 MBAdobe PDF    Request a copy

Items in Dataspace are protected by copyright, with all rights reserved, unless otherwise indicated.