Please use this identifier to cite or link to this item: http://arks.princeton.edu/ark:/88435/dsp01vm40xv717
 Title: Modern Reinforcement Learning Techniques to Deal with Large Action Spaces Authors: Hervieux-Moore, Zachary Thomas John Advisors: Kornhauser, Alain Contributors: Operations Research and Financial Engineering Department Keywords: action spaceAlphaZeroGAILmonte carlo tree searchreinforcement learingRORY Subjects: Computer science Issue Date: 2021 Publisher: Princeton, NJ : Princeton University Abstract: When developing reinforcement learning algorithms, the main issues are dealing withlarge state spaces and action spaces. For the most part, the state space complexity problem was solved with the advent of AlphaZero. AlphaZero is able to deal with unfathomably large state spaces by using a combination of neural networks and Monte Carlo tree search (MCTS). However, dealing with large action spaces remain an active area of research. We generalize the AlphaZero algorithm by introducing the GAIL framework and test a variety of alterations. We find that using Thompson Sampling as a selection procedure during the MCTS could potentially improve upon AlphaZero in two-player zero-sum games. However, AlphaZero is extremely competitive with all variations. We then show the strength of GAIL by applying it to the game of Scrabble which AlphaZero cannot be applied to due to its extremely large action space. Furthermore, GAIL coupled with the Upper Confidence Bound selection procedure and information set MCTS proves to be state of the art in the game of Scrabble. This also establishes that using information set MCTS can be used with a neural network value estimator in reinforcement learning. Finally, we extend these results to the continuous action space domain by developing ROAR. A novel algorithm that drastically lowers the action space complexity by making a finite number of action recommendations based on state context and historical performance via reinforcement learning. We end by successfully training it in a nontrivial robot problem. URI: http://arks.princeton.edu/ark:/88435/dsp01vm40xv717 Alternate format: The Mudd Manuscript Library retains one bound copy of each dissertation. Search for these copies in the library's main catalog: catalog.princeton.edu Type of Material: Academic dissertations (Ph.D.) Language: en Appears in Collections: Operations Research and Financial Engineering

Files in This Item:
File Description SizeFormat