Natural Language Information for Meta-Learning Based Tool Manipulation

Abstract

In this paper, we introduce a virtual environment featuring an agent, a handheld tool, and a task that must be completed by manipulating the handheld tool. To populate this environment, we develop a dataset of 3D models for 30 common handheld tools with various geometries.

Next, we create a natural language benchmark for this dataset using the language model GPT-3 (Generative Pre-trained Transformer 3). Each tool is assigned a number of sentences that describe various aspects of the tool. This benchmark establishes a standard set of sentences to train a language model upon for this tool-based environment.

Finally, we implement a meta-RL (Reinforcement Learning) algorithm that adapts the Reptile meta-learning algorithm and applies it to our virtual environment. We base our adaptation on the recent development of an Actor/Meta-Critic framework, and combine it with a natural language neural network trained on our language benchmark.