Task-Driven Perception and Control for Robust and Efficient Autonomy

Abstract

Modern robotic applications have been propelled by exciting advancements in rich sensor technologies such as LiDAR and RGB-D cameras. However, when we deploy our robots in real environments using these sensors, we see them collide with objects or perform unexpected actions. These failures are often occurring for two reasons: (i) the high-dimensional sensors provide rich information that lead to the downstream control policy being sensitive to task-irrelevant distractors in the environment (e.g., changes in lighting conditions), and (ii) we deploy the robots without formal safety and performance assurances, specifically assurances that account for perception uncertainty, for operating in new environments. We break this dissertation into three parts to address these challenges. We first advocate for a task-driven perception and control design paradigm that aims to find minimalistic representations of the environment that are sufficient for the robot to complete its given task. We demonstrate in the first two parts that such a design paradigm affords robustness to task-irrelevant distractors in the environment and computational efficiency for the robot's control policy. In particular, we explore a memory-based and an attention-based perspective to this design paradigm. In the concluding part, we examine the importance of deploying robots with safety and performance assurances and demonstrate that formal assurances help drive empirical improvements to safety (e.g., reduced number of collisions in new environments). Throughout all of the parts, a central focus revolves around enhancing robot performance for real settings. As such, we showcase the majority of our proposed methodologies through various tracking and navigation tasks on a physical quadruped robot using either LiDAR or RGB-D cameras.