Robust Reinforcement Learning: Theory and Algorithms

Abstract

This research dissertation explores novel algorithms in the field of robust reinforcement learning (RL) that address the challenges of controlling dynamical systems in real-world scenarios. Classical reinforcement learning is a powerful sub-field in machine learning for training intelligent sequential decision-making agents in complex environments. However, these algorithms often face challenges when it comes to uncertainties and variations in the environment, as well as the requirement for a large number of training samples. In this work, we present novel robust reinforcement learning algorithms that address these challenges. Our algorithms focus on robustness to uncertainties in the environment through the transition dynamics variations. By leveraging techniques such as distributionally robust optimization, our algorithms aim to learn policies that can withstand these uncertainties. We study robust reinforcement learning in online environment interactions setting as well as when we are given historical without access to the environment. We also study the problems of imitation learning and offline reinforcement learning that is relevant for real-world applications where the goals differ from robust reinforcement learning, but we see the tools of distributionally robust optimization and model pessimism involves crucial roles in helping improve these areas of learning domain. The experimental results demonstrate the effectiveness of our robust algorithms, showcasing their potential for real-world applications where uncertainties and variations are prevalent.