Abstract
Federated learning aims to solve a global optimization problem by collectively learning from a group of clients without sharing data they possess. In offline reinforcement learning, an agent aims to learn an optimal policy for its behavior without accessing the environment. While federated algorithms function well in the supervised learning regime, extending such an approach to offline reinforcement learning is non-trivial due to the challenges of learning from heterogeneous data.
This work proposes a Federated Ensemble-Directed Offline Reinforcement Learning Algorithm (FEDORA), which utilizes an ensemble learning approach to gather wisdom from clients possessing heterogeneous data collectively. FEDORA is implemented using the Flower framework to enable its utilization in large-scale federated systems. The algorithm is applied to learn a policy for waypoint navigation and obstacle avoidance from a group of mobile robots with varying expertise levels.
Ragothaman, Nitin Kasshyap (2023). Federated Offline Reinforcement Learning. Master's thesis, Texas A&M University. Available electronically from
https : / /hdl .handle .net /1969 .1 /199114.