Deploying autonomous agents (AGs) on edge devices to explore unfamiliar environments is challenging. Indeed, when deploying multiple AGs to navigate unknown environments, a major challenge arises because pre-trained models for navigation, such as monocular depth estimation, may not perform well in such new settings. Therefore, it is essential to continuously adapt the pre-trained models to the new environment through ongoing learning processes. Additionally, the AGs need to make real-time predictions to navigate effectively in the new environment. While federated learning (FL) can help speed up the learning process by utilizing multiple AGs, there is uncertainty about how to efficiently manage simultaneous training and inference on edge devices.

To tackle these complex challenges, we propose a hardware/software online training and inference (OTI) framework, which we distribute using FL. This framework allows for efficient simultaneous learning and inference on edge devices from continuous data streams. We then prototype and validate this FL framework on multiple edge devices, demonstrating real-time FL using OTI alongside monocular depth estimation models.

Team Members:

Afnan Mir
Chloe Tang
Joon Song
Prithvi Senthilkumar
Tiani Chen-Troester
Vincent Liew