Adaptive micro-locomotion in a dynamically changing environment via context detection

Abstract

<p>Substantial efforts have exploited reinforcement learning (RL) in the development of micro-robotic locomotion. These RL-powered micro-robots are capable of learning a locomotory policy based on their experience interacting with the surroundings, without requiring prior knowledge on the physics of locomotion in that environment. However, in their applications, micro-robots often encounter changes in the environment and need to adapt their locomotory gaits like living organisms in order to achieve robust locomotion performance. In standard RL methods, such a non-stationary environment can cause the micro-robots to continuously relearn the policy from scratch, degrading their locomotion performance. In this work, we explore a first use of a recently developed context detection method combined with deep RL to facilitate micro-robotic locomotion<br>in a dynamically changing environment. As a proof-of-principle, we consider a simple micro-robot immersed in non-stationary environments switching between a viscous fluid environment and a dry frictional environment. We show that the RL with context detection approach enables the micro-robot to effectively detect changes in the environment and deploy specialized locomotory gaits for different environments accordingly<br>to achieve significantly improved locomotion. Our results suggest the integration of deep RL with context detection as a potential tool for robust micro-robotic locomotion across different environments.</p>