Robotic Cell Micromanipulation Skill Learning via Imitation-Enhanced Reinforcement Learning

Abstract

Humans can learn complex and dexterous manipulation tasks by observing videos, imitating and exploring. Multiple end-effectors manipulation of free micron-sized deformable cells is one of the challenging tasks in robotic micromanipulation. We propose an imitation-enhanced reinforcement learning method inspired by the human learning process that enables robots to learn cell micromanipulation skills from videos. Firstly, for the microscopic robot micromanipulation videos, a multi-task observation (MTO) network is designed to identify the two end-effectors and the manipulated objects to obtain the spatiotemporal trajectories. The spatiotemporal constraints of the robot's actions are obtained by the task-parameterised hidden Markov model (THMM). To simultaneously address the safety and dexterity of robot micromanipulation, an imitation learning optimisation-based soft actor-critic (ILOSAC) algorithm is proposed in which the robot can perform skill learning by demonstration and exploration. The proposed method is capable of performing complex cell manipulation tasks in a realistic physical environment. Experiments indicated that compared with current methods and manual remote manipulation, the proposed framework achieved a shorter operation time and less deformation of cells, which is expected to facilitate the development of robot skill learning.