Deep reinforcement learning for aggressive quadrotor maneuvers

Abstract

Quadrotors are highly agile and versatile aerial vehicles, rendering them ideal for complicated tasks within cluttered environments. The agility of quadrotors is being exploited, posing challenges with escalated performance demands. Meanwhile, reinforcement learning has demonstrated its strong potential of exploiting the robots’ agility. Therefore, this thesis discusses the domain of aggressive quadrotor flights with the deep reinforcement learning method.

Optimization-based agile quadrotor flight methods typically decouple trajectory planning and control. Among these, differential-flatness-based planning and control are widely used for their computational convenience. Leveraging the differential flat property, the author extends the minimum-jerk motion primitive algorithm to minimum snap, formulating closed-loop solutions of minimum snap trajectories. The proposed algorithm achieves rapid generation of one million minimum-snap primitives per second on a standard onboard computer. The method is validated through repetitive acrobatics flights.

In contrast to traditional optimization-based approaches, learning-based methods address the problem by learning an end-to-end policy that predicts control commands directly from high-dimensional observations. This thesis proposes a learning-based method to tackle the specific problem of traversing through unknown, tilted, and narrow gaps for quadrotors. This problem is challenging for two reasons: a precise and robust whole-body planning and control policy is required to accommodate variable-tilt and narrow gaps, and an effective Sim2Real approach is necessary for successful real-world experiments. To this end, the author propounds a novel learning framework for gap traversal flights. The algorithm effectively trains the quadrotor to negotiate gap centers with an approximate tilt attitude to the gap. The policy trained solely in simulation demonstrates domain transfer-ability without necessitating additional training data, while maintaining the success rate. The author further incorporates onboard sensing, eliminating the reliance on precise environmental knowledge, including gap dimensions and orientation. Our proposed framework, encompassing onboard sensing and a neural network controller, attains a success rate of 87.36% in real-world experiments, with gap orientations up to 60°. To the best of our knowledge, this is the first work that performs the learning-based variable-tilted narrow gap traversal flight in the real world, without prior knowledge of the environment.

The inherent under-actuation of conventional quadrotors constrains their agility. To enhance quadrotor agility while preserving conciseness, a novel tilt-rotor quadrotor is introduced lastly. This design incorporates a compact uniaxial rotor tilting mechanism, enabling approximate 6-degree-of-freedom control with only one additional servo motor. The modeling and control framework is derived and then validated through a custom MATLAB simulator. A practical application scenario, horizontal aerial photography around a central objective, is proposed and demonstrated, highlighting the real-world application potential of the proposed tilt-rotor quadrotor. A comparative study over traditional quadrotors is conducted, demonstrating the superiority of the proposed tilt-rotor quadrotor system in aggressive tasks.