Joint calibration of intrinsic and extrinsic parameters for LiDAR-camera system in targetless environments

Abstract

In the realm of robotic perception, LiDAR-camera fusion has gained increasing research attention in recent years. Essential to this fusion process is the precise calibration of the intrinsic and extrinsic parameters of the sensor system. Although traditional target-based calibration methods are known for their accuracy, the dynamic nature of real-world applications often necessitates on-site, target-free calibration solutions. Despite its critical importance, targetless calibration for LiDAR-camera systems remains a formidable challenge, typically resulting in larger errors when compared to target-based methods. This underscores an ongoing need within the research community for the development of innovative and more efficient calibration techniques to surmount these obstacles.

In this thesis, a novel targetless method is proposed for joint intrinsic and extrinsic calibration of LiDAR-camera systems. The method leverages LiDAR point cloud measurements from planes in the scene, alongside visual points derived from those planes. The core novelty of the method lies in the integration of visual bundle adjustment with the registration between visual points and LiDAR point cloud planes, which is formulated as a plane-constrained bundle adjustment problem. This unified formulation achieves concurrent intrinsic and extrinsic calibration, while also imparting depth constraints to the visual points to enhance the accuracy of intrinsic calibration. Experiments are conducted on both self-collected dataset and public data sequences. The results showcase that the approach not only surpasses other state-of-the-art methods, but also maintains remarkable calibration accuracy even within challenging environments. For the benefits of the robotics community, the calibration software is open-sourced.