Understanding unstructured 3D point clouds for creating digital twin city: An unsupervised hierarchical clustering approach

Abstract

Digital twin city (DTC) is a critical information infrastructure that enables many innovative applications for smart and resilient city development. Thanks to the recent advances in remote sensing and photogrammetry, accurate, dense, and large-scale 3D urban point clouds become increasingly available for many cities for creating and updating their DTCs. Because of the immense amount and the high update frequency of urban point clouds, it is too time-consuming and labor-intensive to create and update DTCs solely by human experts. Researchers have developed a wealth of automatic and semi-automatic methods for processing 3D urban point clouds using expert knowledge of the built environment, supervised learning, and reinforced learning of geometric primitives and components. However, these methods are restricted, ironically by the embedded knowledge, in the scalability to sophisticated scenes and the availability of standardized components. Inspired by the success of Google’s unsupervised learning program AlphaZero, this paper proposes a novel hierarchical clustering approach for semantic enrichment of point clouds. Unlike the existing approaches relying on fixed domain knowledge, extra correlational training examples, or available 3Dreferences, the proposed approach exploits the similarities between patches of point clouds without explicit domain knowledge. The proposed approach first segments patches from the input point cloud through the connected subgraphs of voxel grids, then computes the dissimilarity matrix between the patches via iterative optimization. Subsequently, the dissimilarity engenders a hierarchy of clusters for understanding the relatedness between the patches. A pilot study on a real urban scene showed that the proposed approach is feasible and potent to cluster and detect objects automatically. Another experiment showed that the dissimilarity-based clusters and associated transformations can help create semantic objects for DTC, as referential 3D models are available.