An interactive system for 3D spatial relationship query by integrating tree-based element indexing and LLM-based agent

Abstract

<p>The spatial relationships of building information modeling (BIM) elements are essential to support various applications (e.g., compliance checking, path planning). While the large language models (LLMs) have shown promise in querying the BIM spatial relationship in an efficient and user-friendly manner, three critical challenges persist: extracting the complex spatial geometric information is error-prone, processing large number of elements is low efficient, and the exploitation integrating LLM and BIM spatial query task is still inadequate. Addressing these challenges, this paper develops an interactive natural language spatial query system based on the LLM-based agent system, with three major contributions: (1) an openBIM standards-based algorithm to extract complex geometric information; (2) a two-level spatial index to improve search efficiency; (3) an LLM-based multi-agent collaboration framework to deeply integrate LLM and spatial query task. Our proposed query system is verified in the case study with three BIM models. In our case study, our query can successfully extract geometric information from BIM models with complex layout to facilitate answering spatial query. The spatial index in our query system can significantly improve the efficiency of element search, which reduce 70% of average total query time. Furthermore, our query system shows 92.1% correctness rate in query understanding test. With such high understanding performance, the multi-agent system can decompose the spatial query task and assign it to LLM agents with different functionalities to cooperatively complete the spatial query task, which enhances the applicability of the query system.</p>