Monitoring street-level improper dumpsites via a multi-modal and LLM-based framework

Abstract

Effective monitoring and management of urban improper dumpsites have become increasingly critical due to the rising volumes of solid waste and their adverse environmental and public health impacts. Identifying the locations and types of street-level dumpsites is a necessary first step for waste management; however, existing studies lack automated and accurate methods for detecting and categorizing these sites. As a result, governments face substantial labor and financial burdens in managing illegal dumping. To address these gaps, this study presents MultiSense DumpSpotter, a novel cascade model framework that integrates a multimodal deep learning architecture with Large Language Models (LLMs) to identify, classify, and analyze improper dumpsites with greater accuracy than traditional unimodal vision models. To support this framework, we developed UrbanDumpSight, the first annotated street-level urban dumpsite dataset, consisting of over 4000 street view images with metadata that includes geospatial and demographic information. This study contribute to the literature by demonstrating the effectiveness of multimodal data fusion in urban studies and the potential of LLMs in interpreting urban semantics. From a practical standpoint, it introduces a deployable, user-friendly system designed to meet the needs of urban managers, enabling efficient monitoring of improper dumping hotspots, uncovering root causes, and facilitating the implementation of effective governance actions. Overall, this research provides a novel and scalable solution for addressing urban waste challenges, offering insights to support sustainable waste management and policy-making.