Contrastive Learning for Urban Land Cover Classification With Multimodal Siamese Network

Abstract

The Earth observation era has bestowed dividends upon supervised land cover classification based on deep learning and optical data. However, limitations, such as insufficient spectral information and reduced quality during inclement weather for optical data, coupled with the need for extensive labeled samples, impede accurate classification. This letter harnesses multimodal images with deep contrastive learning to reduce reliance on labeled data and classify land covers. By employing a well-designed contrastive learning method with triangular similarity loss, our model can learn effective multimodal features without labeled samples. Moreover, the learned features are fused at the early feature level and used for the downstream classification task with fewer labeled samples. Experimental results demonstrate the benefits of incorporating multiple modalities, highlighting the potential of combining multimodal image analysis and contrastive learning for land cover classification with limited labeled samples.