Progressive spatiotemporal image fusion with deep neural networks

Abstract

Spatiotemporal image fusion (STIF) provides a feasible and effective solution for generating satellite images with high spatial and temporal resolution. As deep learning-based fusion algorithms show great potential in generating high-quality images, we propose a novel deep learning model, namely a deep progressive spatiotemporal fusion network (DPSTFN), which is coupled with pansharpening and super-resolution learning processes to satisfy requirements of STIF based on Moderate Resolution Imaging Spectroradiometer (MODIS) and Landsat data. First, a pansharpening process is adopted to make full use of two MODIS bands with 250 m spatial resolution. Second, a super-resolution process enhances the spatial information that existed in coarse-resolution images to alleviate the enormous spatial resolution gap between MODIS and Landsat images. Third, combining the aforementioned two auxiliary processes, a progressive spatiotemporal fusion framework is proposed to generate deliberate and robust fusion results. Experiments are conducted using two MODIS-Landsat datasets of distinctive landforms to evaluate the performance of DPSTFN. The results of the subjective and objective evaluation show that our proposed network performs better than the state-of-the-art traditional STIF algorithms Fit-FC and RASTFM, and the deep learning-based algorithms EDCSTFN and StfNet.