Creating cloud-free satellite imagery from image time series with deep learning

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

10 Citations (Scopus)
164 Downloads (Pure)

Abstract

Optical satellite images are important for environmental monitoring. Unfortunately, such images are often affected by distortions, such as clouds, shadows, or missing data. This work proposes a deep learning approach for cleaning and imputing satellite images, which could serve as a reliable preprocessing step for spatial and spatio-temporal analyzes. More specifically, a coherent and cloud-free image for a specific target date and region is created based on a sequence of images of that region obtained at previous dates. Our model first extracts information from the previous time steps via a special gating function and then resorts to a modified version of the well-known U-Net architecture to obtain the desired output image. The model uses supplementary data, namely the approximate cloud coverage of input images, the temporal distance to the target time, and a missing data mask for each input time step. During the training phase we condition our model with the targets cloud coverage and missing values (disabled in production), which allows us to use data afflicted by distortion during training and thus does not require pre-selection of distortion-free data. Our experimental evaluation, conducted on data of the Landsat missions, shows that our approach outperforms the commonly utilized approach that resorts to taking the median of cloud-free pixels for a given position. This is especially the case when the quality of the data for the considered period is poor (e.g., lack of cloud free-images during the winter/fall periods). Our deep learning approach allows to improve the utility of the entire Landsat archive, the only existing global medium-resolution free-access satellite archive dating back to the 1970s. It therefore holds scientific and societal potential for future analyses conducted on data from this and other satellite imagery repositories.

Original languageEnglish
Title of host publicationProceedings of the 9th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data, BIGSPATIAL 2020
EditorsVarun Chandola, Ranga Raju Vatsavai, Ashwin Shashidharan
Number of pages10
PublisherAssociation for Computing Machinery
Publication date3 Nov 2020
Article number3
ISBN (Electronic)9781450381628
DOIs
Publication statusPublished - 3 Nov 2020
Event9th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data - BIGSPATIAL 2020 - San Diego / Virtuel, United States
Duration: 3 Nov 20206 Nov 2020

Conference

Conference9th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data - BIGSPATIAL 2020
Country/TerritoryUnited States
CitySan Diego / Virtuel
Period03/11/202006/11/2020

Keywords

  • image reconstruction
  • remote sensing
  • satellite imagery

Cite this