Article: End-to-End Learned Early Classification of Time Series for In-Season Crop Type Mapping
The International Society for Photogrammetry and Remote Sensing publishes a monthly journal that serves as a communication channel for scientists and professionals working in disciplines involving photogrammetry, remote sensing, spatial information systems and computer vision. Our Data Intelligence team recently published a paper which presents a deep time series classification model applied to satellite images, offering early predictions while enhancing their accuracy.
Machine learning is widely used for making predictions, including in the context of time series data. As a practical example, we can consider the classification of crop types for large-scale agricultural monitoring and yield prediction. From yearly multi-spectral satellite images it is indeed possible to predict the type of crop cultivated in a given field, information that is particularly useful for post-hoc statistics and analysis.
A less common machine learning task consists in “early classification” of time series, where the objective is not only to make an accurate prediction (here of the crop type) but also to make it as early as possible. For instance, a rapeseed (colza) field will exhibit a strong yellow color during its flowering phase, typically occurring from April to mid-May. The prediction of the crop type can therefore be made early and accurately, so that the automated system can be used not only for post-hoc analysis but also for continuous monitoring and early decision making.
In this article, we introduce a generic approach for early time-series classification. Our method builds on any existing sequential model and augments it so that it learns to predict, at each time step, both a class distribution and a stopping probability. This stopping probability can be viewed as a measure of the model's confidence. We evaluate and illustrate the effectiveness and efficiency of our approach using existing crop classification datasets.
Figure 1 below:
Illustration of the ELECTS method
Figure 2 below:
llustration of the model process: class labels predictions and stoppage of individual fields, for a deployment setting over one year.
Abstract
Remote sensing satellites capture the cyclic dynamics of our Planet in regular time intervals recorded in satellite time series data. End-to-end trained deep learning models use this time series data to make predictions at a large scale, for instance, to produce up-to-date crop cover maps. Most time series classification approaches focus on the accuracy of predictions. However, the earliness of the prediction is also of great importance since coming to an early decision can make a crucial difference in time-sensitive applications.
In this work, we present an End-to-End Learned Early Classification of Time Series (ELECTS) model that estimates a classification score and a probability of whether sufficient data has been observed to come to an early and still accurate decision. ELECTS is modular: any deep time series classification model can adopt the ELECTS conceptual idea by adding a second prediction head that outputs a probability of stopping the classification. The ELECTS loss function then optimizes the overall model on a balanced objective of earliness and accuracy. Our experiments on four crop classification datasets from Europe and Africa show that ELECTS allows reaching state-of-the-art accuracy while reducing the quantity of data massively to be downloaded, stored, and processed.
Download the source code here.
Read the full article here.