Spatio-Temporal Analysis by Recognition within Complex Images for Remote Sensing of Environment – ASTERIX

The ASTERIX project led to significant developments of several recent paradigms in data analytics and processing. Morphological hierarchies provide a framework tailored for fast remote sensing image analysis, on which can be applied kernel-based learning methods. Gaussian mixture models allows us to select relevant features to better represent complex data. Combining segmentation or classification results by relying on their uncertainty brings solutions for multi-source data. Non-negative matrix factorization methods can be employed in manifold learning for solving problems such as feature selection, anomaly detection, classification or spectral unmixing. Learning of time series can be obtained with active methods or bag-of-words approaches, and their reconstruction achieved with data assimilation techniques. Optimal transport provides an adequate framework for domain adaptation, while dimension reduction can lead to anomaly detection.

The ASTERIX project contributed to state-of-the-art progress in data science and its applications to Earth observation, with better methods in terms of results accuracy and/or (memory and computation) resource consumption, as well as the ability to process complex data still underexploited. In the context of new Earth Observation programs such as Sentinel, the targeted applications are numerous and various: land cover and land use mapping, biomass estimation, urban monitoring, etc. These applications will be further fostered by the Kermap start-up.

During this project, the deep learning paradigm widely spreads in machine learning, computer vision but also remote sensing. Far from overcoming the developments made in the ASTERIX project, the proliferation of deep learning calls for a coupling with other paradigms that have been mastered by the consortium during the project (e.g. hierarchical representations, optimal transport, time series analysis, etc.). In this context, the ASTERIX partners have recently engaged in various projects aiming to explore such an interface between different paradigms. While deep learning allows some significant improvements in recognition tasks in remote sensing, it still faces many challenges such as the data complexity, the need for training data, or the simplicity of tasks considered until now. Addressing such challenges is the future step of the works initiated in the ASTERIX project.

The ASTERIX project led to numerous international publications, with 21 papers in journals and 32 communications. Its results have been disseminated in several communities: machine learning (1 paper in Machine Learning journal and 4 in ECML-PKDD), pattern recognition (including ACCV and ICIP), and mostly remote sensing (including 4 papers published in IEEE TGRS, 5 in IEEE JSTARS, 2 in Remote Sensing journal, 2 in ISPRS IJGI; 7 papers in IGARSS, 4 in WHISPERS, 5 in BiDS/IIM, and 2 in GEOBIA).

Full list available on the project web page: anr-asterix.irisa.fr

Following the growth of multisource data with high spatial, spectral, and temporal resolutions, the problem of complex image information mining in remote sensing of environment becomes a great challenge, with many potential applications raising. However, there is no or only a few methodological frameworks for dealing with data with multiple spatial and temporal scales: recognition methods are most often straight applications of standard classification and modelisation methods. Besides, dealing with spatial and temporal neighborhood, with various kinds of data, is expected to improve significantly resulting performances.

The goal of the ASTERIX project (Spatio-Temporal Analysis by Recognition within Complex Images for Remote Sensing of Environment) and its originality is to bring new methods, algorithms, softwares in the field of image analysis and machine learning in order to support recognition within complex image, by explicitly dealing with the specificity of remote sensing complex images. In this context, main challenges are related to high dimensionality, heterogeneity, volume and spatio-temporal behaviour of images.

Besides methodological achievements supporting the development of the state-of-the-art in image processing and machine learning in the context of recognition within complex images, expected results from the ASTERIX project consist in a set of concrete solutions to crucial problems in remote sensing of environment, and especially in two environment: coastal and montains. More precisely, applications considered are related to the dynamic of environmental objects which help to understand coastal evolution, the dynamic of ash tree colonization in an agricultural mountain landscape, and the dynamic of geological process.