Logics, structural representations, mathematical morphology and uncertainty for semantic interpretation of images and videos – LOGIMA

Automatic image interpretation has been an active field of research for several years. In this large field, this project focuses on extracting high level information from images or video sequences, when the detection and recognition of structures can benefit from prior structural knowledge (such as spatial interactions). This is in particular the case in video sequences related to a specific context (sport events for instance), in medical imaging (using anatomical knowledge), or in aerial and satellite imaging (man made structures such as airports and towns for instance).
The main objective of this project is thus to extract, analyze and interpret the content (including dynamic content) of visual information supports using structural knowledge and reasoning tools, in order to enrich the visual information with semantics. The breakthrough in this project, at the cross-road of logic-based knowledge representation and reasoning, uncertainty management and spatial reasoning, is to develop a unified lattice-based theory for spatial reasoning under uncertainty with the aim of semantic image interpretation. Based on the general framework of complete lattices and on mathematical morphology, we propose, by exploiting the power of Formal Concept Analysis tools, to extend Description Logics with non-monotonic reasoning tools and with a greater ability to represent complex structural knowledge such as those involved in scene understanding. Furthermore, this proposed new unified framework is intended to represent a priori knowledge in an operational way for image interpretation and to provide reasoning tools which combine imprecise and uncertain logical and numerical reasoning, hence addressing the challenging problem of bridging the gap between symbolic representations and real data. Another original contribution of this project is to introduce bipolarity to handle positive and negative information in the framework. Two other important scientific issues are also addressed in this proposal: dynamic knowledge representation and reasoning in order to consider knowledge as a matter of belief that can evolve both in time and space, and the study of the potential of graph based representations and grammars to model and to solve the computational problem of structural scene recognition in images. The originality of the proposal is not only to provide and develop theoretically this new qualitative and quantitative framework for image interpretation but also to apply and to evaluate it on real data.