Analysis and Understanding of Document Images in Network Media – AUDINM

Fast image categorization , Image database, Method for fast categorization of Web images
Scene text detection: Text confidence computation, Text component verification
Layout analysis and graphics recognition: Method(s) for multiple layer separation

Fast image categorization (WP1) – Partner: NLPR
o Image database – Deliverable 1.1 as a database and a publication – Month 12
o Method for fast categorization of Web images – Deliverable 1.2 as a publication and a software prototype – Month 12
Scene text detection (WP2) – Partner: NLPR (with collaboration from L3i)
o Text confidence computation – Deliverable 2.1 as a publication and a software prototype – Month 12
o Text component verification using CRF – Deliverable 2.2 as a publication and a software prototype – Month 18
Layout analysis and graphics recognition (WP4) – Partner L3i
o Method(s) for multiple layer separation – Deliverable 4.1 as a publication and a software prototype – Month 12

Continue research and development in WP2 (text detection), and work on WP3 (text recognition) with focus on deep learning techniques.
During research visits, enforce collaborative work between the two partners in terms of co-authored publications.

- Many publications (cited in our report) - both conference and journal publications
- Image database

There is a huge growth in the amount of multimedia data on social network media. With such large data collections of weakly structured content, current information retrieval methods face difficulties in mining such data. The objective of the proposed project is to develop a system for mining and retrieval of heterogeneous documents, mainly focusing on weakly structured documents such as born-digital documents and scene images with embedded text. Analyzing the contents of such documents is very challenging because of complex background, complex layout, perspective distortion, lighting variance, defocus, variation of font type, size and color, mixed graphics and text, multi-languages within the same text and sometimes low resolution.

The research plan of the proposed system is composed of complementary parts that finally form a pipeline of a complete system. First, different image types are received as input; they will be classified by the “fast image categorization” part. Then, scene images will be analyzed by the “scene text detection and extraction” part, whereas born-digital documents will be analyzed by the “layout analysis and page segmentation” part. The text extracted from different images types from the previous two parts will be analyzed by the “multi-lingual text recognition” part. Finally, the “contextual interpretation and information integration” part will combine the information analyzed from the previous parts and integrate them in order to reach a meaningful representation of the document database.

The two project partners will collaborate on solving the different problems in accordance with their respective expertise. The expected research achievements in analyzing the contents of document images in network media will provide research experience and visibility to the partners, and will be very useful for different social applications -- such as interactive tourists guidance --, cyber security and commercial data mining.