AN ABSTRACT ARGUMENTATION-BASEDSTRATEGY FOR READING ORDER DETECTION

IT@LIA 2015 - 1st AI*IA Workshop on

Intelligent Techniques At LIbraries and Archives

Stefano Ferilli and Andrea Pazienza

Tuesday 22th September 2015

Overview

1. Introduction

2. Document Processing: DoMInUS

3. Abstract Argumentation Framework

4. Argumentation-based Reading Order Detection

5. Evaluation

6. Conclusions

2

Introduction

Introduction

Document Image Analysis (DIA): branch of AutomaticDocument Processing that aims at extracting high-level

information from the low-level representation of a document.

Reading Order Detection (ROD): hot problem and new

approaches are needed to tackle di�cult cases.

# Use of an Abstract Argumentation Framework (AF) to

solve this problem.

4

Document Processing: DoMInUS

DoMInUS

DoMInUS (DOcument Management INtelligent Universal

System): intensive exploitation of intelligent techniques in each

step of document processing.

Any document can be progressively partitioned into a

hierarchy of abstract representations, called its layout structure.

Several techniques to extract the high-level geometrical

structure of a document.

6

DoMInUS

Input: Vectorial description of each document page in terms of

blocks.

Output: set of frames, de�ned as collections of basic blocks.

7

Abstract Argumentation Framework

Argumentation Basics

Abstract structure AF � 〈A,R〉 in which

# A represents a set of abstract arguments# R represents an attack binary relation on A between

arguments

Directed graph representation, in which each node representing

an argument and each edge representing an attack.

The objective is to determine which subset(s) of its nodes can be

justi�ed.

αβ

γ

δ

ε

Figure: Graph representation of an AF

9

Extension-based Semantics

The justi�cation state in an AF can be determined according to

suitable semantics.

# Determine which subset (extension-based semantics) ofAF’s nodes can be de�ned as ’justi�ed’.

Let S ⊆ A a subset of arguments

# con�ict-free, i.e. @α, β ∈ S s.t. αRβ# acceptable, i.e. ∃F : 2S → 2

Ss.t. F(S) = {α | α is defended by S}

Then

# S is admissible if S ⊆ F(S)

# S is a Complete Ext. if S � F(S)

# S is a Ground Ext. if S is the minimal Complete Ext. (w.r.t. ⊆)

# S is a Preferred Ext. if S is the maximal Complete Ext. (w.r.t. ⊆)

10

Extension-based Semantics

α β γ

δ

ε

Figure: Argumentation Framework Example

# Admissible sets : {∅, {α}, {β}, {β, δ}}# Complete Extensions : {∅, {α}, {β, δ}}# Preferred Extensions : {{α}, {β, δ}}# Ground Extension : {∅}

11

Argumentation-based ReadingOrder Detection

Argumentation-based Reading Order Detection

DoMInUS to obtain the layout structure:

# layout blocks labeled with their type of content,

# image blocks are ignored,

# separators allows to partition the page into independent

portions.

Basic and document-independent reading rules:

# horizontally or vertically adjacent components are

candidates to be read consequently,

# a component at the bottom of the page might be followed

by a component at the top of an adjacent column,

# a rightmost (resp., leftmost) component might be followed

by a leftmost (resp., rightmost) component in an adjacent

row.

13

Argumentation-based Reading Order Detection

Each pair of blocks (A, B) is translated into an argument

representing the claim “components A and B are to be read one

after the other in a document”.

Formally, we express this in First-Order Logic using predicate:

# next/2, as next(A,B) to express reading order,

# attacks/2, as attacks(next(A,B),next(A,C)) toexpress con�icts in Argumentation Framework settings.

Once the Argumentation Framework is build, it is su�cient to

compute an extension to return an acceptable reading order

solution.

14

Argumentation-based Reading Order Detection

The document page in Figure yields the following formal

description:

(0,3), (1,2), (3,1), (4,5), (4,6), (4,7),

(5,6), (5,9), (6,7), (8,0), (8,4), (8,9)15

Argumentation-based Reading Order Detection

The following attacks are automatically derived:

(4,5)-(4,6), (4,6)-(4,5), (4,6)-(4,7), (4,7)-(4,6), (4,5)-(4,7), (4,7)-(4,5),

(5,6)-(5,9), (5,9)-(5,6), (8,0)-(8,4), (8,4)-(8,0), (8,0)-(8,9), (8,9)-(8,0),

(8,4)-(8,9), (8,9)-(8,4), (4,6)-(5,6), (5,6)-(4,6), (4,7)-(6,7), (6,7)-(4,7),

(5,9)-(8,9), (8,9)-(5,9)

16

Argumentation-based Reading Order Detection

The correct reading order is the following:

(3,1), (1,2), (2,4), (4,5), (5,6), (6,7)

17

Evaluation

Evaluation

The proposed technique was tested on a dataset including 103

document pages of di�erent layout complexity.

The justi�ed set of arguments was determined using PreferredExtensions.

For each extension, the recallwas evaluated as the ratio of

correct next/2 items retrieved over next/2 items in the correct

order sequence

Paper Magazine Newspaper Overall

#Documents 43 40 20 103

#Blocks 16.60 11.42 48.20 20.73

#Arguments 24.13 9.20 44.45 22.98

#Attacks 61.67 14.20 164.3 63.17

#Preferred Ext. 6.65 6.83 5 053.85 986.76

Recall(%) 91.32 71.75 70.74 77.94

19

Evaluation

Figure: A very complex document in the dataset

Newspaper pages in the dataset have a signi�cant impact on

the overall complexity of the dataset.

20

Evaluation

Qualitative analysis:

# Very complex non-Manhattan layouts handling,

# Deal with any kind of document independently of the

language in which it is written.

The technique might fail:

# On papers, when header and footer are not separated by

lines;

# On magazines, when multi-line titles are across di�erent

backgrounds;

# On newspapers, when columns of the same article are

separated by lines.

These problems might be tackled by re�ning the rules toconsider additional layout information, such as spacing and

font size.

21

Conclusions

Conclusions

Automatic strategy for identifying the correct reading order

of a document page’s components based on Abstract

Argumentation Framework.

# Unsupervised technique.

# It works on any kind of document based only on general

assumptions about how humans behave when reading

documents.

# Experimental results show that it is very e�ective, alsocompared to previous solutions that have been proposed

in the literature.

# Qualitative analysis of the results suggested possible

directions for further improvement of the approach.

23