Embed Size (px)
Citation preview
Metadata improvement for image information retrieval – ISKO 2010 1
Co
log
ne
Un
iver
sity
of
Ap
plie
d S
cien
ces
& p
rom
eth
eus
Kla
us
Lep
sky
– T
hom
as M
ülle
r –
Jens
Will
e
Klaus Lepsky – Thomas Müller – Jens Wille
Metadata improvement
for
image information retrieval
Metadata improvement for image information retrieval – ISKO 2010 2
Co
log
ne
Un
iver
sity
of
Ap
plie
d S
cien
ces
& p
rom
eth
eus
Kla
us
Lep
sky
– T
hom
as M
ülle
r –
Jens
Will
e
Rogier van der Weyden. St. Columba Altarpiece. Adoration of the Magi. Central part. c.1455. Oil on panel. Alte Pinakothek, Munich, Germany
Title: St Columba Altarpiece (central panel)
Author: ROGIER VAN DER WEYDEN
Date: c. 1455
Technique: Oil on oak panel, 138 x 153 cm
Type: religious
Form: painting
Location: Alte Pinakothek, Munich
Triptych with the Adoration of the Magi, so-called Columba Altarpiece, Munich, Alte Pinakothek, inv. nr. WAF 1189-1191. Center panel: ca. 140 x 153, Wings each: ca. 140 x 73 cm.
Problem I
Metadata improvement for image information retrieval – ISKO 2010 3
Co
log
ne
Un
iver
sity
of
Ap
plie
d S
cien
ces
& p
rom
eth
eus
Kla
us
Lep
sky
– T
hom
as M
ülle
r –
Jens
Will
e
Adoration of the magi
St. Columba Altarpiece
Triptych with the Adoration of the Magi
St Columba Altarpiece (central panel)
St. Columba Altarpiece. Adoration of the Magi. Central part
Triptych with the Adoration of the Magi, so-called Columba Altarpiece
Columba Altarpiece Adoration of the Magi. Central part
Rogier van der Weyden
Weyden, Rogier /van
Rogier <van der Weyden>
Pasture, Rogier /de le
Rogier <de LePasture>
Roger <de LePasture>
Rogier <de le Pasture>
Roger <de le Pasture>
Regelet <de le Pasture>
Rogelet <de le Pasture>
Pasture, Rogier /de
Weyden, Roger /van der
Roger <von Brügge>
Roger <de la Pasture>
The prometheus database with
• more than 400.000 pictures from 50 different collections
• descriptions without any standardization
• weak subject indexing
• resulting in low recall !
Problem II
Metadata improvement for image information retrieval – ISKO 2010 4
Co
log
ne
Un
iver
sity
of
Ap
plie
d S
cien
ces
& p
rom
eth
eus
Kla
us
Lep
sky
– T
hom
as M
ülle
r –
Jens
Will
e
The central Adoration of the Magi, the only depiction of this kind in Rogier's extant work, was a particularly popular subject in Cologne, for since the 12th century the city had kept relics of the Magi, its most treasured possessions, in the cathedral. The Netherlandish artist not only created this work for a patron in Cologne, but also drew inspiration from an outstanding painting done in Cologne at this period, Stefan Lochner's Altarpiece of the Patron Saints of Cologne, sometimes known as the Cathedral Altarpiece. This large retable, painted in the 1440s for the chapel of Cologne town council, unites the patron saints of the city in a magnificent, monumental work from which any trace of historical narrative has been eliminated. Rogier, perhaps at the special wish of his patron, adopted the almost central position of the Virgin Mary in the depiction of the Adoration; this was an unusual situation for her in Netherlandish art. He also echoed other motifs, including the figure of a woman seen in profile, dressed in green and with a plait of hair, who appeared on the left wing of Lochner's altarpiece and again in Rogier's Presentation; details like the sword and fluttering hatband of the youngest king; and, behind the Magi, the figure of a bearded man holding a hat in front of his chest. Traces of the late Romanesque church of St. Gereon in Cologne seem to have entered into the architecture of the temple on the right wing - the polygonal rotunda, the gallery, and the small buttresses on the outside.(http://www.wga.hu/frames-e.html?/html/w/weyden/rogier/11columb/0columb.html)
Art historical texts are texts about works of art and therefore may be used as ‘metadata’ for these works.
Idea I
Improving information retrieval of digital images by automatically connecting them with text based description
Metadata improvement for image information retrieval – ISKO 2010 5
Co
log
ne
Un
iver
sity
of
Ap
plie
d S
cien
ces
& p
rom
eth
eus
Kla
us
Lep
sky
– T
hom
as M
ülle
r –
Jens
Will
e
homepage of rdk-web
Idea II
article “Drei Könige” (three kings)
Metadata improvement for image information retrieval – ISKO 2010 6
Co
log
ne
Un
iver
sity
of
Ap
plie
d S
cien
ces
& p
rom
eth
eus
Kla
us
Lep
sky
– T
hom
as M
ülle
r –
Jens
Will
e
• identifying corresponding text by
• finding names of artists in text
• finding names of works of art in text
• bringing together pictures and corresponding text
identifiying names and bringing together synonymous forms
matching text and descriptions of pictures
developing a system for an automatic standardization of metadata and a matching algorithm for metadata and
text to improve poorly standardized metadata
Idea III
Metadata improvement for image information retrieval – ISKO 2010 7
Co
log
ne
Un
iver
sity
of
Ap
plie
d S
cien
ces
& p
rom
eth
eus
Kla
us
Lep
sky
– T
hom
as M
ülle
r –
Jens
Will
e
wordsearcher:linguistic analysis, grammatical variants are derived to one basic form
multiworder:linguistic/lexical analysis, dictionary-based identification of phrases
sequencer:algorithmic identification of phrases, based on sequences of the word classes
synonymer:identifying and generating of synonyms
kings > king #nreligious > religious #a, religion #n
„rogier van der weyden“„adoration of the magi“
holy kings > #a #n > „holy kings“
Pasture, Rogier /de le > „rogier van der weyden“adoration of kings > „adoration of the magi“
lingo - linguistic approach to automatic text indexing
Metadata improvement for image information retrieval – ISKO 2010 8
Co
log
ne
Un
iver
sity
of
Ap
plie
d S
cien
ces
& p
rom
eth
eus
Kla
us
Lep
sky
– T
hom
as M
ülle
r –
Jens
Will
e
Adoration of Kings
The Adoration of Kings
The Adoration of the three holy Kings
Adoration of the magi
Rogier van der Weyden
Weyden, Rogier /van
Rogier <van der Weyden>
Pasture, Rogier /de le
Rogier <de LePasture>
Roger <de LePasture>
synonyms of names of pictures
synonyms of names of artists
importance of synonyms
Metadata improvement for image information retrieval – ISKO 2010 9
Co
log
ne
Un
iver
sity
of
Ap
plie
d S
cien
ces
& p
rom
eth
eus
Kla
us
Lep
sky
– T
hom
as M
ülle
r –
Jens
Will
eperseusMatch
matching algorithm with levenshtein distance
(1) grammatical variantsediting distance for characters('königs', 'könige') = 1(editing distance for characters('kings', 'king') = 1)editing distance for words('königs', 'könige') = 0(editing distance for words('kings', 'king') = 0) becausebasic form('königs') = basic form('könige')(basic form('kings') = basic form('king'))
(2) synonymsediting distance for characters('könige','weise') = 6(editing distance for characters('kings','magi') = 5)editing distance for words(‘könige’,’weise’)= 0(editing distance for words(‘kings’,’magi’)= 0) becausesynonyms('könige') contains 'weise'(synonyms('kings') contains 'magi')
using word distance!
Metadata improvement for image information retrieval – ISKO 2010 10
Co
log
ne
Un
iver
sity
of
Ap
plie
d S
cien
ces
& p
rom
eth
eus
Kla
us
Lep
sky
– T
hom
as M
ülle
r –
Jens
Will
e
"[...] Nic. Lancret stellte in der vor 1732 für den Marquis de Béringhen gemalten Elementenfolge das Wasser durch eine ländliche Ff.szene dar, die auch Allegorie des Liebeswerbens ist (s. Sp. 256; Abb. 52; Georges Wildenstein, L., Paris 1924, S. 70 Kat.nr. 2, s. auch Nr. 3, Abb. 5). [...] Luna Diana als Herrin über Fischerei und Schiffahrt zeigt ein Stich von Jan Saenredam nach Hendrick Goltzius (um 1559: Abb. 22)."
KünstlerIn: Hendrik GoltziusTitel: AllegorieStandort: Basel, Öffentliche KunstsammlungGattung: MalereiDatierung: 1611
lingo & perseus match
connecting text & pictures
Metadata improvement for image information retrieval – ISKO 2010 11
Co
log
ne
Un
iver
sity
of
Ap
plie
d S
cien
ces
& p
rom
eth
eus
Kla
us
Lep
sky
– T
hom
as M
ülle
r –
Jens
Will
e
Thank You for listening!
