Expand. Learn. Interact: Enabling Digital Humanities

Expand. Learn. Interact. Enabling (Digital) Humani9es

http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

Lora Aroyo VU University Amsterdam

“The Gallery of Cornelis van der Geest” (Willem van Haecht)



is a rich digital space

to explore TOPICS, e.g hun9ng, dogs

Quen9n Metsys The Madonna and Child

to explore ARTISTS, e.g. their pain9ngs, biographies, links

to explore EVENTS, e.g types, par9cipants, links


Mul9-‐dimensional

“Jungle of scales” Mul9-‐granularity

Mul9-‐perspec9ve

Mul9tude of collec9ons

Digital Heritage Mission

Lora Aroyo

“We enable everyone to u/lize the collec/ons to learn, experience & create.” (mission Sound and Vision) “a place where a na/on nourishes its memory, and exerts its imagina/on -‐ where it connects with its past and invents its future” (Pierre Ryckmans, 1996) „places that foster curiosity, celebrate ideas and enable discovery.” (sxswLAM community, hMp://www.sxswlam.com/) Help people

finding their way

Informa(on: Museums & Archives as Inventories of the World

André Malraux, The Imaginary Museum of World Sculpture, 1953

Interpreta(on: Museums & Archives as a Place to Engage with the World

effec/ve & efficient in delivering change and tangible benefits strategic advantage through the innova/on inherent in this digital ac/vity economic benefit to community that demonstrate the value of endeavours in monetary terms ( the community has been changed by the resource in beneficial ways that can be clearly iden/fied

Source: Simon Tanner, Kings College London , hBp://www.kdcs.kcl.ac.uk/innovaGon/impact.html


New Era of Compu9ng

Lora Aroyo

•  amount of data & scale of computa9on available have increased by a previously inconceivable amount

•  CS & AI moved out of thought problems – entered empirical science

IBM Confiden/al

Open Domain Question-Answering Machine, that given – Rich Natural Language Questions – Over a Broad Domain of Knowledge

Won a 2-game Jeopardy match against the all-time winners –  viewed by over 50,000,000

Expand. Learn. Interact.

Enabling (Digital) Humani9es


How to Do It?

Lora Aroyo

1. Expand

“EXPAND human cogni/on, makes the jobs we do easier, like a cogni9ve prosthesis, especially when dealing with processing massive data, or data that requires human interpreta9on”

•  Support for Interpreta9on •  Structured Seman9cs

… Digital Humanities researchers


… often don’t find what they were searching for


HOW DO WE FILL THE GAP?


Concep9ons of eHuman9es

INTERPRETATION

•  a theory of interpretaGon of informaGon •  bringing people and technology together to explore:

–  how to model and represent informa/on –  how to provide engaging interac/on –  how to support interpreta/on

“Digital HermeneuGcs: Agora and the online understanding of cultural heritage” In proceedings of Web Science

Conference, (ACM: New York, 2011)

Digital Hermeneu9cs

Simple Event Model (SEM) OpenAnnota/on (OA) & SKOS

DIVE:MEDIA OBJECT

SEM:EVENT

SEM:PLACE

SEM:TIME

SEM:ACTOR

SKOS:CONCEPT

OA:ANNOTATION

•  Links to EUROPEANA (mul/lingual) •  Links to DBPedia, other vocabularies

Erp, M. van; Oomen, J.; Segers, R.; Akker, C. van de; Aroyo, L.; Jacobs, G.; Legêne, S; Meij, L. van der;O ssenbruggen, J.R. van; Schreiber, G. Automa/c Heritage Metadata Enrichment with Historic Events Museums and the Web 2011 hMp://www.museumsandtheweb.com/mw2011/papers/automa/c_heritage_metadata_enrichment_with_hi

engaging users

through event

narratives

hMp://sealinc.ops.few.vu.nl/invenit/

Exploring with Structured Seman9cs

CLUSTERING ! RANKING ! TRUST ! INTEREST

hMp://sealinc.ops.few.vu.nl/invenit/

Lora Aroyo

“LEARN as you use it – most machine errors are easy for a human to detect, and we can instrument usage of systems to beMer understand the system and the problem it solves”

2. Learn

•  Teach machines •  Deal with diversity of interpreta9on

Teaching Machines


Diversity, Opinions, Interpreta9ons


Human Disagreement IS Essential in Helping Machines with Semantic Interpretation!


“best collec9ve decisions are result of disagreement,

not consensus or compromise” James Surowiecki

Crowdsourcing Diversity of Interpreta9on

A novel approach to gather diversity of perspec9ves & opinions from the crowd, expand expert vocabularies with these and gather new type of gold standard for machines

L. Aroyo, C. Welty: Crowd Truth: Harnessing disagreement in crowdsourcing a rela/on extrac/on gold standard. ACM WebSci 2013.

CrowdTruth.org


Crowdsourcing for Video Analysis


CrowdTruth.org

Following the grandeur of Baroque, Rococo art is often dismissed as frivolous and unserious, but Waldemar Januszczak disagrees. […] The first episode is about travel in the 18th century and how it impacted greatly on some of the finest art ever made. The world was getting smaller and took on new influences shown in the glorious Bavarian pilgrimage architecture, Canaletto's romantic Venice and the blossoming of exotic designs and tastes all over Europe.

Rococo: Travel, pleasure, madness

A boarding school where boys from the Dutch East Indies receive a voca/onal educa/on has been set up by the ministry of Social Affairs near Batavia. Trumpeter sounds the reveille; -‐ the boys get out of bed; -‐ the muster is held in front of the building; akerwards the boys stand in line with a food bowl; they get rice pudding and eat this around long tables in the open air; -‐ the boys get instruc/ons and prac/ce a.o. drawing shapes, forging, metalworking, filing and woodworking;

News from Indonesia: youth care

hdp://vista-‐tv.eu

hdp://dive.beeldengeluid.nl

Crowdsourcing for Image Analysis


CrowdTruth.org

Crowdsourcing for Text Analysis


CrowdTruth.org

•  One truth: data collec/on efforts assume one correct interpreta/on for every example

•  All examples are created equal: ground truth treats all examples the same – either match the correct result or not

•  Detailed guidelines help: if examples cause disagreement -‐ add instruc/ons to limit interpreta/ons

•  Disagreement is bad: increase quality of annota/on data by reducing disagreement among the annotators

•  One is enough: most of the annotated examples are evaluated by one person

•  Experts are beder: annotators with domain knowledge provide beMer annota/ons

•  Once done, forever valid: annota/ons are not updated; new data not aligned with previous

7 Myths of Human Annota9on

myths directly influence the prac/ce of collec/ng human annotated data; Need to be revisited in the context of new changing world & in the face of a new

theory of truth (CrowdTruth)

Lora Aroyo

Lora Aroyo

“INTERACT naturally. We need to bring machines closer to their users, we have adapted ourselves enough to them, they should understand natural language, spoken or wriMen, be able to process images and videos. These simple human problems are extremely complex for machines, but are hallmarks of a new compu9ng era.”

3. Interact

•  Gathering Human Seman9cs •  Engaging Users

Engage the Crowd in Cultural Heritage

hMp://2-‐dot-‐rma-‐accurator.appspot.com/

Engage the Niche in Cultural Heritage

Engage with Games in Cultural Heritage

Engage with Games in Sciences


hdp://game.crowdtruth.org/


hdp://waisda.nl/ hdp://spotvogel.vroegevogels.vara.nl/

Engage with Games in Entertainment

Engage with Novel Interfaces Event-‐based Browsing Of Linked Historical Media

dive.beeldengeluid.nl

3rd Price at the Seman9c Web Challenge 2014

OPENIMAGES.EU •  3000 videos •  NL Ins/tute for Sound & Vision •  mostly news broadcasts

DELPHER.NL •  1.5 Million Scans of •  Radio bulle/ns •  (hand annotated) •  1937 – 1984

Digital Submarine UI Infinity of Explora9on

Events Linking Objects

Crowd Bringing Human Perspec9ves

Linked (Open) Data

Expand. Learn. Interact.

Enabling (Digital) Humani9es


The Road Ahead

Lora Aroyo

•  Quality measures •  Capturing Trust •  Dealing with Provenance •  Complexity of Analy/cs •  Workflows for Impact •  Ac/ve Learning

Technology Challenges

Lora Aroyo

•  Influence directly Heritage organiza/ons

•  Interdisciplinary research with computer sciences

•  Smart, connected, open heritage

•  Visible, usable, durable heritage

•  Shared knowledge infrastructure

Digital Humani9es Roadmap

Agora Team: Lora Aroyo, Guus Schreiber, Lourens van der Meij, Marieke van Erp, Chiel van den Akker, Susan Legêne, Geertje Jacobs,

Johan Oomen

DIVE Team: Victor de Boer, Oana Inel, Lora Aroyo, Johan Oomen, Elco Van Staveren, Werner Helmich & Dennis De Beurs

dive.beeldengeluid.nl

SealincMedia Team: Alessandro Bozzon, Geert-‐Jan Houben, Lora Aroyo, Lizzy Jongma, Guus Schreiber, Chris Dijkshoorn, Jasper Oosterman, Jacco van Ossenbruggen, Archana NoMamkandath,

Myriam Traub

http://sealinc.ops.few.vu.nl/invenit/

CrowdTruth Team: Lora Aroyo, Chris Welty Robert-‐Jan Sips, Carlos Mar9nez Or9z, Anca Dumitrache, Oana Inel, Benjamin Timmermans, Susanna van de Ven, Merel van Empel, Jelle v.d. Ploeg, Ta/ana Cristea, Khalid Khamkham, HarriëMe Smook, Rens van Honschooten, Arne Rutjes

CrowdTruth.org github.com/CrowdTruth

Links


On the Web •  http://waida.nl •  http://prestoprime.org •  http://agora.cs.vu.nl •  http://sealincmedia.wordpress.com •  http://notube.tv •  http://vista-tv.eu •  http://dive.beeldengeluid.nl •  http://crowdtruth.org •  http://game.crowdtruth.org •  http://wm.cs.vu.nl

On Twider @waisda @agora-‐project @sealincmedia @prestocenter @notubeproject @vistatv #CrowdTruth

lora-aroyo.org slideshare.com/laroyo

@laroyo

Documents

Expand. Learn. Interact: Enabling Digital Humanities