Facilitating Access to Video Oral Histories through Informedia Technologies and a

Multimedia Web Portal

October 29, 2010

Presented at Oral History Association Meeting, Atlanta, GA

Mike Christelchristel@cs.cmu.edu

Entertainment Technology CenterCarnegie Mellon University

Talk OutlineTalk Outline

(Heavy on demonstrations, light on slides….)(Heavy on demonstrations, light on slides….)

• Introduction to oral history concerns/opportunitiesIntroduction to oral history concerns/opportunities

• Quick overview of CMU Informedia researchQuick overview of CMU Informedia research

• The HistoryMakers The HistoryMakers African American oral history archiveAfrican American oral history archive

• Field tests with Field tests with The HistoryMakersThe HistoryMakers

• Lessons LearnedLessons Learned

• Flash Web SiteFlash Web Site

• Next StepsNext Steps

Reference: http://www.idvl.orgReference: http://www.idvl.org

For more info or to express interest: Email info@idvl.orgFor more info or to express interest: Email info@idvl.org

Why Oral Histories?

• Oral history has been embraced as a useful tool for: • “…giving voice to those who have been excluded from the

historical record”• “…“…adding richness and personal perspective to the historical adding richness and personal perspective to the historical

record and engaging students and scholars in a lively study of record and engaging students and scholars in a lively study of history”history”

• Contextual inquiry has found that oral history recordings are Contextual inquiry has found that oral history recordings are considered a central historical artifact considered a central historical artifact

• Significant hurdles in oral history access are brought upon by the linear nature of speech and the extended length of some interviews: Jong, F. D., Oard, D. W., et al. 2008. Jong, F. D., Oard, D. W., et al. 2008. Access to recorded interviews: A research agenda. Access to recorded interviews: A research agenda. Journal Journal on Computing and Cultural Heritage on Computing and Cultural Heritage 1, 1 (Jun. 2008), 1-27 1, 1 (Jun. 2008), 1-27

CMU Informedia Digital Video Research

• 1994-2007 details at: http://www.informedia.cs.cmu.edu

• New work for oral histories at http://www.idvl.org• Speech recognition and alignment• Image processing• Named entity tagging • Synchronized metadata for search and navigation

• Fast, direct video access to oral histories, news, etc.

• Demonstrated successes (NIST TRECVID) with broadcast news, documentaries, surveillance video in nursing homes, oral histories – over 45 TB

• Oral history work has included work with Harrisburg, PA 150 and The HistoryMakers (over 1000 hours and 20,000 stories) – funded via NSF IIS-0705491

The HistoryMakers The HistoryMakers Oral History ArchiveOral History Archive

• http://www.thehistorymakers.comhttp://www.thehistorymakers.com

• Funded in part by IMLS Grant LG-03-03-0048-03Funded in part by IMLS Grant LG-03-03-0048-03

• The world’s largest African American oral history archive The world’s largest African American oral history archive with accomplished African Americans with accomplished African Americans

• Purpose: Purpose: • To educate and show the breadth and depth of this To educate and show the breadth and depth of this

important American history as told by the first personimportant American history as told by the first person• To highlight the accomplishments of individual African To highlight the accomplishments of individual African

Americans across a variety of disciplinesAmericans across a variety of disciplines• To preserve this material for generations to comeTo preserve this material for generations to come

• Committed to exposing the archive to the widest audience Committed to exposing the archive to the widest audience possible, making use of new technologies as appropriatepossible, making use of new technologies as appropriate

Screen Shot 1Screen Shot 1

Screen Shot 2Screen Shot 2

Screen Shot 3Screen Shot 3

Statistics from Field Tests, 2008-2009Statistics from Field Tests, 2008-2009

• Field-tested at Drexel University, Field-tested at Drexel University, The HistoryMakers The HistoryMakers office, 3 campuses of the University of Illinoisoffice, 3 campuses of the University of Illinois

• 266 participants (158 female, 108 male) used system 266 participants (158 female, 108 male) used system

• 210 people used it for only one logged session, 34 210 people used it for only one logged session, 34 participants used it twice, 18 people used it three to five participants used it twice, 18 people used it three to five times, with 4 people using it six or more timestimes, with 4 people using it six or more times

• 381 sessions were logged, representing 329 hours of 381 sessions were logged, representing 329 hours of use, based on recorded session start and end times.use, based on recorded session start and end times.

Participant DemographicsParticipant Demographics

Transaction Log ResultsTransaction Log Results381 Total Sessions

First Time Sessions (n=266)

Return Sessions (n=115)

Avg. Session Time (as mm:ss) 44:13 52:43

Number of Text Queries Issued 4.20 3.39

Avg. Answer Size per Query 413 318

Number of Played Video Stories 10.2 10.7

Time Spent Playing Video (mm:ss)

32:39 35:58

Use of Query Preview Browser 0.18 0.32

Use of Advanced Search Options 0.61 0.97

Use of Movie Table of Contents 0.64 0.12

Use of Timeline View 0.21 0.10

Use of Common Text View 0.15 0.18

Use of Storyboard View 0.14 0.19

Use of Tree Browser View 0.14 0.30

Use of Geographic Map View 0.06 0.13

Lessons LearnedLessons Learned

• Improved delivery infrastructure (web client vs. stand-Improved delivery infrastructure (web client vs. stand-alone lab-only access)alone lab-only access)

• Clean, simple access Clean, simple access

• Intuitive “standard” use of text searchIntuitive “standard” use of text search

• Intuitive filteringIntuitive filtering

A web portal (in Flash) was developed for The A web portal (in Flash) was developed for The HistoryMakers, and then tested by 27 participants (13 HistoryMakers, and then tested by 27 participants (13 female, average age 25) in which users were asked to female, average age 25) in which users were asked to do two tasks, one a direct lookup “treasure hunt” and do two tasks, one a direct lookup “treasure hunt” and

the other open-ended and exploratory. the other open-ended and exploratory.

Web Portal Empirical Study ResultsWeb Portal Empirical Study Results

(27 Participants) Treasure Hunt Task Exploratory Task

Session Time (as mm:ss)

20:00 20:00

Number of Text Queries Issued

45.2 14.1

Avg. Answer Size per Query

52 139

Number of Played Video Stories

18.4 8.7

Time Spent Playing Video (mm:ss)

7:08 13:41

Empirical Study, Post-Task Survey Data

5-Point Scale (left = 1, right = 5) Treasure Hunt

Task Exploratory

Task

Unsatisfying – Satisfying 4.0 4.0

Not Inspiring – Inspiring 3.6 4.0

Not Entertaining – Entertaining 3.7 4.0

Inaccurate – Accurate 4.2 4.0

Text Search: Not Useful – Useful 3.8 3.8

Map Search: Not Useful – Useful 2.4 2.8

Search filtering: Not Useful – Useful

3.3 3.7

Video player: Not Useful – Useful

3.4 4.5

Text transcript: Not Useful – Useful

4.3 4.2

Value of Video Varies by Task

• The value of representation modality can be masked by the task:

• If only direct lookup (treasure hunt) is studied, the video modality may be seen as unnecessary over searchable transcript text.

• By considering exploratory tasks, the value of video for life oral histories is acknowledged.

• Interestingly, even for open-ended exploratory tasks, an aligned text transcript modality remains highly valued for oral histories.

More details at: Christel, M., Stevens, S., Maher, B., and Richardson, J. Enhanced Exploration of Oral History Archives through Processed Video and Synchronized Text Transcripts. Proc. ACM Multimedia Conference (Florence, Italy, October 2010).

Web Site DemoWeb Site Demo

• Adobe Flex used to build Flash interface to oral histories Adobe Flex used to build Flash interface to oral histories

providing:providing:

• Text search (using Lemur open source indexing)Text search (using Lemur open source indexing)

• Map search (using Yahoo! Maps)Map search (using Yahoo! Maps)

• Tag searchTag search

• Video play with synchronized transcriptsVideo play with synchronized transcripts

• Filtering (based on provenance data such as narrator Filtering (based on provenance data such as narrator

gender and birth year)gender and birth year)

• Try it yourself at www.idvl.org Try it yourself at www.idvl.org

Future Work

• Faceted search (e.g., showing at a glance the distribution of stories across categories, allowing for quick interactive filtering and review)

• Simple to use menu bar (as is popular on e-commerce sites)

• Emphasis on aesthetics (improved interface coming soon!)

• Further iterations based on web metrics

• Streamlining on the processing side (currently a text xml file and a video file are the input)