CWI AmsterdamThe Netherlands
Supporting the Generation of Argument Structurewithin Video Sequences
Talk Outline
The motivation and vision of the work What is needed
Annotations Editing Process
Editor Support Conclusions
Existing Documentaries
Traditional video authoring: the footage is selected and edited for the final cut
there is only one final version, what is shown is the choice of the author / editor
Material can be very rich and controversial (e.g. Voices of Iraq)
New Paradigm Proposed video authoring:
Annotate the video material semantics Edit it automatically, selecting what the
user asks to see Use the Web as an interactive
distribution mean More than a sequence of matching
video fragments): Argumentation/rhetoric Narrative
Video material
Interview with America: video footage with interviews and background material about the opinion of American people after 9-11www.interviewwithamerica.com
Annotations: 1 hour annotated, 15 interviews, 60 interview segments, 120 statements
What do you think of the war in Afghanistan?
Example Explained
Claim
Concession
Claim
contradict
supportClaim
I am not a fanof military actions
War has never solved anything
Two billions dollar bombs on tents
I cannot think of amore effective solution
weaken
The annotations
Rhetorical Argumentation model: Toulmin model Rhetorical Statement (mostly verbal, but
visual also possible) Descriptive
Question asked Interviewee (social) Filmic (e.g. location/time/framing/gaze)
Encode statements Statement formally annotated:
<subject> <modifier> <predicate> E.g. “war best solution”
A thesaurus containing: Terms for each part (155 in total) Relations between terms: similar (72), opposite
(108), generalization (10), specialization (10) E.g. war opposite diplomacy
Relations in the thesaurus determine link type between statements
Automatic Linking Process STEP1: Using the thesaurus, generate
related statements, by replacing iteratively terms: E.g. from “war best solution” “diplomacy
best solution”, “war not solution” STEP2: Query the repository to see
whether the statement is present
Connect statements
Create a graph of related statements Nodes are the statements (video
segments), edges are either support or contradict
S1
S2
S3 S5
S4
S7
S6
S8
S9
S0
= support
= contradict
Author/Annotator support Capability of generating different
arguments depends on the quality of the Semantic Graph
Statements (and corresponding video segments) not connect are lost for generation: Our case: out of 118 statements 54 were not
connected Measure the performance of the automatic
linking process
Indices for statements
Measure how many statements are generated from a given one: depends on the quantity of the relations
in the thesaurus Measure how many generates
statements are present in the repository Depends on correctness of the relations
in the thesaurus
Index for relations
Measure how a particular relation in the thesaurus is performing: If a generated statement is present in
the repository, the relations used to generate it get one point on a hit score, otherwise one point on the miss score
The ratio hit/miss gives an idea of the semantic accuracy of the relation with respect to the repository
Current/future Work Automatic Relation suggestion:
Start with a fully connected thesaurus, keep only best relations
Suggest best relation to add to existing ones
Linking Process tuning Currently 3 iterations for performance,
but the process runs of-line: more iterations possible
Different repositories (VJ project)
Conclusions
New documentary production mechanism, multiple versions
Different authoring, author does not have full control anymore
Authoring support needed
Questions?
Thanks for your attention
Pointers & Acknowledgments
This presentation and Demo available at:http://www.cwi.nl/~media/demo/IWA/
This research was funded by the Dutch national ToKeN2000 I2RP and CHIME projects.
Author/Annotator support
Provide means to measure the performance of the creation of the Semantic Graph
Reengineer the Semantic Graph generation: Changing annotations Changing relations in the Thesaurus
What do you think of the war in Afghanistan?
I am not a fan of military actions
War has neversolvedanything
I cannot think of a more effective solution
Two billionsdollar bombson tents
Toulmin model
ClaimData
Qualifier
Warrant
Backing
Condition
Concession
57 Claims, 16 Data, 4 Concessions, 3 Warrants, 1 Condition