Upload
abra-weaver
View
46
Download
1
Tags:
Embed Size (px)
DESCRIPTION
error handling – Higgins / Galatea. Dialogs on Dialogs Group July 2005. work by …. Gabriel Skantze ph.d. student KTH, Stockholm. - PowerPoint PPT Presentation
Citation preview
1
error handling – Higgins / Galatea
Dialogs on Dialogs Group
July 2005
2
work by …
Gabriel Skantzeph.d. studentKTH, Stockholm
“I am doing research on spoken dialogue systems. More specifically, I am interested in studying miscommunication and error handling, but also in the representation and modelling of utterances and dialogue, as well as conducting experiments with users.“
and co-authors: J. Edlund, D. House, R. Carlson
3
3 papers
HigginsHiggins – a spoken dialogue system for investigating error handling techniques,
Edlund, Skantze, Carlson [2004]
GalateaGALATEA: A Discourse Modeller Supporting Concept-Level Error Handling in
Spoken Dialog Systems, Skantze [2005]
Prosody & ClarificationsThe Effects of Prosodic Features on the Interpretation of Clarification Ellipses, Edlund, House, Skantze [2004]
4
1st paper
HigginsHiggins – a spoken dialogue system for investigating error handling techniques,
Edlund, Skantze, Carlson [2004]
GalateaGALATEA: A Discourse Modeller Supporting Concept-Level Error Handling in
Spoken Dialog Systems, Skantze [2005]
Prosody & ClarificationsThe Effects of Prosodic Features on the Interpretation of Clarification Ellipses, Edlund, House, Skantze [2004]
5
Higgins
practical goal of Higgins project build a collaborative dialog system in which error handling
ideas can be tested empirically
error handling issues, plus incremental dialogue processing on-line prosodic feature extraction robust interpretation flexible generation and output
6
domain
pedestrian city navigation and guiding user gives system a destination system guides user by giving verbal instructions
complex large variety of error types semantic structures can be quite complex reference resolution
domain can be extended even further
7
architecture
follow-up from Adapt everything is XML
domain objects utterance semantics discourse model database content system output (before surface) 3D city model
8
research issues
early detection and correction late detection incrementality error recovery
9
early detection and correction
KTH LVCSR – output likely to contain errors
robust interpretation Pickering: some syntactic analysis is needed
e.g. relations between objects but handles insertions and non-agreement phrases
humans - good at early detection (woz)
10
late detection and correction
discourse modeller (GALATEA) joins several results from Pickering into a discourse model adds grounding information can be manipulated later
remove concepts which turn out not to be grounded
11
incrementality
end-pointers cause trouble even more so in this domain
better:
12
incrementality [2]
all components support incremental processing
several issues when to barge in? (semantic content and prosody) longer-than-utterance units: interpreter or dialog manager? rapid and unobtrusive feedback: challenge for synthesis
13
error recovery
signaling non-understandings decreased experience of task success slower recovery
ask other task-related question
14
2nd paper
HigginsHiggins – a spoken dialogue system for investigating error handling techniques,
Edlund, Skantze, Carlson [2004]
GalateaGALATEA: A Discourse Modeller Supporting Concept-Level Error Handling in
Spoken Dialog Systems, Skantze [2005]
Prosody & ClarificationsThe Effects of Prosodic Features on the Interpretation of Clarification Ellipses, Edlund, House, Skantze [2004]
15
GALATEA
a discourse modeller for conversational spoken dialog systems builds a discourse model (what has been said during the
discourse)
resolution of ellipses & anaphora tracks the grounding status
who said what when (plus confidence information) can be used for concept-level error handling
16
should do grounding at concept level
explicit and implicit verification on whole utterance can be tedious and unnatural
45% of clarifications in BNC are fragmentary / elliptical
17
should do grounding at concept level
Traum (1994) – utterance level computational model of grounding
Larsson (2002) – issue-level computational model of grounding in Issue-Based DM
Rieser (2004), Schlangen (2004): systems capable of fragmentary clarification requests, but models do not handle user reactions
systems should keep grounding information at the concept level
like RavenClaw?
18
semantic representation
rooted unordered trees of semantic concepts nodes: attr-value pairs, objects, relations, properties
19
semantic representation
enhanced with “meta”-information confidence communicative acts info is new / given
20
ellipsis resolution
transforms ellipsis into full propositions
rule based ~10 rules domain-specific
21
anaphora resolution
keeps a list of entities (talked about) assigns ids
when given entities are added to the discourse, look up the antecedent
if found, unification (and move to the top of the entity list)
unification also allows entities to be referred to in new ways
how does this fare and compare?
22
grounding status
who added the concept? in which turn? how confident?
may be used by the action manager for instance remove all items with high grounding status
when referring to an entity
23
updating grounding status
24
late error detection
discover inconsistencies in discourse model
look at grounding status to see where error may be concept can be removed
25
future
methods for automatic tuning of strategy selection
extend to track confidence and grounding status at different levels
evaluate how people respond to incorrect
confirmations, and how can that information be used to update grounding status
error recovery after non-understandings other domains
26
3rd paper
HigginsHiggins – a spoken dialogue system for investigating error handling techniques,
Edlund, Skantze, Carlson [2004]
GalateaGALATEA: A Discourse Modeller Supporting Concept-Level Error Handling in
Spoken Dialog Systems, Skantze [2005]
Prosody & ClarificationsThe Effects of Prosodic Features on the Interpretation of Clarification Ellipses, Edlund, House, Skantze [2004]
27
prosody in clarifications
effects of prosodic features on interpretation of elliptical clarifications U: Further ahead on the right I see a red building… S: Red (?)
vary prosodic features study impact on user’s understanding
of the system’s intention
28
motivation
long (whole utterance) confirmations are not good tedious, unnatural BNC corpus: 45% of clarifications are elliptical
short confirmations make dialog more efficient by focusing on the actual
problematic fragments however
interpretation depends on context and prosody
29
3 readings
U: Further ahead on the right I see a red building… S: Red (?)
Ok, red [all positive]
Do you really mean red? What do you mean by red? [positive perception, negative understanding]
Did you say red? [positive contact, negative perception]
30
stimuli
3 test words [red, blue, yellow] di-phone voice (MBROLA) manipulated
peak position [mid, early, late / 100ms] peak height [130Hz / 160 Hz] vowel duration [normal, long / +100ms]
31
subjects + design
8 speakers: 2f / 6m, 2nn / 6n introduced to Higgins listen to all 42 (only once); random
order 3 options
Okay, X Did you really mean X? Did you say X?
32
results
no effects for color, subject, duration
significant effects for peak position, peak height, & their interaction
33
results
Statement: early, low peak Question: late, high peak Clear division between “did you mean” and “did you say”
34
food for thought
how about English? red red? red!? how many ways can you say it?
35
conclusion
strong relationship between intonation and meaning
statement: early, low peak question: late, high peak clear division between “did you mean”
and “did you say”
36
the end