Embed Size (px)
Citation preview
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
Representational Lego for ECAs
Brigitte Krenn
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
Motivation
• Background• representations for multimodal
behaviour generation• use representations at the
interfaces of system components
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
Motivation
• Wish• reusable, flexible
representational “standards”• to devise interface representations
that ease • exchange of system components• integration of new modules
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
Motivation
• Current • everybody does their own language• there is a wealth of different
representations• partially overlapping• partially differing
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
up to date attempts to design a standard representation language for ECAs have failed
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
Goal
to have • reusable• extendable • mappable
bits and pieces of representations
of ECA relevant information
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
Formal Requirements
• separation of declarative and procedural information
• mapping between high-level concepts and their low-level representations
• mapping across concepts • extendibility
• granularity of descriptions• incorporation of new concepts• ability to embed existing XML
representations
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Content Side
?• units of information common to
existing ECA systems
• information ECA systems ideally should have
• allow for optionality
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Current State
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Current State Terminology
• Markup Languages
• Representation Languages
• Scripting Languages
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Current State Markup Languages
• for non-expert users• to annotate text• with high-level expert information• e.g.
• VoiceXML for creating voice enabled applications
• VHML for creating interactive applications with ECAs
• APML for annotating text with high-level ECA controls
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Current State Representation Languages
• technically detailed annotations of theory-specific information
• high- and low-level concepts
• for expert use
• function as data representation formats inside a system
• e.g. RRL
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Current State Scripting Languages
• combine declarative and procedural knowledge
• comparable to high-level programming languages
• e.g. STEP/XSTEP, ABL, PAR/EMOTE
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Current State Summary
• Markup languages • high-level concepts• are indispensable for application
development
• Representation languages • mix high- and low-level concepts• are crucial in research contexts
• Scripting languages • add procedural information
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Current State Summary
RepresentationLanguage
ScriptingLanguage
MarkupLanguage
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Current State Summary
• standardisation efforts up to now concentrate on markup languages
• they are • application oriented
• to design representations for ECAs in the spirit of VoiceXML
• text/utterance oriented• to design multimodal behaviour
control as markup for text
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
Pros for XML-based Representation Languages as Interfaces in ECA systems
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
Advantages of XML Encoding
• XML is • flexible • easy to share
• tools for XML processing
• standardization efforts (w3c)
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
Advantages of the Use of Representation Languages
• to encode information flow between system components
• to map between high-level concepts and low-level realizations
• to ease integration/replacement of system components
• to support a plug-and-play approach • to support the development of
mockup systems
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
Affective, Interactive ECARelevant Components and Concepts
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Content SideCurrent Foci
• speaking ECA • simulation of mm-dialogues• no/little “true” interactivity• APML, RRL
• moving ECA• XSTEP, MURML, (?MiraLab)
• speaking and moving ECA• there are some gaps to bridge
• interacting ECA• approaching• PML, ABL
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Content SideRepresentations & Architectures
• information relevant for a certain ECA system depends on the architecture and system components used
? Is it possible to identify a common core of relevant components and concepts
? Is it possible to provide reusable representations for these concepts
! Allow for flexibility of the representations
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Content Side Relevant Parts
• World parameters• Scenes and story lines• MM-dialogue generation• Speech• Animation (body, face)• Affect (emotion, personality traits)+ Temporal control and synchronization? Interactivity? MM-comprehension
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Content Side Temporal Control and Synchronization
• time-alignment of mm-behaviour of an agent
• temporal ordering of the actions/behaviours of agents interacting with the outside world• agent-object• agent-agent• agent-user
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Content Side Time-Alignment of MM-Behaviour
• speech as guiding medium(phoneme durations)
• motion: beats as smallest units(e.g. XSTEP)
• synchronization of speech rate and motor activity
• motor activity can also constrain voice quality
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Content Side Interactivity
• What are the desired smallest communicative units?
• speech• dialogue• interactive drama
• What are the technically manageable smallest units?
• What are the technological challenges?
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
The Content Side Interactivity
• Multimodal understanding? What is relevant information
• How do we manage the information flow→ agent technology (Lola et al.)
• Models of the listening ECA
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
Next Steps
• examine existing languages for a common core
• compare their representations
• consider architectural aspects
• define XML representations for bits and pieces
• make them publicly available
HUMAINE Workshop Paris 10./11. March 2005 [email protected]
Öst
erre
ichi
sche
s F
orsc
hnun
gsin
stitu
t fü
rA
rtifi
cial
Int
ellig
ence
talk me off
join in !
