Upload
cornelia-wright
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Language Technologies Institute
Flexible Dialog Management
for In-vehicle Dialog Systems
Jeongwoo [email protected]
2
Language Technologies Institute
Outline
Introduction Approach in CAMMIA Dialog Management Framework
System Architecture ScenarioTemplate/ScenarioXML ScenarioXML Development Kit
Pilot Systems & Experiments Current Research Focus Future Work
3
Language Technologies Institute
Introduction of CAMMIA Project(Conversational Agent for Multilingual Mobile Information Access)
4
Language Technologies Institute
Sample DialogS1: How may I help you?U1: I want to go to Carnegie Mellon University.S2: Do you want to go to Carnegie Mellon University?U2: Yes.S3: The distance to the destination is 100 miles. It takes about 2 hours.
U3: I would like to know weather.S4: Please tell me the area and the date.U4: Pittsburgh
(Navigation System sends the next direction to the dialog manager)
S5: To go to Carnegie Mellon University, please make a left turn here. Please tell me the date for Pittsburgh.U5: TomorrowS6: The weather for Pittsburgh tomorrow will be fine.
5
Language Technologies Institute
Some Related Research (Speech Interface)
Pellom et al, HLT 2001 (CU-Move) Route planning and navigation Noise suppression front-end and back-end navigation
information retrieval Download driving instructions from the Internet after route
planning User can ask route information during travel (ex. What’s my next turn?)
Coletti et al, IEEE 2003 Hotel retrieval/reservation, POI retrieval, Simple Route
Query Car Wide Web module: XML-based DB interface Local database for tour and geographic data
6
Language Technologies Institute
Some Related Research (Multimodal Interface) Minker et al, ICSLP 2002
(SmartKom Mobile) Provide framework for modality control Prototype on Compac iPAG H3630 handheld
computer Apply to pedestrian and driver environment Display maps for route information Slide shows for sight information
7
Language Technologies Institute
Approach in CAMMIA Asynchronous communication with navigation
system When getting the next direction, interrupt the current dialog Notify the next turn direction Resume the pending dialog
Maintain dialog history for smooth dialog switching
Flexible & robust Dialog Manager based on VoiceXML (Voice eXtensible Markup Language)
Error handling Correction, Explicit/Implicit confirmation
Support multi-lingual (Japanese, English)
8
Language Technologies Institute
Next Introduction Approach in CAMMIA
Dialog Management Framework System Architecture ScenarioTemplate/ScenarioXML ScenarioXML Development Kit
Pilot Systems & Experiments Current Research Focus Future Work
9
Language Technologies Institute
ADM
DialogController
Dialog Manager (DM)
Dialog historyDialog scenarioData source
Dialog Management Framework
VXI: VoiceXML InterpreterADM: Asynchronous Dialog Manager
VXI
Julius
Speech Interface
Navigation System
Simulator
Navigation System (NS)
Map Display
VoiceXML
URL to get next direction
HTTP Request
Destination
Direction
Direction
10
Language Technologies Institute
Dialog Manager Support multi-user/multi-dialogs Keep track of user session and dialog flows
for smooth task switching Convert user utterances to database query Mixed-initiative interaction Create VoiceXML with dynamic contents => Hard to write VoiceXML by hands => Provide abstract level of dialog
description
11
Language Technologies Institute
Dialog Description ScenarioTemplate (ST)
Designed to facilitate new dialog creation Consist of prompts and variables => Dialog designer does not have to know
grammar Support explicit/implicit confirmation Compiled into SXML
ScenarioXML (SXML) Consist of dialog states and transitions Dialog developers need to know grammar, but
don’t have to add other dialogs for dialog switching Compiled into JSP to create VXML with dynamic
contents
12
Language Technologies Institute
Example of ST and SXML<variables>weather_area: Stringweather_date: Stringweather_result: String
<states>prompt: Please tell me the area and
date.variable: weather_area weather_date
prompt: Please tell me the area.variable: weather_area
prompt: please tell me the date.variable: weather_date
confirmation: Would you like to know the weather for weather_area weather_date?
response: The weather for weather_date weather_area is weather_result.
<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE function SYSTEM "dtd/function.dtd"><function name="Weather"> <state name="ask_weather" position="start"> <grammar src="grammars/ask_weather_jp.gad"/> <prompt> <text>Please tell me the area and date</text> </prompt> <jump> <nextstate next="ask_weather_area"> <field>weather_date</field> </nextstate> <nextstate next="ask_weather_date"> <field>weather_area</field> </nextstate> <nextstate next="confirm_weather"> <field>weather_area</field> <field>weather_date</field> </nextstate> <default>Can you tell me again?</default> </jump> </state> <state name="ask_weather_area" position="transition"> … </state></function>
13
Language Technologies Institute
Example of SXML and VoiceXML<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE function SYSTEM "dtd/function.dtd"><function name="Weather"> <state name="ask_weather" position="start"> <grammar src="grammars/ask_weather_jp.gad"/> <prompt> <text>Please tell me the area and date</text> </prompt> <jump> <nextstate next="ask_weather_area"> <field>weather_date</field> </nextstate> <nextstate next="ask_weather_date"> <field>weather_area</field> </nextstate> <nextstate next="confirm_weather"> <field>weather_area</field> <field>weather_date</field> </nextstate> <default>Can you tell me again?</default> </jump> </state> <state name="ask_weather_area" position="transition"> … </state></function>
<?xml version="1.0" encoding="UTF-8" ?><vxml version="2.0" xmlns="http://www.w3.org/2001/vxml"><form><property name="message" value="'SessionID=99 IP=127.0.0.1 PORT=1001 COM=GRAMMAR PARAM=C:\CAMMIA\grammars/;enter_direction_jp.gad;enter_parking_jp.gad;enter_restaurant_jp.gad;cancel_jp.gad;ask_weather_jp.gad;Correction.gad'" /> <block><prompt> Please tell me the area and date</prompt> </block><field name="weather_intention" /> <field name="weather_area" /> <field name="weather_date" /> …<filled namelist="weather_intention"> <if cond="(weather_date!=undefined)"> <submit namelist="recog_result weather_date" next="ask_weather_area.jsp" /> </if> <if cond="(weather_area!=undefined)"> <submit namelist="recog_result weather_area" next="ask_weather_date.jsp" /> </if>…</filled> …</form></vxml>
<field name=“parking_intention" /><field name=“restaurant_intention" /><field name=“direction_intention" />
<filled namelist=“direction_intention"> <goto next="ask_direction.jsp"/></filled>
14
Language Technologies Institute
ScenarioTemplate Format
<template> ::= <variables><states><variables> ::= <variable+><variable> ::= <statename>_<id>:<type><statename> ::= <letter+><id> ::= <letter+><type> ::= String|ArrayList<letter> ::= a|b|c|d|e|f|g|h|i|j|k|l|m|n|o|p|q|r|s|t|u|v|w|x|y|z|
A|B|C|D|E|F|G|H|I|J|K|L|M|N|O|P|Q|R|S|T|U|V|W|X|Y|Z
<states> ::= <state+><state> ::= {<question+><confirmation><response>}<question> ::= <variable+><prompt><backprompt><prompt> ::= { <letter> | <variable> | " " }<backprompt> ::= <prompt><confirmation> ::= <prompt><response> ::= <prompt>
15
Language Technologies Institute
ScenarioXML DTD<?xml version="1.0" encoding="UTF-8"?><!ELEMENT function (state*)><!ATTLIST function name CDATA #REQUIRED><!ELEMENT state (grammar?, result*, prompt?, backprompt?, filled?, jump?, nofound?)><!ATTLIST state name CDATA #REQUIRED position CDATA #IMPLIED><!ELEMENT grammar EMPTY><!ATTLIST grammar src CDATA #REQUIRED><!ELEMENT result (field*, list*, param*)><!ATTLIST result class CDATA #REQUIRED method CDATA #REQUIRED><!ELEMENT field (#PCDATA)><!ELEMENT list (#PCDATA)><!ATTLIST field condition CDATA #IMPLIED><!ELEMENT param (#PCDATA)><!ELEMENT prompt ANY><!ELEMENT text (#PCDATA)><!ELEMENT backprompt ANY><!ELEMENT filled (field*)><!ELEMENT jump (nextstate*, default?)><!ELEMENT nextstate (field*)><!ATTLIST nextstate next CDATA #REQUIRED><!ELEMENT default (#PCDATA)><!ATTLIST default next CDATA #IMPLIED><!ELEMENT nofound (#PCDATA)>
16
Language Technologies Institute
SXML Development Kit (SXMLDK) Two-step compilation by SXMLDK
ScenarioXML Development Kit
ST CompilerScenario
Template
SXML Compiler
ScenarioXML
JSP files
Dialog Names Grammars
Voice XML
18
Language Technologies Institute
Next Introduction Approach in CAMMIA Dialog Management Framework
System Architecture ScenarioTemplate/ScenarioXML ScenarioXML Development Kit
Pilot Systems & Experiments Current Research Focus Future Work
19
Language Technologies Institute
Pilot Systems
Prototype I (2002) Focus on building general architecture to support
SXML HTTP is session-less and VoiceXML needs to keep
user’s utterances in each session SXML provides session variable to keep user’s
utterances <session-variables>
<persistent names="DMEnv_persistent_topic DMEnv_persistent_weather_date DMEnv_persistent_weather_area"/>
<transient names="DMEnv_answer_weather_date DMEnv_answer_weather_area DMEnv_answer"/>
</session-variables>
20
Language Technologies Institute
Pilot Systems
Prototype II (2003) Improve Prototype I Develop ScenarioTemplate & SXMLDK Filled values and dialog states are stored
in back-end dialog controller Only the current values are stored in
VoiceXML
21
Language Technologies Institute
Pilot Experiments Compare Prototype I & II Test environments
VXI: Intel Pentium III 800MHz, 512M RAM DM: Intel Pentium II 400MHz, 224M RAM
Sample Dialog U1: Weather information, please
S1: Please tell me the area and the date U2: Tomorrow S2: Please tell me the area U3: Pittsburgh S3: Are you asking the weather for Pittsburgh tomorrow? U4: Yes. S4: The weather for Pittsburgh tomorrow will be sunny.
date
area
confirmation
response
22
Language Technologies Institute
Comparison of Bandwidth Utilization
To compare bandwidth utilization, VXML file sizes were measured
Sub Task Prototype I Prototype II
Ask date 6266 931
Ask area 1949 618
Confirmation 1709 607
Response 3110 283
Avg VXML Size 3258 610
Unit: Byte
23
Language Technologies Institute
Comparison of User Wait Time
To compare end-user wait time, loading time in VXI were measured
Sub Task Prototype I Prototype II
Ask date 5.25 0.88
Ask area 3.92 1.28
Confirmation 4.47 1.50
Response 3.44 1.14
Avg Loading Time
4.27 1.20
Unit: second
24
Language Technologies Institute
Summary of Prototype II
Improvement Efficient VXML management: VXML includes only
the current state information => faster & less use of network bandwidth Easy to add new dialogs using ScenarioTemplate Easy to support multiple languages by changing
ScenarioTemplate prompts and grammars Issues
Predefined dialog states and prompts Hard to support more complex dialogs
25
Language Technologies Institute
Example of Complex DialogU1: Tour guide, please.S1: Tour guidance is started. Where is a destination?U2: Carnegie Mellon UniversityS2: The distance to the destination is 100 miles. It takes about 2
hours.
U3: Do you know a good restaurant near here?S3: What kind of food do you like?U4: I want to eat Italian food.S4: There are Olive garden, Bravo and Laromana.U5: Well, I'd like to go to Laromana.S5: It does not have a parking lot. Is it OK? U6: Tell me the one which has parking lots.S6: Olive garden and Bravo have parking lots. U7: Which one is closer?S7: BravoU8: I would like to go there.S8: Do you want to add it as a way point?U9: YesS9: I set Bravo as the waypoint.
26
Language Technologies Institute
Next Introduction Approach in CAMMIA Dialog Management Framework
System Architecture ScenarioTemplate/ScenarioXML ScenarioXML Development Kit
Pilot Systems & Experiments
Current Research Focus Future Work
27
Language Technologies Institute
Current Research Focus
Flexible dialog management Push model: Offer important information even
though user does not request Ex) do not take credit card, do not have parking
lot Search data from the list which was already
retrieved Support comparison (the cheapest, closer) Anaphora resolution (there, it) Add way point to Navigation Map
28
Language Technologies Institute
Current Research Focus
Robust dialog management for signal loss Task Manager
- Retrieval task management for signal loss Dialog Manager
- Located in vehicle Info Manager
- Retrieve data from the remote database servers- Maintain local cache with timestamps
29
Language Technologies Institute
Extended Architecture
ADM
DialogController
Dialog Manager
Voice
User Interface
InfoManager
Navigation System
Map Display
Julius
Navigation System
Simulator
LocalCache
DB
DataServer
…
VehicleRemote Server
DB
TaskManager
30
Language Technologies Institute
Dialog Manager To support information seeking dialogs, it has three
dialog states Search
- Ask the user to fill the minimum search constraints- Display the search resultsEx) There are three Italian restaurants near hear, A, B and C
Search refinement - User can narrow down the searched items with different search
options such as price and distanceEx) Which one is closer?
Selection - Automatically check important information (Push model)- Add way point to the Navigation mapEx) I set it as the waypoint
31
Language Technologies Institute
Dynamic State TransitionCheck search
constraints
Display the results
End dialog
need constraints=yes
Ask constraints
need constraints=yes
Selection
Display detailed infoAdd way point if user wants
satisfiedNot satisfied
need features=no
need selection=no
Notify it to the user
Find new lists from the previous resultsDisplay it
Search options
Search options
Check special info
need features=no
Search Refinement
Selection
Search
Search options
Selection
New constraints
32
Language Technologies Institute
Future work
More work on Task Manager Anaphora and ellipsis resolution Multi-modal interface
Integration of screen with Navigation map Complementary prompts (voice and screen)
Dynamic grammar generation More intelligent push model based on user
preference Missed turn