Learning Agents Center Computer Science Department George Mason University

Learning Agents CenterComputer Science Department

George Mason University

Dorin Marcu02-09-2004

IT 803 Spring 2004 – Mixed-Initiative Intelligent Systems – Prof. G. Tecuci

COLLAGEN:Mixed-Initiative Interaction with a Collaborative Agent

Presentation Outline

A brief introduction to COLLAGEN An air travel application built with COLLAGEN A more detailed look at COLLAGEN:

architecture communication shared awareness tasks control evaluation

Summary of the main ideas in COLLAGEN Lessons learned References

Research Motivation

current systems lack support for the user’s problem solving process during extended periods of time:

– the order in which actions must be performed by the user and the system is often inflexible

– it’s hard to recover from mistakes

– each system has its own interaction conventions

Explored Solution

support the problem-solving level of human-computer interaction via a collaborative software agent

develop a new paradigm for human-computer interaction which explicitly supports the user’s problem-solving process based on current theories of collaborative discourse

Assumption

A human-computer interface based on familiar human discourse rules and conventions will be easier for people to learn and use than one that is not.

Collaborative Interface Agents

From (Rich and Sidner, 1998), page 317

COLLAGEN: A Collaboration Manager

A collaboration manager is a software component that:- mediates the interaction between a software interface agent and a user

- keeps track of the linguistic and attentional state of a discourse (similarly to a discourse manager)

- keeps track of the collaborative intentions of the participants

COLLAGEN: A Collaboration Manager (contd.)

A collaboration manager is less than a fully automated planning system because:

- it does not by itself decide what the agent should do or say next (though it may provide some candidates)

- it provides a representation for recording the decisions that the agent has made and communicated.

Definition of Mixed-Initiative in COLLAGEN

Mixed-initiative reasoning in COLLAGEN can be defined as a discourse-based collaboration between a human user and an interface agent that attempt to achieve shared goals by decomposing them into sub-goals and primary actions for which they have complementary solving capabilities.

Characteristics of Mixed-Initiative in COLLAGEN

one human – one agent, discourse-oriented collaboration partial, task-oriented mixed-initiative system:

some tasks have fixed-initiative flags, specifying which participants can solve them

the human participant has precedence over the agent, and can ignore the contributions of the agent

initiative taking: proposing a goal proposing a recipe for solving a goal solving a (part of a) recipe delegation

Main Features of COLLAGEN

both participants know and intend that all their actions are observed:

reporting communication (‘I have done x’)

direct observation

Main Features of COLLAGEN (contd.)

the mixed-initiative capabilities of the agent arise from the interplay of two sources:

application-independent algorithms and data structures in COLLAGEN

application-specific code and libraries in the agent

a library of recipes that specify the typical steps and constraints for achieving certain goals

arbitrary pattern-action rules


supports mixed-initiative by:

interpreting discourse acts

maintaining a model of the achieved and expected tasks and goals of the user and agent

the user makes the final decision for the problem-solving process and can ignore the contributions of the agent


the interaction model is based on a formal representation of the mutual beliefs about the goals and actions to be performed, and the capabilities, intentions, and commitments of the participants



architecture communication shared awareness tasks

control evaluation


Air Travel Planning System

You are a Boston-based sales representative planning a trip to visit customers in Dallas, Denver, and San Francisco next week.

You would prefer to leave on Wednesday morning, but can leave on Tuesday night if necessary.

Your customer in Denver is only available between 11 a.m. and 3 p.m. on Thursday.

You would prefer to fly as much as possible on American Airlines, because you have almost enough frequent-flier miles to qualify for a free trip this summer.

You absolutely must be home by 5 p.m. on Friday to attend your son’s piano recital.

Air Travel Planner Interface


Air Travel Planner Interface (contd.)


Traditional Use of Planner

Seven visitors and staff members were asked to solve this and similar problems and their behavior was recorded via informal notes and the logging facilities built into the application

A typical problem solving session lasted about 15 minutes and entailed about 150 user actions (mouse clicks).

User Problems with the Traditional Planner

Various forms of getting stuck and getting lost:

trouble knowing what to try next when the trip has been over- or under-constrained

trouble keeping track of which combinations of routes and constraints were already examined

workflow interruptions caused by the use of application’s functions (e.g. “Snapshot” – save context)

Collaborative Interaction Example

(Rich and Sidner, 1998), page 323

Collaborative Interaction Example (contd.)





control evaluation


The Architecture of COLLAGEN


The Interface Agent

COLLAGEN does not provide tools for building a complete agent – different types of agents can be supported (rule-based expert systems, neural nets, or a completely ad hoc collection of code)

COLLAGEN provides a generic framework for recording the decisions made and communicated by the agent (and the user), but not for making them.

Question

Why would such a generic framework be useful? Effective?

Answer

Why would such a generic framework be useful? Effective?

Because it will allow the reuse of components for:- building collaborative agents- application-independent discourse manager

The Execution Cycle a communication or observation event arrives at the discourse interpretation module

the discourse interpretation module updates the discourse state

a new agenda of expected communication and manipulation acts is computed by the discourse generation module

the agent may decide to select an entry in this new agenda for immediate execution (according to the agent’s initiative strategy)

the user communication menu is updated with all the communication actions in the agenda for which the actor is either unspecified or the user

The Default Agent Initiative Strategy

The default agent implementation that is included in COLLAGEN always chooses to perform the highest priority action in the current agenda for which the actor is either unspecified or itself.

The priorities are manually associated with actions.Previous experiments also used pair-wise comparison rules. (Rich, 2002)

The Architecture Issue

COLLAGEN has a generic framework for developing interface agents that can collaborate with human users to solve shared goals.

The generic framework must be customized for a specific application by developing application-dependent :- recipes- action models- methods for performing actions and for observing actions performed by the user- collaboration behavior



architecture communication tasks shared awareness control evaluation


Window Sharing

both the user and the agent have a dedicated “home” window that is used for communication between them

each home window contains an identifying face and has an associated cursor

the home windows are serviced by separate processes to support asynchronous mixed-initiative interaction


Types of Agent Communication

printing English text in the agent’s home window

acting on the application’s interface with its cursor while approved and observed by the user

The user can ignore the suggestions proposed by the agent.


Types of User Communication

selecting from a menu of communications expected by the discourse interpretation algorithm (that were generated from the underlying model of discourse)

acting directly on the application with his/her cursor (the agent always observes the user’s actions through a generic layer in the application that mirrors semantic actions into the input buffer of the agent process)



The Communication Issue

COLLAGEN is based on a discourse theory of collaboration in which:

the agent communicates by: generating natural language text or user-selectable menus/buttons performing direct actions in the interface that the user can observe

the user communicates by: selecting agent generated menus/buttons performing direct actions in the interface that the agent can observe




control evaluation


Collaborative Discourse Theory

Collaboration is a process in which two or more participants coordinate their actions toward achieving shared goals. Most collaborations between humans involves communication.

Discourse is a technical term for an extended communication between two or more participants in a shared context, such as a collaboration.

Collaborative Discourse Theory (contd.)

Three interrelated types of collaborative discourse structures (Grosz and Sidner, 1986):

intentional structure (formalized as partial SharedPlans)

linguistic structure (includes the hierarchical grouping of actions into segments)

attentional structure (captured by a focus stack of segments)

Intentional Structure for the Air Travel Planner

the user knows the constraints on travel

the agent has access to a data base of all possible flights

both participants have a common goal (to find an itinerary that satisfies the constraints)

Intentional Structure for the Air Travel Planner (contd.)

both participants:

have agreed on a sequence of actions (a recipe) to accomplish the common goal (e.g., choose a route, specify some constraints on each leg, etc.)

are capable of performing their assigned actions

intend to do their assigned actions

are committed to the overall success of the collaboration (not just the successful completion of their own parts).

SharedPlans

SharedPlans is a formal representation of the participants’ mutual beliefs about the goals and actions to be performed, and of their capabilities, intentions, and commitments.

Representation of SharedPlans in COLLAGEN


Features of SharedPlans

partial (due to incomplete knowledge) – communication is required to fully specify them

recursive

their planning and execution is usually interleaved for each participant and among participants

SharedPlans in COLLAGEN

COLLAGEN provides a generic framework only for recording the order in which planning and execution occur.

COLLAGEN does not currently provide a generic framework for execution (interleaving planning and execution).

Discussion

No generally accepted domain-independent theory of how people manage the interleaving of planning and execution.

Best candidate: the BDI (belief/desire/intention) frameworks.

Discourse Segments

There is general agreement that discourse in human-human interactions has a natural hierarchical structure, the elements of which are called segments.

A segment is a contiguous sequence of communicative actions that serve some purpose (e.g. to achieve shared knowledge of some fact).

Example of Discourse Segments


Focus Stack

The focus stack contains discourse segments in the order in which they are created during the natural flow of a collaborative discourse.

It captures the shifting focus of attention in a discourse.

New segments and sub-segments are created, pushed onto the focus stack, completed, and then popped off the stack as the SharedPlan unfolds in the conversation.

Example of Focus Stack


Discourse State in COLLAGEN


DiscourseState

Representation

Discourse State Representation (contd.)

The discourse state representation contains:

the plan tree, which is an approximate representation of a partial SharedPlan)

the focus stack

the history list, which contains top level segments that have been completed and removed from the focus stack

Discourse Interpretation in COLLAGEN


Discourse Interpretation

The discourse interpretation module determines how the current direct communication or observed manipulation action can be viewed as contributing to the current discourse purpose (of the top segment in the focus stack).

Recognized Acts

The discourse interpretation module recognizes acts that: directly achieve the current purpose

correspond to one of the steps in a recipe for the current purpose

identify the recipe to be used to achieve the current purpose

identify who should perform the current purpose or a step in the current recipe

identify an unspecified parameter of the current purpose or a step in the current recipe.

Questions

What types of acts that may occur are not recognized by the discourse interpretation module?

What can be done in those cases?

Answers

What types of acts that may occur are not recognized by the discourse interpretation module? What can be done in those cases?

interruptions (a segment that does not contribute to its parent)

one participant decides to address a new goal the participants pursue multiple goal in the same time

focus stack management

incomplete recipe let the user manage the focus stack

Questions

Discourse interpretation looks similar to plan recognition, which is known to be exponential in the worst case.

Is discourse interpretation necessarily exponential? What can be done to avoid this?

Answers

Discourse Interpretation is generally not exponential because:

the participants can try to make sure that their intentions can be understood without a large cognitive search

the participants can ask clarification questions to minimize search

the search is performed through the steps of the current recipe or all known recipes for the current segment

Discourse Generation in COLLAGEN


Discourse Generation

Discourse generation is the inverse of discourse interpretation.

It produces a prioritized agenda of (possibly partially specified) actions which would contribute to the current discourse segment from the focus stack, and its associated SharedPlan.

Dynamic Generation of Agent Responses

Develop a plug-in architecture for reusable components that generate agent responses based on the last event (communication or action) and the current discourse status.

Develop:- application-independent- application-dependent- role-dependent (e.g. assistant vs. tutor)plug-ins for the developed architecture.

Plug-in Architecture

(Rich et all., 2002), page 784

Plug-ins for Generating Agent Responses

A plug-in for generating agent responses is an object that is associated with an internal state and an algorithm for generating responses based on its state, the discourse state and the event received by the discourse interpreter module.

A plug-in: can access and modify the discourse state. can access the recipes library. has a priority associated with all the responses it generates

The Plug-in Interface

update(event):Updates the private state of the plug-in (if any) and modifies the discourse state (if necessary) – for example by adding a goal or binding a parameter.

generate(event):Returns a (possibly empty) list of agenda items which would be an immediate response to event.

visit(node):Returns a (possibly empty) list of agenda items which would contribute to the goal of node.

Application-Independent Plug-ins (6)

Execute(goal): (default priority 0)If goal is primitive and all of its parameters are bound and its first parameter is bound to theagent, then return goal.

ProposeShouldUser(goal): (default priority 0)If goal is primitive and its first parameter is bound to the user, then return an utterance of the form “Please perform/say goal.”

AskWho(goal): (default priority 0)If goal is primitive and its first parameter is unbound, then return an utterance of the form “Who should perform/say goal?”

Role-Dependent Plug-ins

TeachStep(goal): (priority 100)If goal is teachable, then return an utterance of the form “The first/next step is goal.”

TeachInitiative(goal): (priority 130)If the student model indicates that the student should already know how to achieve goal, then return “You take it from here.”

PositiveFeedback(action): (priority 160)If action was live and expected, then return an utterance such as “Great”, “Nice”, “Right”.

Algorithm for Choosing Responses

add to the agenda the results in the order returned, of invoking the generate method of each plug-in on the received event, in the default plug-in order

add to the agenda the results in the order returned, of invoking the visit method of each plug-in, in the default plug-in order on the live, expected nodes (visited in breadth-first order)

do a post-pass to reorder the agenda based on any additional explicit preferences

Segmented Interaction History

provides the user with a structured guide to his/her problem solving process

can be used to retry, re-apply or undo past actions

displayed at the user’s request


Producing the Segmented Interaction History

Starting from the current discourse state:

list the purpose of each top level segment on the history list starting with the oldest

recursively list the purpose of each closed sub-segment followed by the purpose and elements of each open sub-segment, starting with the top level open segment (the base segment of the focus stack)

for each open sub-segment list the unexecuted recipe steps (if any) in the plan tree for that segment’s purpose, starting with the most deeply nested

Question

What are some advantages or disadvantages of COLLAGEN’s segmented interaction history?

Answer

Some advantages or disadvantages of COLLAGEN’s segmented interaction history:

is structured (interaction histories are typically flat)

reflects the user’s actual problem solving process

requires a structured discourse (interaction) model and its use in the problem solving process (interpretation, etc.)

The Shared Awareness Issue

Shared awareness in COLLAGEN is maintained by the agent through the discourse state, which is accessible to the user via the segmented interaction history.

It contains descriptions of the:- past interactions- current “stack” of execution reflecting the goals being solved.

It is updated based on:- direct manipulation actions performed by the user or the agent- user selections of agent generated menus/buttons




control evaluation


Agent Customization Through Task Models

The goal is to develop a formal model of the collaborative tasks being performed by the agent and the user.

Since the agent does not have to rely on the task model alone for its decision making, the model only needs to be complete enough to support communication and collaboration with the user.

The task model in COLLAGEN contains recipes and primitive actions.

The Recipe Library in COLLAGEN


Recipes

a recipe is an application specific resource used to derive a sequence of steps to achieve a given goal (the objective of the recipe)

(Garland at all., 2001), page 45

Recipes (contd.)

It is easy to represent recipes with a fixed number of steps in a simple recipe formalism.

More complicated recipes require step structures that depend on some parameters of the objective (e.g. the top level recipes for scheduling a trip working forwards or backwards).

Question

How can a recipe with variable structure be specified?

Answer

How can a recipe with variable structure be specified?

In COLLAGEN, a general-purpose procedural approach, called recipe generators, was developed:

recipes with variable structure are represented as procedures which, given an objective, return a recipe

a predicate can also be associated with a recipe to test whether it is still applicable as it is being executed

Manual Development of the Task Model initial versions of a task model are inferred from a small number of examples that show the most common solutions to key domain tasks

the model is generalized to cover additional examples that demonstrate solutions involving alternate orderings for actions, optional behavior, or alternate task decompositions (defining additional examples may force the expert to re-conceptualize the entire domain necessitating reworking many previous examples)

correct and complete learned models may need to be tweaked by the expert for other reasons (e.g. organization of a complete and accurate task model may be inappropriate for a collaborative agent)

verify the behavior of the model using the collection of past examples

Learning The Task Model From Examples

programming by demonstration

incremental learning from examples

allows updating past examples used in learning

regression testing with all examples

Learning The Task Model From Examples

Step one (optional): provide an initial list of primitive and non-primitive acts

Non-Primitives: MakeMealPrimitives: Boil, CookPasta, PrepareSauce, ServeDinner

Learning The Task Model From Examples (contd.)

Step two: define un-annotated example

GetPastaBoilCookPastaPrepareSauceServeDinner


Step three: (partially) annotate example

[MakeMeal][PreparePasta]

GetPastaBoilCookPasta

PrepareSauceServeDinner


Step four: learn (partial) recipe from example

nonprimitive act PreparePasta

primitive act GetPasta

recipe Boil_CookPasta_GetPasta achieves PreparePasta

steps Boil boilCookPasta cookPastaGetPasta getPasta

constraints getPasta precedes boilboil precedes cookPasta


Step five: define additional example(s)

[MakeMeal]GoToKitchen(kitchen2)[PrepareSauce]

Boil(water3)CookClams(clams8,water3)MakeClamSauce(clams8)

[PreparePasta]Boil(water3)CookPasta(linguini11,water3)

ServeDinner(kitchen2)


Step six: refine learned recipe

nonprimitive act PreparePastaparameter Water water

primitive act GetPasta

recipe Boil_CookPasta achieves PreparePasta

steps Boil boilCookPasta cookPastaoptional GetPasta getPasta

bindings achieves.water = cookPasta.waterconstraints boil.water = cookPasta.water

cookPasta.water = water3cookPasta.pasta = linguini11getPasta precedes boilboil precedes cookPasta

Revive Viewer

(Garland et all., 2001), page 49

The Learning Problem

Input: (partially annotated example)

Output: recipe(s) + refined examples (Garland et all., 2001), page 46

Experimental Results

The impact of annotations on the number of needed examples:

O: orderingE: equalitiesP: propagatorsAll: includes optional (Garland et all., 2001), page 47

Recipe Library Statistics

8 recipes defined in terms of 15 different goal or action types for the air planner system (the authors estimate that the final number should be about twice as much) (Rich and Sidner, 1998)

29 recipes, 67 recipe steps, 36 primitive acts and 29 non-primitive acts for the Symbol Editor application (Garland el all., 2001)

8 recipes, 19, recipe steps, 13 primitive acts and 4 non-primitive acts for the artificial cooking world model (Garland el all., 2001)

The Task Issue

Collaboration is modeled in COLLAGEN through recipes and primary actions.

At task modeling time, some tasks can be designated to be performed by the user or by the agent, the rest being open.

Example of user-designated tasks: specifying route or constraints.

Example of agent-designated tasks: search data-base for flight info.

Open tasks can be negotiated based on user preferences (e.g. asking the user who should perform a task) and roles (tutoring vs. assisting).




control evaluation


Control Principles

a recipe needs to be identified for the current goal (identification of a recipe may be achieved by asking the other participant)

a goal or action may be performed when all of its parameters are known and all of its predecessors (in the current recipe) have been achieved

a goal or action may be performed by any participant who is capable, unless a specific participant has been specified

Global vs. Local Initiative

most work is done on local initiative and its aspects:

turn taking and conversational control

interruptions, and

grounding (how the current speaker indicates that she has heard and understood the content of the previous speaker’s turn)

true mixed-initiative systems must address both global and local initiative

Global Initiative

application-independent: based on the current discourse state and the agenda computed from it, a COLLAGEN-based agent has an explicit choice of relevant things to say at most points in a conversation

application-dependent: the agent can choose between answering a question, performing an interface action or some other behavior, according to the specifics of the application domain

Negotiation as Global Initiative

negotiation is the ability to resolve differences in beliefs that are relevant to some shared goal

through negotiation, collaborators with differing points of view about the goals and recipes they undertake, as well as the state of the world at any point in time, can reach an agreement

currently, COLLAGEN does not incorporate a negotiation facility

Local Initiative

COLLANGEN does not provide a general algorithm for turn taking because of the limitations of the current theories of local initiative in conversation

Ad-Hoc mechanism developed (detailed in the following slides):

provide the user with means to relinquish control to the agent

provide the agent with means to get the user’s attention when the agent does not have control

Relinquish User Control

The user relinquishes the control when he/she does not want to contribute any further to the current SharedPlan and instead would like to see what the agent can contribute.

The agent was designed to interpret an ‘OK’ selected by the user (other than as an answer to a direct Yes/No question) as an signal of relinquishing control.

Request User’s Attention

The agent was designed with the ability to wave its cursor ‘hand’ when it has something important to contribute to the conversation.

The resulting behavior is considered very humanlike and affecting.

History-based Transformations

History-based transformations are used to alter the course of the problem solving process by reusing past actions (without modifying the history).

History-based transformation types (at segment level):

stop revisit retry undo reply

The availability of such transformations is application-dependent (the extent to which the application provides methods to alter its internal state)


The ‘Stop’ Transformation

motivation: the user wants to let the agent know that she/he is no longer working towards the current goal the agent has misunderstood what the current goal is

actions: pops the current segment off the focus stack without changing the application state if the purpose of the popped segment contributes to its parent, the appropriate un-bindings are also performed in the plan tree

applicability: only to open segments

The ‘Retry’ Transformation

motivation: the user wants to return to working on an earlier goal – achieved or not – and try achieving it in a different way

actions: stop and reset the stack and plan tree to their states at the start of the selected segment the recipe becomes unbound

applicability: to any type of segments

The ‘Revisit’ Transformation

motivation: the user wants to pick up where he/she left off working on an earlier goal

actions: stop and reset the stack and plan tree to their states at the end of the selected segment the recipe is preserved

applicability: only to closed segments

The Replay Transformation

motivation: the user wants to reuse earlier work in the current context

actions: all of the application acts in the selected segment are put together into one (possibly hierarchical) recipe which is then executed by the agent in the current context

Questions

What happens if some of the acts may not be valid in the current context?

To what type of segments is the ‘Replay’ transformation applicable?

Answers

What happens if some of the acts may not be valid in the current context?

Depending on the specific details of the agent’s interface to the application, such errors may need to be handled by application-specific code in the agent, or may be taken care of by the application’s existing API or graphical interface.

To what type of segments is the ‘Replay’ transformation applicable?

Open and close segments.Why?

The Control IssueThe control of initiative in COLLAGEN is partly managed:- in a generic way by the application-independent collaborative algorithms- in an application-dependent way by the related-designations in the recipes- in a role-dependent way by the discourse generation plug-ins.

The agent can negotiate with the user by asking who to perform a non-designated task, or by asking whether the agent-generated solution is acceptable to the user.

The user can relinquish control by asking the agent to perform a task (where possible).

Through history-based transformations, the user has direct control over the problem solving process.




control evaluation


Evaluation

No experimental evaluation to assess any strengths/weaknesses of the mixed-initiative approach vs. the classic approach (direct manipulation of the GUI-based applications.

My assessment:

since the user has the final decision on the performed steps and the communication model is straight-forward and natural, the collaborative approach should lead to better and faster problem-solving episodes.




control evaluation


Key Ideas in COLLAGEN multidisciplinary research in user interface, linguistics, and artificial intelligence collaboration between one user and one interface agent that is similar to human-human collaboration) guided and structured interaction based on formal model of collaboration (SharedPlans) separation of application-independent structures (collaboration theory, general framework) from application-dependent structures (recipes, agent customization, plug-ins) supports a user’s problem solving process by relating current actions to the global context and history of the interaction (discourse interpretation, discourse generation, discourse status, recipe library) oriented toward applications with direct manipulation of interfaces




control evaluation


Lessons Learned

using a formal task model for guiding collaboration and mixed-initiative interaction is effective (either SharedPlans or GOMS-like models) using a menu-driven interaction is simpler that natural language processing (although it is not clear if it is generally sufficient) using “plug-ins” to customize behavior seems to be an effective approach recognizing user actions and performing agent actions is application dependent, and might be challenging with applications that are not restricted to direct manipulation of GUIs more advanced user/agent models are necessary for different roles (e.g. tutoring) and for multi-human multi-agent collaborations




control evaluation


Charles Rich and Candace L. Sidner, “COLLAGEN: A Collaboration Manager for Software Interface Agents”, User Modeling and User-Adapted Interaction 8(3), 1998, Kluwer Academic Publishers, pp 315-350.

Charles Rich and Candace L. Sidner, “COLLAGEN: When Agents Collaborate with People”, in Proceedings of the First International Conference on Autonomous Agents, February 1997, Marina Del Rey, California, ACM Press, pp 284-291.

Charles Rich, Neal Lesh, Jeff Rickel and Andrew Garland, “A Plug-in Architecture for Generating Collaborative Agent Responses”, in Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems – AAMAS’02, July 15-19, 2002, Bologna, Italy, ACM Press, pp 782-789.

Andrew Garland, Kathy Ryall and Charles Rich, “Learning Hierarchical Task Models by Defining and Refining Examples”, in Proceedings of the International Conference on Knowledge Capture – K-CAP’01, October 22-23, 2001, Victoria, British Columbia, Canada, ACM Press, pp 44-51.

Papers Used For Presentation

Other Interesting Related Papers

Andrew Garland, Neal Lesh and Charles Rich, “Responding to and Recovering from Mistakes during Collaboration”, in Proceedings of the Workshop on Mixed-Initiative Intelligent Systems, the 18th International Joint Conference on Artificial Intelligence – IJCAI’03, August 9, 2003, Acapulco, Mexico, AAAI Press, pp 59-64.

Jacob Eisenstein and Charles Rich, “Agents and GUIs from Task Models”, in Proceedings of the 7th International Conference on Intelligent User Interfaces – IUI’02, January 13-16, 2002, San Francisco, ACM Press, pp 47-54.

Charles Rich, “Window Sharing with Collaborative Interface Agents”, ACM SIGCHI Bulletin, Volume 28 Issue 1, January 1996, ACM Press, pp 70-78.

Barbara J. Grosz and Candace L. Sidner, ‘Attention, Intentions, and the Structure of Discourse’. Computational Linguistics 12(3), 1986, pp 175–204.

Jim R. Davies, Abigail S. Gertner, Neal Lesh, Charles Rich, Candace L. Sidner and Jeff Rickel, “Incorporating Tutorial Strategies Into an Intelligent Assistant”, in Proceedings of the 6th International Conference on Intelligent User Interfaces – IUI’01, January 14-17, 2001, Santa Fe, New Mexico, ACM Press, pp 53-56.

Documents

Learning Agents Center Computer Science Department George Mason University