SIG Center for Computer Graphics Pragbot 2.0 Moving Pragbot Language Interactions Toward More Realistic Situations Oct 2009 Review Chris Czyzewicz, Norm

SIG Center for Computer Graphics

Pragbot 2.0Moving Pragbot Language Interactions Toward More

Realistic Situations

Oct 2009 ReviewChris Czyzewicz, Norm Badler


Purpose

We were tasked with collecting a corpus of data to study language interaction

– Emphasis on intentions Last year, we presented Pragbot, which

collected 50 transcripts

– Thanks Karl Schultz et. al.


Pragbot

% pragbot_chat_log_2007.11.01 AD at 19.48.20 EDT.txt

Player 2: i have 4H

Player 1: I want it!

Player 1: where is it?

Player 2: should i leave it for you somewhere?

Player 1: sure

Player 1: where are you?

Player 2: okay, where are you?

Player 1: I'm near the top

Player 2: i'm left side.

Player 1: next to the gap near the middle

Player 2: i'll leave the card in the upper left corner.

Player 1: awesome


Pragbot 1.0 Review


Pragbot 1.0

• Two player game played through an applet

• Players searched through an environment to find cards

• Objective was to find 6 cards in a row, all from the same suit

• Players discussed strategy

• Transcripts were recorded and studied


Pragbot 1.0

• Very successful in gathering transcripts

• Flexible architecture– No installation (played through an

applet)– Platform independent

• Game requires some strategy, without a steep learning curve

– Easy controls and interface


Pragbot 1.0 panel comments

• We received feedback from the panel last year

• 2D experience was not immersive

• Not very realistic for the target of our application

• Metaphor of card finding should be changed


Pragbot 2.0

• Objective of Pragbot 2.0 is the same as the original

– Collect transcripts from participants

– Determine how they convey intention

– Specifically how a human might convey intention to a robot

– Examine any other interesting bits of the conversation


Pragbot 2.0

• Important that we retain the portability of the original Pragbot

– Shouldn't need to install anything– Should be platform independent

• Maintain ease of control and interface– Game should be simple to play


Pragbot 2.0


Pragbot 2.0 implementation

• First Person Shooter view

• Nothing to install

– Uses JNLP to download and launch the game

– Needs >= Java 1.5

– Windows/Linux at the moment

– Runs in its own, full screen window



• Game metaphor has changed

– A commander and a robot looking to save hostages, eliminate bombs and terrorists

– Game takes place in a Wolfenstein like environment

• Characters have different roles in the game

– Requires coordination between the two players



Guidance: “Develop a common simulation and experimentation scenario … which is demanding and sophisticated….”,

• The new metaphor and implementation of the game have helped us with the scenario

– On the same page with scenario demo

• Promoted good discussion during the year about what would be good/bad for a demo

• Nice sandbox for testing potential scenario ideas


Pragbot 2.0


Pragbot 2.0


Pragbot 2.0


Pragbot 2.0


Pragbot 2.0


Game Dynamics

• Two possible characters

– Commander (CMDR), a human

– Junior (JR), a robot

• Hostages that need to be rescued

• Terrorists

– Being improved as we speak

• Bombs


Transcripts

• Data can be collected in the same way as Pragbot 1.0

• Players communicate via chat


JR

• One player plays as JR

• Defuses bombs, cannot remove barriers

• Cannot rescue hostages

• He does the dangerous work

• Does not receive mission updates


CMDR

• Other player is CMDR

• Kills terrorists

• Is killed by bombs if he is in the same room/hallway

• Rescues hostages

• Gets mission updates from HQ


Typical Gameplay

A mission is described to CMDR He relays that information to JR, who

then begins to explore JR updates CMDR, and requests help

when he can't handle something Game is completed when mission is

accomplished (terrorists killed, hostages rescued, etc.)


Sample Transcript


Pragbot 2.0 video


Data Collection

• We have collected several transcripts

• Some transcripts not incredibly interesting

• Good ideas for next game objective

– More strategy aspects


Website

• If you would like to try out Pragbot 2.0, you can go to the website and play a game

– Plus we would love your transcript!

• www.seas.upenn.edu/~eczyzewi/pragbot

• Alpha game is being updated regularly

• Original game is stable and playable


Future Work

• Currently in the process of adding more intelligent enemies

• Terrorists will attempt to flee the building

• Players will have to coordinate to keep them pinned down

• Multiple bots?


Future PhD Work

• Bots with memory of what they, and other bots, have done

– Also memory of objects, world

• Memory = list of PARs• Reasoning based on this

memory

bot/agent

PAR-1 PAR-2 PAR-3 PAR-4‘me’ …

PAR-a PAR-b PAR-c PAR-d‘b2’ …

PAR-A PAR-B PAR-C PAR-D‘b6’ …


Future PhD Work

• As mentioned earlier, would be ideal to play the game with only one person

• Person gives commands, bot reacts• Will require memory and

'envisionment'– Ability to determine whether or not

proposed action is possible, or potential pitfalls

• Please wait until next year to grill me on this stuff


Thanks!

• Questions?

Documents

SIG Center for Computer Graphics Pragbot 2.0 Moving Pragbot Language Interactions Toward More Realistic Situations Oct 2009 Review Chris Czyzewicz, Norm