Http:// 1 of 25 Primatech: Models and Future Directions GVU Brownbag 9/30/1999 by Jim Davies, Tim Keenan,

  • View

  • Download

Embed Size (px)

Text of Http:// 1 of 25 Primatech: Models and Future Directions GVU Brownbag 9/30/1999 by Jim Davies, Tim...

  • Primatech: Models and Future DirectionsGVU Brownbag 9/30/1999by Jim Davies, Tim Keenan, Wasinee Rungsarityotin &Irfan Essa{jimmyd, tk, wasinee, irfan}

  • What is Primatech?Chantek: An orangutan who can signWill be on display at Zoo Atlanta; visitors can interact with himHe will be overloaded. Primatech simulates having a conversation with him

  • Schematic

  • Perception module150 Signs to recognize (Starner et al. recognized 40)Controlled environment (any background, etc.)Novice users (few signs used)Supporting speech data

  • Graphics moduleMemory of sign animationsHome positions?Essence with on the fly transformationsRemember every transition? (22500)

  • Chantek and trainer Lyn Miles

  • Using a gorilla model

  • An animated sign for like

  • Ape language researchLanguage trained apes know 140 signs, deaf children of 3 know 132ChantekDr. Lyn Miles150 signs (strict criterion)Anthropological approach

  • Three cognitive models1: Simple activation with Wordnet2: Express words that correspond to desire satisfaction3: Sentence verification

  • Model 1: Simple activation with WordnetACT-R uses spreading activation Relevance links only (from WordNet)distinguish words, signs, conceptsRetrieve anything, let spreading activation from goal do the work

  • Model 1: RepresentationInstead of:

    (banana ISA fruit color yellow)

    (yellow ISA color)

    I used:

    (cup-concept ISA concept word cup-word sign cup-sign r1 drink-concept)

    (cup-sign ISA sign meaning cup-concept)

    (cup-word ISA sign meaning cup-concept)

  • What do people ask? (Collected by Tara Stoinski)1. What do you like to eat? (5 people asked)2. What do you like to do? (3)3. Are you bored? (3)4. What is it like to have fur? (1)5. Do you like the animals that you live with? (1)6. What is it like to live in a cage? (1)7. Do you mind people staring at you all the time? (1)8. Do you have a favorite keeper? (1)9. Are you happy? (7)10. What do you want that you don't have? (4)11. Do you want to go back to the wild? (2)12. Tell me about evolution (1)

  • Model 1: Results1. What do you like to eat? response: food-eat-sign3. Are you bored? response: you-sign5. Do you like the animals that you live with? response: you-sign9. Are you happy? response: you-sign11. Do you want to go back to the wild? response: go-sign

  • Why so bad?definitions too dictionary-likebanana n 1: any of several tropical and subtropical treelike herbs of the genus Musa having a terminal crown of large entire leaves and usually bearing hanging clusters of elongated fruitsWhat about being yellow? Eating it? Deliciousness?Only 20% of words linkedYou eat and go were the only words in questions that were in memory.

  • Model 2: Express words that correspond to desire satisfactionACT-R has a goal structure but no way to express desires that are always around but no specifically acted upon all the time.Desires: food, drink, sex, freedom, and stimulation-- represented as memory chunksWords were linked to desires

  • Model 2: Relationships (screwdriver-word-meaning ISA relation2 relation meaning-relation first screwdriver-word second screwdriver-concept)

    (dog-isa-animal ISA relation2 relation subclass-relation first dog-concept second animal-concept)

  • Model 2: Input Representation (goal-1-1 ISA goal-talk sentence-id 1 word what-word next-word goal-1-2)

    (goal-1-2 ISA goal-talk sentence-id 1 word do-word next-word goal-1-3)

    (goal-1-3 ISA goal-talk sentence-id 1 word you-word next-word goal-1-4)

    (goal-1-4 ISA goal-talk sentence-id 1 word like-word next-word unknown)

  • Model 2: ResultsOften responds appropriately when offered something desiredLinguistically still impoverished

  • Model 3: Sentence verificationImplemented a simple grammarPart of speech relation addedWalked through input sentence, putting words in sentence templateChecks the memory for a fact that has the same subject, verb, and objectIf found, express verb

  • Model 3: ResultsCould verify relations found in the memory (Apple is a fruit, Chantek likes meat, Banana is yellow, etc.) Combination of Models 2 and 3 might account for much of his conversation

  • Cognitive Model contributionsInput sentence representationDesire representation in ACT-RNew relation representation/ relations can become activated themselvesFirst computer modeling of ape language

  • Future directionsPerception: Apply machine learning to signs, installation designGraphics: Collision detection, closer modeling of the apes actual motionCognitive Model: Modeling transcript data, integration of models 2 and 3

  • ThanksGraphics, Visualization & Usability seed grant 98-99Tara Stoinski for question collectionVictor Zordan for graphics helpLyn Miles for Chantek

  • More informationWe are looking for collaborators!

    Email me at