Upload
rupali
View
26
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Jacob an instruction agent in VR. Marc Evers ([email protected]) 6 March 2000. Contents. context & project description approach & technology current state & next steps architecture of the system design details conclusions. Context. pilot project of VR Valley Twente - PowerPoint PPT Presentation
Citation preview
2
Contents
context & project descriptionapproach & technologycurrent state & next stepsarchitecture of the systemdesign detailsconclusions
3
Context
pilot project of VR Valley Twente objective: establishing a regional
knowledge centre on VR organisations: UT (INF, WB), HE, HSA,
Carp, MST, ... 6 pilot projects http://www.vr-valley.com/
4
Jacob Project
Jacob as assistant tasks: instruction, navigation, etc.
Jacob shows intelligent behaviourmultimodal interactioninteraction between multiple agentsmanipulation of objects in the virtual
world
5
Approach
use existing knowledge and theories Intelligent Tutoring Systems, Computer
Graphics, AI, … software engineering focus
how to build such a system in a maintainable, adaptable, …, way
use OO techniques & software architecture knowledge
6
Technology
runs on any PC or Workstation with web browser+VRML browser+3D graphics card
programming languages Java 1.1; VRML 97
standards H-Anim for Jacob’s body
under consideration: Java 3D, JavaSpeech
7
Current state
Jacob teaches construction task Towers of Hanoi
manipulation of objects in virtual world by Jacob and by user
Jacob has explicit model of the tasksimulation of physical processes
gravity, collisionssimple animation
8
Next steps
natural language dialogue‘intelligence’ for Jacobmulti user Jacob (DeepMatrix)integration in VMC, in 2 steps:
first, Jacob will play isolated role in VMC second, Jacob interacts with VMC
9
Basic architecture
3D platform: VRML/EAI; Java 3D; ...
abstraction of 3D platform
abstract 3D World: agents, cubes, tables, ...
‘task’ layer (?)
10
World View
Cube Avatar
VWorld
avoidCollision()enforceWorldConstraints()agentMovesObject()update()
VEObject
position : Vec3DboundinBox : BoundingBox
canCollide()avoidCollision()getBoundingBox()startGravity()isOnTopOf()lock()isLocked()setPosition()
0..*
+agents
0..*
+objects
0..*0..*
Jacob
lookAt()moveObject()walkTo()shakeHead()
11
Class VWorld
simulation of physical aspects gravity, collisions other physical constraints
management of VE objects agents & objects
keeps track of the state of VE objects
observations
12
Class VEObject
defines general behaviour of VE objects bounding box, position, size gravity, (b)locking relations like isOnTopOf
13
Subclasses of VEObject
Cube detects picking, dropping, dragging by
userAvatar
knows position and orientation of userJacob
Jacob’s manifestation in the world lookAt, moveObject, walktTo, shakeHead
14
‘Physics’ protocolc : Cube : VWorld : GravityThread : World
Listener : DraggingSensor
NodeProxy
2: agentMovesObject(c)
1: drop performed
3: enforceWorldConstraints( )
4: startGravity( )
5: start( )
7: update( )
6: setPosition( )
8: worldStateChanged()
15
‘Physics’ protocol
c : Cube : VWorld
: GravityThread
: DraggingSensorNodeProxy
: WorldListener
3: enforceWorldConstraints( )7: update( )
2: agentMovesObject(c)
4: startGravity( )
5: start( )
6: setPosition( )
8: worldStateChanged()1: drop performed
16
Collision avoidance protocolc : Cube : VWorld : VEObject b1 : Bounding
Boxb2 : Bounding
Box
1: setPosition(v)2: avoidCollision(c,v)
3: avoidCollision(c,v)
4: canCollide( )
6: getBoundingBox( )
5: getBoundingBox( )
7: avoidCollision(b2, v)8: get...
17
Collision avoidance protocol
c : Cube
: VWorld
: VEObject
b1 : BoundingBox
b2 : BoundingBox
4: canCollide( )5: getBoundingBox( )
1: setPosition(v)
2: avoidCollision(c,v)
3: avoidCollision(c,v)
6: getBoundingBox( )
7: avoidCollision(b2, v)
8: get...
18
Instruction View
InstructionMethod
start()
InstructionRule
trigger()
0..*
0..*
0..*
0..*
InstructionModel
taskStateChanged()
Jacob
lookAt()moveObject()walkTo()shakeHead()
manipulates
VWorld
avoidCollision()enforceWorldConstraints()agentMovesObject()update()
Schema<<Interface>>
SchemaIterator
updateCurrent()
Task
worldStateChanged()currentSchema()
observes
observes
19
Class Task
knows how to create the ‘scene’has knowledge on performing the task
Schema: hierarchical structure Composite design pattern
action, explanation, pre/postcondition, invariant
schema interface is generic SchemaIterator determines current task
state
20
Class InstructionModel
InstructionRule rules that fire other rules or start
instruction methods based on conditionsInstructionMethod
executed until goal or state is reached example:
while user does nothing: wait, give hint, wait, show move
21
Task update protocol : VWorld t : Task : Schema
Iterator : Schema : Instruction
Model
1: worldStateChanged( )
2: updateCurrent( ) 3: precondition(t)
4: postcondition(t)
5: invariant(t)
6: getSubSchemas( )
7: taskStateChanged( )
8: currentSchema( )
22
Task update protocol
: VWorld
t : Task : SchemaIterator
: Schema
: InstructionModel
1: worldStateChanged( ) 2: updateCurrent( )
7: taskStateChanged( )
8: currentSchema( )3: precondition(t)4: postcondition(t)
5: invariant(t)6: getSubSchemas( )
23
Concluding remarks
current results can be generalised Jacob based on existing knowledge and
theories instruction agent concept is very
generalinstruction, assistance, design guidance…
demo & documentation http://www.cs.utwente.nl/~evers/jacob/
24
Jacob Screenshot