Intelligent Education Systems Group Dr Fuhua Lin 7/25/20111Fuhua
Lin, SCIS, FST, Athabasca University Slide 2 UT Austin Villa Wins
World RoboCup Championships 2011
http://www.utexas.edu/news/2011/07/19/villa_wins/ The key to
victory, says Peter Stone, was that he and his graduate and
undergraduate students taught their robots to teach themselves.
7/25/20112Fuhua Lin, SCIS, FST, Athabasca University Slide 3
Immersive Learning Environments Commercial platforms such as: World
of Warcraft for online gaming Second Life for online social
networking Positive outcomes of these environments a high level of
realism associated levels of engagement supporting and encouraging
social interaction Whether these positive outcomes can be
generalized and applied to the education community and weather
institution can adopt these environments and provide them as part
of their online ICT infrastructure ? 7/25/20113Fuhua Lin, SCIS,
FST, Athabasca University Slide 4 Game-based E-Learning the use of
a computer games based approach to deliver, support, and enhance
teaching, learning, assessment, and evaluation. (Connolly et al.,
2004) 7/25/20114Fuhua Lin, SCIS, FST, Athabasca University Slide 5
Goal Devise virtual learning environments that integrate AI and
game engines. 3Es: Effective, Efficient, Engaging 3Is: Intelligent,
Interactive, Immersive Adaptive Motivational elements: 4Cs:
challenge, curiosity, control, and context, creativity is the new
emerging C a (Lepper & Henderlong, 2000) Social relationship
Play and learn Target users: For all learners, both within and
outside of the classroom. What to do? Infrastructure for Building
Virtual Classrooms Research Development Evaluation 7/25/20115Fuhua
Lin, SCIS, FST, Athabasca University Slide 6 Requirements (1) ---
Believability Body Language how to detect it how to express it
adaptively and automatically. 7/25/20116Fuhua Lin, SCIS, FST,
Athabasca University The believability of what we communicate is
influenced 55% by body language For a six month baby, when you
smile to him/her, he/she may smile to you. Slide 7 Requirements (2)
---- Autonomy Game engines are still too complex for most educators
to implement their own learning games. avatar animation and facial
expressions, but these features must be controlled manually there
is no way to associate them with game events. stops short of
providing tools essential to many educational game activities, such
as question banks, score keeping, and user modeling.
7/25/20117Fuhua Lin, SCIS, FST, Athabasca University Slide 8
Requirements (3) Easy to Build The time and expertise required to
create believable NPCs and engaging learning activities based on
virtual-world technologies remains a significant barrier. How to
incorporate intelligence into NPCs? How to make the agents learn?
7/25/20118Fuhua Lin, SCIS, FST, Athabasca University Slide 9 Why
Agents and MAS? Complexity it is not practical to analyse and code
for every possible game state and every possible interaction
between the various game elements. Why Multiagent Systems? For
cooperative problem solving. For emergent behaviours. For global
optimization.... 7/25/20119Fuhua Lin, SCIS, FST, Athabasca
University Slide 10 Agents Agent Goal-oriented agent Reactive agent
7/25/201110Fuhua Lin, SCIS, FST, Athabasca University Slide 11
Agents Agent NPC Agent System Agent Student Agent Goal-oriented
agent Reactive agent 7/25/201111Fuhua Lin, SCIS, FST, Athabasca
University Slide 12 Agents Agent NPC Agent System Agent Student
Agent Goal-oriented agent Reactive agent Avatar NPC
7/25/201112Fuhua Lin, SCIS, FST, Athabasca University Slide 13
Agents Agent NPC Agent System Agent Pedagogical Agent Virtual
Student Virtual Audience Student Agent TimerScorekeeperAMA
Goal-oriented agent Reactive agent Avatar NPC 7/25/201113Fuhua Lin,
SCIS, FST, Athabasca University Slide 14 Agents Agent NPC Agent
System Agent Pedagogical Agent Virtual Student Virtual Audience
Student Agent TimerScorekeeperAMA Goal-oriented agent Reactive
agent Avatar NPC Tasks 1. Expressing verbal/non-verbal
communication actions 2. Learning from the student agents about the
students 3. Generating quizzes and hints 4. Reasoning about the
situation. 5. Providing narrative and dialogue functions that
provide increased engagement and immersion in the environment 14
Slide 15 Agents Agent NPC Agent System Agent Pedagogical Agent
Virtual Student Virtual Audience Student Agent TimerScorekeeperAMA
Goal-oriented agent Reactive agent Avatar NPC Tasks 1. Expressing
verbal/non-verbal communication actions Adaptively to increase user
engagement level. 2. Learning from the student agents about the
students 3. Generating quizzes and hints 4. Reasoning about the
situation. 5. Providing narrative and dialogue functions that
provide increased engagement and immersion in the environment TSI-
enhanced 15 Slide 16 Agents Agent NPC Agent System Agent
Pedagogical Agent Virtual Student Virtual Audience Student Agent
TimerScorekeeper Tasks: 1. Detecting and analyzing
verbal/non-verbal communication and behavior(e.g. emotion, gesture)
2. Learning from other agents 3. Modeling student (level,
preferences, learning styles, profiles) 4. Identifying social
relations AMA Goal-oriented agent Reactive agent Avatar NPC TSI-
enhanced 16 Tasks 1. Expressing verbal/non-verbal communication
actions Adaptively to increase user engagement level. 2. Learning
from the student agents about the students 3. Generating quizzes
and hints 4. Reasoning about the situation. 5. Providing narrative
and dialogue functions that provide increased engagement and
immersion in the environment Slide 17 Agents Agent NPC Agent System
Agent Pedagogical Agent Virtual Student Virtual Audience Student
Agent TimerScorekeeper Tasks: 1. Detecting and analyzing
verbal/non-verbal communication and behavior(e.g. emotion, gesture)
2. Learning from other agents 3. Modeling student (level,
preferences, learning styles, profiles) 4. Identifying social
relations AMA Goal-oriented agent Reactive agent Avatar NPC TSI-
enhanced Artifact 17 Tasks 1. Expressing verbal/non-verbal
communication actions Adaptively to increase user engagement level.
2. Learning from the student agents about the students 3.
Generating quizzes and hints 4. Reasoning about the situation. 5.
Providing narrative and dialogue functions that provide increased
engagement and immersion in the environment Slide 18 Environment
Programming in MAS Environment 7/25/201118Fuhua Lin, SCIS, FST,
Athabasca University Slide 19 Environment Programming in MAS
Environment 7/25/201119Fuhua Lin, SCIS, FST, Athabasca University
Slide 20 Environment Programming in MAS Environment
7/25/201120Fuhua Lin, SCIS, FST, Athabasca University Slide 21
Reactive Agents Perceive events Simple set of rules event action
(i.e., activation of a specific behavior) Actions are often known
as behaviours Example of a simple mail agent: if send mail then
check virus If new mail then check spam If spam then send message
to friends agents If new message then get new spam information
Pros: simple and efficient Cons: Action depending only on stimuli
Not flexible Not really autonomous 7/25/2011Fuhua Lin, SCIS, FST,
Athabasca University21 Slide 22 Reactive Agents with State Internal
state (internal knowledge) Update of internal state New state =
actual perception + old state The update may require Knowledge on
how the world evolves which can also dynamically acquired by the
agent Knowledge on how the agent actions influence the world Select
action (i.e., behavior) accordingly 7/25/2011Fuhua Lin, SCIS, FST,
Athabasca University22 An object is a sort of reactive agents, but
- It has no rule for action selection - It actions are directly
commanded by the external Example A mail agents that keeps track of
the users marking some messages as spams and take these into
account in future actions Slide 23 Goal-oriented agents Goal a
desired situation to eventually achieve The agent exploits the goal
and its knowledge select actions whose effect would be that of
approaching the goal How can a goal be selected? Search in the
state space Plannings Heuristics sub-optimal actions 7/25/2011Fuhua
Lin, SCIS, FST, Athabasca University23 Example: an agent to
minimize fragmentation in a hard disk - Knapsack problem - Do not
know the future but know the past - Select allocation of new files
based on some heuristics - An action does not necessarily minimize
the current fragmentation - Perform de-fragmentation action when
the computer is idle Slide 24 Utility-oriented Agents The Goal is
that of maximizing the current utility opportunistic behavior
Utility A function of some parameter, measuring the state of
goodness (with respect to the agent) of a situation Often, it
measures a trade-off between contrasting objectives Example An
agent to maximize CPU utilization Always select the ready process
The current choice may be sub- optimal with regard to the global
execution time of processes 7/25/2011Fuhua Lin, SCIS, FST,
Athabasca University24 Slide 25 Hybrid Architectures Mixing utility
and goals An agent that has to achieve a goal and, at the same
time, has to maximize a specific utility function Trade-off between
the two goals, which may be contrasting Often, the various ways to
approach a goal can be quantified by a utility function Do the
actions that approach the goal with the maximal utility Mixing
reactive and goal-oriented behavior A long terms goal that include
several short term actions on the environment That could lead to
sub-optimal choices 7/25/2011Fuhua Lin, SCIS, FST, Athabasca
University25 Slide 26 A A BDI agent model 7/25/2011Fuhua Lin, SCIS,
FST, Athabasca University26 Beliefs Intentions Plan library Desires
Interpreter Sensor input Action output Agent BDI agents are most
suitable to implement intelligent behavior in games The use of
goal-oriented action planning in gaming They make explicit use of
goals and planning They incorporate mechanisms to effectively use
communication and other interaction mechanisms in their action
deliberation Emotions EBDI Model Emotion reasoning --- one of the
common sense reasoning! Slide 27 OCC Emotion Model The theory of
human emotions and emotional reactions to events proposed by A.
Ortony, G. L. Clore, and A. Collins. (1988) (OCC model) Emotion
classes, each consisting of several emotions or emotional
reactions, for each emotion, eliciting conditions determine under
what circumstance the emotion is elicited Well-being
Fortunes-of-others Prospect-based Attribute Compound Attraction
7/25/201127Fuhua Lin, SCIS, FST, Athabasca University Slide 28
Formalization of Part of OCC model (Mueller, 2006) Agent sort: a,
a1, a2, Belief sort: a, b1, b2, Event sort: e, e1, e2, , and Not(e)
Fluent sort: f, f1, f2, Object sort: o, o1, o2, Real number sort:
x, x1, x2, 7/25/201128Fuhua Lin, SCIS, FST, Athabasca University
Slide 29 Fluent --- represent facts (factors) used to specify the
electing conditions of emotions Believe(a, e) --- agent a believes
that b has occurred Believe(a, Not(e) ---- agent a believes that e
has not occurred Desirability(a 1, a 2, e, x): agent a1 believes
that the desirability of event e to agent a2 is x, where -1 x 1. a
1 and a 2 may be the same. Praiseworthiness(a 1, a 2, e, x): agent
a1 believes that the praiseworthiness of event e performed by agent
a2 is x, where -1 x 1. a 1 and a 2 may be the same. Anticipate(a,
e, x): agent a anticipates that event e will occur with likelihood
x, where 0 x 1 7/25/201129Fuhua Lin, SCIS, FST, Athabasca
University Slide 30 Emotion expression functions Joy(a, e): agent a
is joyful about event e Distress(a, e): agent a is distressed about
event e. Happyfor(a1, a2, e): agent a1 is happy for agent a2
regarding event e SorryFor(a1, a2, e): agent a1 is sorry for agent
a2 regarding event e Resentment(a1, a2, e): agent a1 is resentful
of agent a2 regarding event e Gloating(a1, a2, e): agent a1 gloats
toward agent a2 regarding event e. 7/25/201130Fuhua Lin, SCIS, FST,
Athabasca University Slide 31 Learning What to learn Body language
(emotions) Communication patterns Domain knowledge (quiz bank,
determine the degree of difficulty, students levels, Game play
knowledge How to learn Centralized learning Distributed learning
Case-based learning Student agents --- how to build student models,
learn from human users, pedagogical agents, and other agents,
Pedagogical agents --- how to group students, how to generate
quizzes, how to provide hints, how to score NPCs --- how to learn
from humans and their agents? 7/25/201131Fuhua Lin, SCIS, FST,
Athabasca University Slide 32 Pedagogical Agents Algorithms How to
determine a student 's level and to assign a correct game room;
Decide what kinds of peer virtual students will be best for this
student, assuming we have a repository of virtual agents (NPCs)
available; During the game-play, quiz generation and sequencing,
given a group of real players and virtual players. 7/25/201132Fuhua
Lin, SCIS, FST, Athabasca University Slide 33 Question Item
Metadata 7/25/201133Fuhua Lin, SCIS, FST, Athabasca University
Slide 34 Simulating Human Communicative Strategies Simulating Human
communicative strategies Compose explanations spoken or textual;
Deliver encouragement, critiques and maintain a mixed initiative
dialogue; Analyze a student explanation, spoken or textual;
Question students approach Recognize students affect (emotion,
focus of attention, or motivation) Engage students in role playing;
hire partners for training interactive skills. (Woolf, 2009)
7/25/201134Fuhua Lin, SCIS, FST, Athabasca University Slide 35 Our
Publications S. Leung, S. Virwaney, F. Lin (2011, Submitted),
TSI-enhanced Pedagogical Agents, TESL 2011, Dalian, China. Martin
Weng, Fuhua Lin, Timothy K. Shih, Maiga Chang, Ireti Fakinlede, A
Conceptual Design of Multi-Agent based Personalized Quiz Game, The
11th IEEE International Conference on Advanced Learning
Technologies (ICALT 2011) July 6-8, 2011, Athens, Georgia, USA
(accepted) Ning Xia, Fuhua Lin, Aishuang Li, MODELING AND
VISUALIZATION OF FRUIT TREES IN HORTICULTURE, in a book "Computers
and Education" edited by Sergei Abramovich Blair, Jeanne & F.
Lin (2011). An Approach for Integrating 3D Virtual Worlds with
Multiagent Systems, ISeRim workshop - IANA 2011 (March, 2011;
Singapore) Armstrong, AJ & F. Lin, (2010) Modelling and
Personalizing Curriculum with Colored Petri Nets, ICCE 2010 (WIPP).
F Lin, Kinshuk, & M Dutchuk, (2009). Multiagent architecture
for incorporating adaptivity feature into 3D learning environments,
The 6th International Workshop on Mobile and Ubiquitous Learning
Environments (MULE 2009), Sept 8-12, 2009, Athabasca University,
Canada, pp 33-35, Mark Dutchuk, Khalid Aziz Muhammadi, Fuhua Lin
(2009), QuizMASter - A Multi- Agent Game-Style Learning Activity,
EduTainment 2009, Aug 2009, Banff, Canada, Learning by Doing,
(eds.), M Chang, R. Kuo, Kinshuk, G-D Chen, M. Hirose, LNCS 5670,
263-272. 7/25/201135Fuhua Lin, SCIS, FST, Athabasca University
