Upload
georg-oettl
View
199
Download
0
Tags:
Embed Size (px)
Citation preview
Knowledge Management Institute
1
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Non-invasive data tracking in educational
games
Combination of Logfiles and Natural
Language Processing
Stephanie B. Linek, Georg Öttl, Dietrich Albert
Knowledge Management Institute
2
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Outline
• Introduction
• Core factors of enjoyment in educational games
• Available data sources
• Open framework for a combined assessment
methodology
Knowledge Management Institute
3
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Introduction
• Genre: adaptive educational games• Transformative, Adaptive, Responsive and enGaging EnvironmenT
• Joy as essential factor in game based learning• Do not destroy the overall game-play experience
• Address player’s needs
• Non-invasive user monitoring and data-tracking
Knowledge Management Institute
4
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Core factors for enjoyment in games and
educational games
• Address the player’s need for reasonable parasocial
interaction
• Address the player’s individual preferences and
actual mood
• Address players capabilities
Knowledge Management Institute
5
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Reasonable parasocial interaction
• Communication preferences are reflected by the
selected game genres
• Provide options to address different communication
preferences for a player
• Adaption to the communication preferences of the
player
Knowledge Management Institute
6
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Individual preferences and actual mood
• Mood management• Media experiences are used for regulation of one’s own mood [1][2]
• Mood monitoring• Communication preferences provide evidence about mood
• Identification of player specific mood preferences
[1] Zillmann, D. (1988). Mood management through communication choices. American Behavioral Scientist, 31, 327-340.
[2] Zillmann, D. (2006). Dramaturgy for emotions from fictional narration. In J. Bryant & P. Vorderer (Eds.), Psychology of
entertainment (215-238). Mahwah, New Jersey: Lawrence Erlbaum Associates.
Knowledge Management Institute
7
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Players capabilities
• Adapt to the players skills
• Adapt to the players progress during time
• Ongoing monitoring of the skills and cognitive
capabilities
Knowledge Management Institute
8
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Data tracking in educational games
• Questionnaires• Evaluation of the overall quality of the game
• Presented before and after the game
• Online behavior monitoring without interrupting the
ongoing learning process• Logfiles
• Natural language
Knowledge Management Institute
9
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Data tracking by logfiles
• Automatic protocol of actions in a computer based
environment
• Huge amount of behavioral information
Knowledge Management Institute
10
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Defining associated logfile parameters
Logfile-Data / Probs (Potential) Meaning/interpretation
Mouse-clicking rate (number of clicks/time unit) Activity of the user: confusion/nervousness vs.
ambition
Frequency of tool-usage (of the different available
tools)
Activity and direction/purpose of user-behavior
Frequency help-seeking behaviour Confusion / overload of the learner
Sequencing of material (in relation to the task) Approximation to the solution vs. disorientation
Interaction frequency/rate with NPC or another
player
Affiliation/Intensity of parasocial interaction with an
NPC / another player: positive (sympathy) vs
negative (aversion)
Length of Interaction: Time spent on interaction with
an NPC or another player
Affiliation/Intensity of parasocial interaction with an
NPC / another player: positive (sympathy) vs
negative (aversion)
Eye-contact between player’ avatar and the
NPC/another player’s avatar
Affiliation/Intensity of parasocial interaction with an
NPC / another player: positive (sympathy) vs
negative (aversion)
Number of characters (NPCs and/or other player’s
avatars) the player is interacting with
Integration in the (virtual) community/group
(Integration-Index)
Knowledge Management Institute
11
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Pros and Cons of Logfiles
Pros
•Objective measurement of
user behavior
•Does not disrupt gameplay
•Non reactive method
without demand effects
Cons
•Too much data /
information overload
•Ambiguous data without
subjective meaning – hard
to interpret
Knowledge Management Institute
12
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Data tracking in Natural Language
• Natural language used in • Team speak
• Multiparty chat conversations
• Private 1:1 chats
• Connection between the words an individual uses and mind
[3][4] [5]• Attempt to algorithmically understand the subjective meaning of text with
Natural Language Processing (NLP)
[3] Jackendoff, R. S. (1985). Semantics and Cognition (Current Studies in Linguistics) (p. 304). The MIT
Press.
[4] Noam Chomsky. (1969). Aspects of the theory of syntax. MIT Press.
[5] Schank, R. C. (1972). Conceptual Dependency: A Theory of Natural Language Understanding. Cognitive
Psychology, 631(3), 552-631.
Knowledge Management Institute
13
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Defining associated NLP parameters
• Identify locations, persons, organizations, …
• Identify the topic
• Identify fascinating, acceptable and unsuitable answers [6]
• Interpret the semantic orientation [7]
• Identify gender, deception status and culture [8]
[6] Huang, J., Zhou, M., & Yang, D. (2007). Extracting chatbot knowledge from online discussion forums. In
Proc. of IJCAI (pp. 423-428).
[7] Hatzivassiloglou, V., & McKeown, K. R. (1997). Predicting the semantic orientation of adjectives.
Proceedings of the 35th annual meeting on Association for Computational Linguistics - (pp. 174-181).
Morristown, NJ, USA: Association for Computational Linguistics.
[8] Chung, C. K., & Pennebaker, J. W. (2007). The psychological functions of function words. Social
communication: Frontiers of social psychology, 343-359.
Knowledge Management Institute
14
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Pros and Cons of NLP
Pros
•Works autonomous
•Provides different
qualitative information than
logfiles
Cons
•Error prone
•Ambiguous outcomes
•Mostly relies on supervised
machine learning
Knowledge Management Institute
15
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Combination of logfile analysis and NLP
• Logfiles• Quantitative objective data regarding the activity of the user
• NLP• Qualitative information
Knowledge Management Institute
16
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Multimodal classification using logfile and NLP
data
• Analyse common properties of both data sources
• Augment necessary additional information
• Combination of both• Multimodal classification task
• Indicator selection based on empirical psychological studies
Knowledge Management Institute
17
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Open framework for a combined assessment
methodology
Defining associations between NLP
generated information and potential
subjective meaning
Pool of interesting psychological variables
subjective meaning
Behavioural logfile
parameterNLP indicators
Defining rules for multimodal
classification: Analysis of common
properties and augmenting information
Defining associations between logfile
parameters and potential subjective
meaning
Theoretical and empirical research
as basis for the definition of appropriate indicators, variables, associations and rules
Knowledge Management Institute
18
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Conclusion
• Multimodal classification of logfiles and NLP will lead
to a more holistic view of the user/player
• Future research on games and game-based learning
might lead to new insight on the association of user-
behaviour and subjective meaning
• A open framework in the sense that other additional
data sources could be incorporated and connected
Knowledge Management Institute
19
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
OUTLOOK
TARGET : www.reachyourtarget.org
Knowledge Management Institute
20
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Thank You For Your Attention!
Questions?
Stephanie Linek: [email protected]
Georg Öttl: [email protected]
Knowledge Management Institute
21
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
References
[1] Zillmann, D. (1988). Mood management through communication choices. American Behavioral Scientist, 31, 327-340.
[2] Zillmann, D. (2006). Dramaturgy for emotions from fictional narration. In J. Bryant & P. Vorderer (Eds.), Psychology of entertainment (215-238). Mahwah, New Jersey: Lawrence Erlbaum Associates.
[3] Jackendoff, R. S. (1985). Semantics and Cognition (Current Studies in Linguistics) (p. 304). The MIT Press.
[4] Noam Chomsky. (1969). Aspects of the theory of syntax. MIT Press.
[5] Schank, R. C. (1972). Conceptual Dependency: A Theory of Natural Language Understanding. Cognitive Psychology, 631(3), 552-631.
[6] Huang, J., Zhou, M., & Yang, D. (2007). Extracting chatbot knowledge from online discussion forums. In Proc. of IJCAI (pp. 423-428).
[7] Hatzivassiloglou, V., & McKeown, K. R. (1997). Predicting the semantic orientation of adjectives. Proceedings of the 35th annual meeting on Association for Computational Linguistics - (pp. 174-181). Morristown, NJ, USA: Association for Computational Linguistics. doi: 10.3115/976909.979640.
[8] Chung, C. K., & Pennebaker, J. W. (2007). The psychological functions of function words. Social communication: Frontiers of social psychology, 343-359.
Knowledge Management Institute
22
Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
Defining the interesting psychological
variables
• Subjective psychological indicators • Related to meaningful parasocial interaction
• Related to mood management
• Related to the adaptivity on the learner characteristics
• General Psychological variables• Curiousness of the player
• Confusion
• Positive versus negative mood
• Cognitive resources
• Objective behavioural correlates have to be found