AMBILEARN

Ambient Intelligent Multimodal Learning Environment for Children

Confirmation VivaJune 2009

Jennifer Hyndman, B.Sc. (Hons.)

Supervisors: Dr. Tom Lunney, Prof. Paul Mc Kevitt

School of Computing and Intelligent systems

Faculty of Computing and Engineering

University of Ulster, Magee.

1

OVERVIEW Research Objectives Literature Review Research Motivation Project Proposal

Evaluation and testingSoftware Analysis & Prospective Tools

Relation to other work Potential Contributions Research Plan Thesis Outline Publications Conclusion

2

RESEARCH OBJECTIVES

Investigate potential of an educational game integrated with virtual learning environment Positive impact on children at a primary level Means to merge multiple modalities

Mobile Devices e.g. Smart Phones (Nokia) /Personal Digital Assistant (PDA)

Interface Design /Usability Role of serious games AmbiLearn / TreasureLearn Educational game

e.g. Treasure hunt / puzzels Test and evaluate AmbiLearn/TreasureLearn Demonstrate educational potential

3

LITERATURE REVIEW

Multimodal Interfaces Natural communication methods

Bolt’s Media room (Bolt, 1980) SmartKom (Wahlster et al. 2001)

Mobile interfaces Speech and pen Complementary modalities (Cohen, 1992)

Multimodal interfaces for Children Collaborative environments KidsRoom (Bobick et al. 1998) TICLE (Scarlatos, 2002)

4

Embodied Agents

Persona Effect (Lester et al. 1997)

Gandalf (Thorisson, 1997) Speech, gaze, facial expressions,

gestures and projected screen

Steve (Johnson et al. 1997) Speech, gaze, gesture,

demonstration

Hans Christian Anderson (Corradini et al. 2005) Speech, gesture

5

Learning Environments

Technology enhanced learning M-learning, e-learning, web-based learning

Games in education Pedagogical features Feedback/Motivation (Rosas et al. 2003)

Augmented reality Enhancing real world experiences ReadIt (Weevers et al. 2004)

Virtual reality Rich sensory experience Submerge into learning context (Crumpton et al. 1997)

6

Virtual learning environments Information – anytime/anywhere Unclear pedagogical effects (Dillenbourg, 2002) Monitor and assess

Human Computer Interaction Child computer interaction (ChiCi group, 2007)

Child centred design Interaction styles (Strommen et al. 1996) Dexterity/speech/reading/background/interaction

(Bruckman et al. 2007)

7

RESEARCH MOTIVATION

8

PROJECT PROPOSAL

Multimodal communication collaborative learning mobile handheld device

AmbiLearn architecture Multimodal mobile system Ubiquitous learning environment

TreasureLearn application Treasure hunt / puzzles Designed for children Collaborative learning / problem solving

9

ARCHITECTURE OF AMBILEARN

10

TREASURELEARN ACTIVITY

11

EVALUATION/TESTING OF AMBILEARN/TREASURELEARN

Pre-test / Post-test

Fun Toolkit (Read & MacFarlane, 2006)

12

SOFTWARE ANALYSIS & PROSPECTIVE TOOLS Spoken dialogue systems

WebSphere Voice Toolkit (IBM, 2009) Microsoft Speech (Microsoft, 2008)

Embodied Agents X-Face (Balci, 2004) COLLAGEN (Cassell, 2001)

Development kits Java ME – mobile handheld devices DevKit Pro – Nintendo homebrew Apple SDK – iPod/iTouch/iPhone 13

RELATION TO OTHER WORK

14

POTENTIAL CONTRIBUTIONS

Improve upon existing multimodal learning environments Supporting collaborative learning indirectly

through the use of game Provide user logging module – future analysis

Contribute to knowledge Multimodal mobile games in education Multimodal interfaces on mobile devices Collaborative learning with mobile devices Advance the area in terms of providing a flexible

and rich learning experience for children 15

RESEARCH PLAN ActivitiesSubmissionsDeliverables

16

THESIS OUTLINE

0. Abstract / Acknowledgements / DeclarationChapter 1. IntroductionChapter 2. Literature ReviewChapter 3. Ambient Intelligent Multimodal

Learning Environment for ChildrenChapter 4. AmbiLearn and TreasureLearn

ImplementationChapter 5. Evaluation of AmbiLearn and

TreasureLearnChapter 6. ConclusionAppendices / References 17

PUBLICATIONS Hyndman, J., Lunney, T. And Mc Kevitt, P. (2009) AmbiLearn:

Ambient Intelligent Multimodal Learning Environment for Children. In: Proceedings of the 10th Annual PostGraduate Symposium on The Convergence of Telecommunications, Networking & Broadcasting, PGNet, Liverpool, 22nd – 23rd June. Liverpool UK : Liverpool John Moores University. 277 -282.

18

CONCLUSION

19

Focus on multimodal learning environments Mobile Multimodal interfaces Serious games

Design and develop AmbiLearn architecture /TreasureLearn architecture

Demonstrate and test educational potential AmbiLearn and TreasureLearn Evaluate AmbiLearn/TreasureLearn

Improve upon existing multimodal learning environments Supporting collaborative learning indirectly through the

use of game Provide user logging module – future analysis

THANK YOU!

QUESTIONS?20

I never try to teach my students anything, I only try to create an environment in which they can

learn. – Albert Einstein