www.maseltov.e

u

Designing and evaluating incidental learning

Andrew Brasher, Ann Jones, Agnes Kukulska-Hulme, Mark Gaved, Eileen Scanlon, Lucy Norris

IET, The Open University

www.maseltov.e

uMaseltov project

“Mobile Assistance for Social Inclusion and Empowerment of Immigrants with Persuasive Learning Technologies and Social Network Services” www.maseltov.eu

Joanneum Research, AustriaCURE - Centre for Usability Research & Engineering, AustriaUniversity of Applied Sciences, FH Joanneum, AustriaAthens Information Technology, Greecebusuu.com -, Spain

Universitat Oberta de Catalunya, SpainOpen University, UKCoventry University, UK Czech Technical University, Czech RepublicTelecom Italia SpA, ItalyFluidtime Data Service GmbH, AustriaFundacian Desarrollo Sostenido, SpainVerein Danaida, AustriaMigrants Resource Centre, UKPearson Publishing, UK

www.maseltov.e

uContents

• Overview of the Incidental Learning Framework• Evaluation & use

– Workshop at plenary meeting– expert researchers– Iliinsky’s work

• Challenges–Use in evaluation: analysis of MK trial data

• Conclusions

www.maseltov.e

uResearch questions

Research question driving the development of the Incidental Learning Framework

www.maseltov.e

uGoals of the Incidental Learning Framework

• to facilitate the creation of technology rich learning opportunities which emanate from incidental learning i.e. learning that is spontaneous and unplanned;

• to encourage links and triggers to structured and reflective learning to back up and deepen learning that happens incidentally;

• to enhance analysis and evaluation of incidental learning;

• to support discussions around software system design;

• to facilitate the communication of learning design ideas both visually and textually.

www.maseltov.e

uReview of frameworks

• Design frameworks– to support the planning and instantiation of new

learning events and situations• Analytical frameworks

– for analysing, understanding and evaluating learning events

• Learner models – to describe the state of the learner so that a

system can react appropriately

www.maseltov.e

uReview of frameworks

• Design frameworks• Analytical frameworks• Learner models

Design framework

Analytical framework

www.maseltov.e

uExamples of frameworks

Analytical• Theory of Learning for

the Mobile Age (Sharples et al., 2007)

• A Pedagogical Framework for Mobile Learning (Park, 2011)

Design• Ecology of Resources

design framework (Luckin, 2010)

• Four dimensional framework (de Freitas et al., 2010)

• Language learning defined by time and place(Kukulska-Hulme, 2012)

Learner model• Advances in learner &

skill modeling in intelligent learning environments (Desmarais and Baker, 2011)

www.maseltov.e

uKolb’s learning cycle

www.maseltov.e

uILF version 1

 

Outcomes

Social

Time

Tasks

Tools

Place

Learner’s journey

Tim

e

Incident: understanding notice board

Language tool

Start

TextLens

Task: finding station

Task: asking directions

Reflect/plan

Structured learning

Language Tool/Social

Task: navigating

Navigation tool

Language Tool

Language Tool/Social

www.maseltov.e

uExample, stage 1

www.maseltov.e

uExample, stage 2

www.maseltov.e

uEvaluation

• Workshop: adding detail to scenarios • Expert focus group

www.maseltov.e

uEvaluation

The framework • “generates helpful discussions about the learner

journeys which immigrants might make”• Exposes “misunderstandings and different

interpretations of the same problem space through different domain experts views”

• The current representation was sufficient to engage participants and encourage discussion.

(Gaved et al., 2013)

www.maseltov.e

uEvaluation

• Ordering of dimensions was questioned• Outer dimension of “learner’s journey” was

debated: - too general? - how to show e.g. social inclusion in to a new community

(Gaved et al., 2013)

www.maseltov.e

uExtension ideas (1)

Visualisation of specific learner journeys, showing opportunities for planned and incidental language learning

Gaved et al. (2012)

www.maseltov.e

uExtension ideas (2)

Visualisation of the range of affordances that may be associated with particular places that a MASELTOV service user is likely to encounter during their daily activities

www.maseltov.e

uExtension ideas (3)

Visualisation of the range of dimensions associated with motivation and concerns that might be felt by the service user while engaging with mobile situated language learning

www.maseltov.e

uChoosing visual properties

Iliinsky, N. (2013)

www.maseltov.e

uPlace

• Not just a location - also contextual information. • ‘A Place is a geographical and historical context for

Names and Locations’ (Gillies, 2011).• a ‘Place’ has a name and a location, and each may

be applicable for a particular time period. “Andrew’s workplace” currently has - Name: “Jennie Lee

Building” - Location: map

reference SP 88626 37058

www.maseltov.e

uLearning outcomes

Categorical Related

www.maseltov.e

uLinear view

Place

Task

Tools

Outcomes

Social

Time Evening

Language learning

Practise work related vocabulary

Understand work related vocabulary

Sleep/Break19.00 23.00 07.00

Language learning

Practise past tense

Communicate successfully in work related situation

Breakfast08.00

www.maseltov.e

uData gathering

• Pre trial questionnaire:paper

• Post trial interviews: face-to-face

E.g. Lesson access duration: 2’39s

Synchronous & located with MApp use

• Event data from the MApp

• Usage of other apps

• Geo contextual data

Asynchronous & not located

with MApp use Weekly survey: online

www.maseltov.e

uParticipant’s use of Mapp services

I have really liked the English course, it is very practical for the times when you have to use English for different occasions..it has helped me a lot The second is the translation tool because now I take a photo of everything I don't understand and at that moment I know what it says ... It has helped me a lot.

I have learned a lot more English language

I have learned to develop myself in different aspects of speaking English with the help of the course

www.maseltov.e

uUse in analysis

• Example language lesson data key value timestamplesson transport 22/01/2015 13:22publication mas_trav 22/01/2015 13:22score 90.90908813 22/01/2015 13:22lesson plan_journey 24/01/2015 18:41publication mas_trav 24/01/2015 18:41score 0 24/01/2015 18:41lesson plan_journey 24/01/2015 18:41publication mas_trav 24/01/2015 18:41score 0 24/01/2015 18:41lesson transport_chaos 24/01/2015 20:07publication mas_trav 24/01/2015 20:07score 100 24/01/2015 20:07lesson body_and_health 24/01/2015 20:59publication mas_health 24/01/2015 20:59score 85.7142868 24/01/2015 20:59

www.maseltov.e

uConclusions

• Development of ILF as a design tool –Work to do is guided by literature and

experimental findings• Development of ILF as an analytical tool

–Data available complex and multi-faceted;– Initial experiments started; –Transfer of design view not straightforward?–Learner models?;–Suggestions welcome!

Levels of abstraction?

Level of detail?

www.maseltov.e

uTo do

• Include stuff from conclusions slide from 2012 Calrg presentation

• …feedback from eval D7.12.• Other stuff….

• Issues – learning outcomes can be shown at gps locations, but need to show learning journey

• To point to new developments – show new version

www.maseltov.e

uExtensions

• D7.1.2 i.e.- SCAMP- Learning journey

www.maseltov.e

uReferences

• de Freitas, S., Rebolledo-Mendez, G., Liarokapis, F., Magoulas, G. & Poulovassilis, A. 2010. Learning as immersive experiences: Using the four-dimensional framework for designing and evaluating immersive learning experiences in a virtual world. British Journal of Educational Technology, 41, 69-85.

• Desmarais, M. C. & Baker, R. S. J. D. 2011. A review of recent advances in learner and skill modeling in intelligent learning environments. User Modeling and User-Adapted Interaction, 22, 9-38.

• Kukulska-Hulme, A. 2012. Language learning defined by time and place: A framework for next generation designs. In: D´ıAZ-VERA, J. E. (ed.) Left to My Own Devices: Learner Autonomy and Mobile Assisted Language Learning.

• Iliinsky, N. (2013). Choosing visual properties for successful visualizations. Retrieved 24/4/2014, from http://bit.ly/successfulvis

• Luckin, R. 2010. Re-Designing Learning Contexts: Technology-Rich, Learner-Centred Ecologies (Foundations and Futures of Education). 208.

• Park, Y. 2011. A Pedagogical Framework for Mobile Learning: Categorizing Educational Applications of Mobile Technologies into Four Types. International Review of Research in Open and Distance Learning, 12, 78-102

• Sharples, M., Taylor, J. & Vavoula, G. 2007. A Theory of Learning for the Mobile Age. In: Andrew, R. & Haythornwaite, C. (eds.) The Sage Handbook of Elearning Research. London: Sage.