A Standard forAugmented RealityLearning Experience Models(AR-LEM)

Fridolin Wild1), Christine Perey2)1) The Open University, UK 2) Perey Research and Consulting, CH

Call for Potentially Essential Patents

If anyone in this meeting is personally aware of the holder of any patent claims that are potentially essential to implementation of the proposed standard(s) under consideration by this group and that are not already the subject of an Accepted Letter of Assurance: Either speak up now or

Provide the chair of this group with the identity of the holder(s) of any and all such claims as soon as possible or

Cause an LOA to be submitted

Overview and discussionxAPI integration

3

Fridolin Wild1), Paul Lefrere1), Maurizio Megliola2), Gianlugi De Vito2), Roberto Sanguini3) 1) The Open University, UK 2) Piksel, Italy, 3) AgustaWestland, Italy

“control of one's bodily motions,

the capacity to handle objects skillfully, a sense of

timing, a clear sense of the goal of a physical

action, along with the ability to train responses”

-- Gardner, 2011

Bodily-Kinaesthetic Intelligence

4

• World of Repetition (that enabled long cycles of training to be cost-justified) … is gone!

• Short-run or personalised production (with advanced machinery and lower demand in physical dexterity) … is emerging!

• Learning Design Languages (Laurillard & Ljubojevic, 2011; Fuente Valentin, Pardo, & Degado Kloos, 2011; Mueller, Zimmermann, & Peters, 2010; Koper & Tattersall, 2005; Wild, Moedritscher, & Sigurdarson, 2008)

enable modelling of experiences, but fall short of Capturing and codification– Handling hybrid (human-machine) experiences

Context: Professional training

5

BLUE COLLAR WORKER EXPERIENCE

Enquire

Mix

Match

Optimise

?

Traces

Need, Problem

ActivityXML incl.

Constraints

Report,Analytics

Suggestion, Recommendation

Classifiable? Known?

Unknown?

• Navigational positioning in

taxonomy• Discovery

support

• Selection of existing mixes with ranked

search• Authoring of new or

modified mixes

• Personalised suggestions for

improvement of mix / experience

tracking

Constraints:• e.g. returned tool 15• e.g. watched video A

• e.g. 14/15 in PT session• e.g. 0 FOD problems

• e.g. energy < daily limit

activities:• e.g. job cards

• e.g. tasks• e.g. learning paths

• Queries• (Reasoning)

Learning by Experience

(Wild et al., 2014; Wild et al., 2013)

The eXperience API

7

curl -X POST --data @example.json

-H "Content-Type: application/json"

-- user 465ea716cebb2476fa0d8eca90c3d4f594e64b51

http://www…

API call

8

Example statement

9

Example trace (Helicopter Industry)

10

user cleaned

‘corrosion inhibitors’

Open Source LRS: LearningLocker

11

Analytics: cRunch

12

plot(net, usearrows = TRUE, usecurve = T)

“verbs that refer to physical actions are naturally grounded in

representations that encode the temporal flow of events“

-- Roy, 2005:391

Verbs of handling and motion

13

• Specify force dynamics (out of a limited set)• Specify temporal Allen relations

(A ends after B, A starts with B, …)• Specify valency (number of arguments, cf. Palmer et al., 2005)• Primitives are ‘movement’, ‘path’, and ‘location’ (Chatterjee,

2001)

Any higher-level composition can be traced back to these atomic relations

Verbs of handling and motion

14

General taxonomy across applications (Robertson and MacIntyre, 2009)

– ‘Style’ strategies: ‘include’, ‘visible’, ‘find’, ‘label’, ‘recognizable’, ‘focus’, ‘subdue’, ‘visual property’, ‘ghost’, and ‘highlight’ (p.149f)

– Communicative goals signified by these styling operations for visual overlays are: ‘show’, ‘property’, ‘state’, ‘location’, ‘reference’, ‘change’, ‘relative-location’, ‘identify’, ‘action’, ‘move’, and ‘enhancement’ (p.148)

Need to be complemented with workplace specific taxonomy (of handling and motion)

Communicative intent

15

xAPI verb statements Helicopter Industry

16

• Application-driven versus tracking-driven drop statements

• Behaviour validation using automated tracking:– State-based:

Removed cap: cap was there, cap not there– Location-based:

Removed screw: returned screw to used parts bucket– Dependancy-based:

Removed screw: picked up replacement screw• Related industries (Furniture + Textiles):

common core?

Challenges

17

Ralf Klamma, RWTH Aachen, GermanyRequirements Bazaar

18

Lehrstuhl Informatik 5(Information

Systems)Prof. Dr. M. Jarke

19

LearningLayers Requirements Bazaar – Involving End

Users in Requirements Engineering End User Involvement

– Open Innovation [Chesbrough, 2003]

– ideas from the long tail [Anderson, 2006]

– emerging from practices, needs adoption [Denning, 2004]

Requirements Bazaar [Klamma et al., 2011], [Renzel et al., 2013]

– social continuous innovation platform– listening to the long tail communities– mobile first Web application

Lehrstuhl Informatik 5(Information

Systems)Prof. Dr. M. Jarke

20

LearningLayers

https://requirements-bazaar.org

Lehrstuhl Informatik 5(Information

Systems)Prof. Dr. M. Jarke

21

LearningLayers

DevOps Life Cycle Rapid release cycles Strong feedback loop

Continuous integration Continuous delivery Continuous deployment

Containerized microservices

Lehrstuhl Informatik 5(Information

Systems)Prof. Dr. M. Jarke

22

LearningLayers DevOpsUse Life Cycle

for Continuous InnovationInvolving end users in the designand development process

ideas and needs co-design beta testing context adaptation awareness

Lehrstuhl Informatik 5(Information

Systems)Prof. Dr. M. Jarke

23

LearningLayers Requirements Bazaar 2.0 for

Continuous Innovation Redeveloped version launched in April 2015

– Increased maintainability– Based on componentized architecture– Mobile first responsive Web design

Incorporating DevOpsUse practices

Requirements Bazaar Kanban Board Responsive Design

Open Problems

24

Task Force and Open Problems

Task Forces identified:– Use Cases, Storyboards, and Requirements– Test Battery (Authoring of ARLEMs)– Reference Implementations– Validator Service– Pilots: Running Code

Open Problems– Real-time messaging (multiuser, multi-device, smart objects)– Revision needed: xAPI auto-logging– query language for constraint validation– Performance analytics– LEM aggregator (‘Open LEM’)

The END