The Technology Enhanced Learning Living Lab for Manufacturing Environments.

Fridolin Wild1), Peter Scott1), Jaakko Karjalainen2), Kaj Helin2), Erik Isaksson3), Ambjoern Naeve3), Maurizio Megliola4), Gianluigi Di Vito4)

1) The Open University, UK 2) VTT, Finland, 3) Royal Institute of Technology (KTH), Sweden, 4) Piksel, Italy

ARgh! - kinesthetic learning.Augmented World Expo, 27.-29.5., Santa Clara/CA

2

Agenda

• Context ‘Manufacturing’• Process Model (‘Mix, Match, Optimize’)• Activity Model• Workplace Model• xAPI Tracking • (Analytics with cRunch)

Context ‘Manufacturing’

Workforce in EU: 225.6 M total employmentManufacturing EU: > 28.4 M1) employees

> 12.6 %1)

of which in SMEs: 59.9 %2)

Source: Eurostat, NACE R2 (2009), LFSI EMP A (2009)

1) Based on incomplete data, actual numbers higher: e.g. France missing.2) Based on incomplete data.

3

4

Workplace != Workplace

(pictures: d7.1, study: d3.3)

5

Job Performance Aids

- Independent, self-regulated learners- As individuals or in collaboration- Engage in problem solving and devise

new applications (‘prototyping’)

- Extreme levels of memorization training and rehearsal (‘training to memory’)

- Prepared to perform in challenging circumstances

- Scaffold people in carrying out an unfamiliar task Adaptivity

Adaptability

consumingdo-torials

creating do-torials

BLUE COLLAR WORKER EXPERIENCE

Enquire

Mix

Match

Optimise

?

Traces

Need, Problem

ActivityXML incl.

Constraints

Report, Analytics

Suggestion, Recommendation

Classifiable? Known? Unknown?

• Navigational positioning in taxonomy

• Discovery support

• Selection of existing mixes with ranked search

• Authoring of new or modified mixes

• Personalised suggestions for improvement of mix / experience

tracking

Constraints:• e.g. returned tool 15• e.g. watched video A• e.g. 14/15 in PT session• e.g. 0 FOD problems• e.g. energy < daily limit

activities:• e.g. job cards• e.g. tasks• e.g. learning paths

• Queries• (Reasoning)

Learning by Experience

(Wild et al., to appear; Wild et al., 2013)

BLUE COLLAR WORKER EXPERIENCE

Enquire

Mix

Match

Optimise

?

Traces

Need, Problem

ActivityXML incl.

Constraints

Report, Analytics

Suggestion, Recommendation

Classifiable? Known? Unknown?

• Navigational positioning in taxonomy

• Discovery support

• Selection of existing mixes with ranked search

• Authoring of new or modified mixes

• Personalised suggestions for improvement of mix / experience

tracking

Constraints:• e.g. returned tool 15• e.g. watched video A• e.g. 14/15 in PT session• e.g. 0 FOD problems• e.g. energy < daily limit

activities:• e.g. job cards• e.g. tasks• e.g. learning paths

• Queries• (Reasoning)

Learning by Experience

(Wild et al., to appear; Wild et al., 2013)

Sourcing: help to find or devise next-generation training solutions

for blue collar workers in manufacturing

Analytics: allow comparative analysis on such measures as time-to-

competence, long-term recall, maximum productivity achievable, time

to spot and remedy mistakes, …

Transformate: help reduce or eliminate effort needed to upgrade training solutions

8

Scenarios: AR goes HTML5!

explicit tacit

Problem

Summer /Winter cycle

Functional fabrics

Set up of the weaving mill.Each topic may hav.)

Prevent potential defects before they happen.Failed step: waste of

resources.

SB: Improve manufacturing process with attribute changes

The situation: as-is

PROFESSIONAL QUALIFICATION(e.g. courses, titles)

MATERIALS (e.g. yarn 76/2 710)

MACHINE

to-be: do-torial (1)

Each topic may have several alternative learning contents attached (objects, actions, choices...)

DornierMTV38

Yarn76/2 710

Machine parts Materials

OCCUPATIONAL HEALTH AND SAFETY

3D MODEL

Schlagbaum

SET UP PARAMETERS

12

ARgh! Component Architecture

13

ARgh! UI Model

TELL ME project3 May 2023

‘Traditional’ AR layero markerso 3D objectso Tied to object

PAINT GUN USE

PROFESSIONAL QUALIFICATION(e.g. courses, titles)

FURNITURE MODEL, 3D MODEL

MATERIALS (e.g. Pinewood)

do-torial

SOLVENTS, CATALYSTS & VARNISHES

PAINTS

Each topic may have several alternative learning contents attached (objects, actions, choices...)

Tinting and Varnishing (all spraying benefit from same technology)

OCCUPATIONAL HEALTH AND SAFETY

LOCATION

15

Do-torial demo: Varnishing for tropical climate

Interoperability

is a property that emerges, when distinctive information systems (subsystems)cooperatively exchange data

in such a way that they facilitate the successful accomplishment of an overarching task.

17

The Activity Model

18

The Activity Model

“find the spray gun nozzle size 13”

Messaging in the real-time presence channel and tracking to xAPI

Chaining of actions; modeling of ‘reactions’: onEnter / onTrigger

Styling (cascading) of viewports and UI elements

Constraint modeling:specify validation conditions and model workflow branching

e.g. smart player;e.g. request widget

19

Serialized Activity Model<activity id="fodw109” name="do-torial: learn about …” start="SprayGun”> <action id="SprayGun” viewport="actions" type="findtool” location="SprayBooth"> <message type=“trace" channel=“lrs">user %u% searches for tool spraygun</message> <summons type="onClick" removeSelf="true"> <toggle viewport="actions" type="action">video15</toggle> <toggle viewport="objects" type="object">SprayGun</toggle> <message type=“exit" channel="rpc">user %u% interacted spraygun</message> </summons> <constraint channel="rpc">user %u% interacted spraygun</constraint> <instruction><![CDATA[<h1>Find the Spray Gun</h1><p>bla…</p>]]></instruction> </action>

<action id="video15" viewport="actions” type="widget” location="SprayBooth"> <widget type="smartplayer" id="15” /> <message type="invoke" channel="rpc">user %u% accesses video15</message> <summons type="onClick" removeSelf="true"> <toggle viewport="actions" type="action">microass1</toggle> <message type="exit" channel="rpc">user %u% watched video 15</message> </summons> <constraint channel="rpc">user %u% watched video15</constraint> <instruction><![CDATA[<h1>Well done!</h1><p>…!</p> ]]></instruction> </action></activity>

20

The Workplace Model

21

The Workplace Model

The ‘tangibles’:Specific persons, places, things

The ‘configurables’:devices (styling),apps+widgets

The ‘triggers’:Markers trigger Overlays; Overlays trigger human action

22

Unified Reference Space: Workplace<?xml version="1.0" encoding="utf-8"?><Workplace xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><Resources> <Tangibles>

<Place Name="SprayBooth" Category="Place" MarkerID="007"> <YesNoOverlay IsEnabled="true” X_Offset="0” Y_Offset="0” Z_Offset="0” /> <Labels IsEnabled="false">

<Label Name="Name“ X_Offset="0” Y_Offset="0” Z_Offset="0” /> </Labels> </Place>

<Thing Name="SprayGun" Category="Tool" MarkerID="011" Thumbnail=“pistol-blue-small.png"><YesNoOverlay IsEnabled="true” X_Offset="0” Y_Offset="0” Z_Offset="0” /><Labels IsEnabled="false">

<Label Name="Name“ X_Offset="0” Y_Offset="0” Z_Offset="0” /></Labels>

</Thing>

<Person Name="Jake" Category="Person" MarkerID="012"><YesNoOverlay IsEnabled="true” X_Offset="0” Y_Offset="0” Z_Offset="0” /><Labels IsEnabled="false">

<Label Name="Name“ X_Offset="0” Y_Offset="0” Z_Offset="0” /></Labels>

</Person> <Tangibles> </Resources></Workplace>

23

Tracking with the Experience API

24

Analytics

plot(net, usearrows = TRUE, usecurve = T)

Technology Enhanced Learning Living Lab for Manufacturing Environments

25