Cognitive Modelling Goes To SchoolMODELLING CONVERGENT AND DIVERGENT PROCESSES

IN SOCIAL EXPLORATORY SEARCH AND LEARNING

TOBIAS LEYTALLINN UNIVERSITY, ESTONIA

CELDA 2018KEYNOTE

21.10.2018

LEARNING ANALYTICS & EDUCATIONAL INNOVATIONTallinn University, Estonia

http://ceiter.tlu.ee

Research Group on

Horizon 2020

Widening

Kenneth R. Koedinger, John R. Anderson, William H. Hadley, Mary A. Mark. Intelligent Tutoring Goes

To School in the Big City. International Journal of Artificial Intelligence in Education (IJAIED), 1997,

8, pp.30-43.

“This study provides further

evidence that laboratory tutoring

systems can be scaled up and

made to work, both technically

and pedagogically, in real and

unforgiving settings like urban

high schools.”

Koedinger, et al. (1997), pp.30

ADRESSING NEW CHALLENGES- From Direct Instruction to

Exploratory Learning

- Learning goals dynamically change

in the learning process

- Learning is socially mediated

Learning in Realistic Contexts

S O C I A L LY- M E D I AT E D

S E L F - D I R E C T E D L E A R N I N G

Individual

Search and

Sensemaking

Collaborative

Knowledge

Building

Modelling Memory

Processes

Designing Support Mechanisms

(Intelligent Learning Systems)

Collaborative

Information

Search on the

Web

Discovery

Learning in

School

How working

memory and

long term

memory

interact to learn

categories?

Role of

Working

Memory

Capacity?

COLLABORATIVE KNOWLEDGE BUILDING & INDIVIDUAL LEARNING- Individual and collective

knowledge creation tightly

linked

- Negotiation Processes in Social

Software Environments(Cress, Kimmerle, et al. 2016; Jirschitzka et

al. 2017; Kump et al., 2013)

ToiletHow To

Lightning Solution

X3-PJC

X3-POZ

PLC Equipment

S O C I A L

B O O K M A R K I N G

E N V I R O N M E N T

Collaborative Information

Search in groups of university

students (N=24)

Ley, T., & Seitlinger, P. (2015). Dynamics of Human Categorization in a Collaborative Tagging

System: How Social Processes of Semantic Stabilization Shape Individual Sensemaking.

Computers in Human Behavior, 51, 140–151.

C O N V E R G E N C E

I N T H E G R O U P

S T R E N G T H E N S

I N D I V I D U A L

L E A R N I N G

Semantic Stabilization in the

Group leads to stronger

memory traces for the used

conceptsC

um

ula

tive

re

lative

fre

qu

en

cy

0.0

0.2

0.4

0.6

6 7 8 9 10

GeneralMediumSpecific

Low semantic stabilization

Weeks

Cu

mu

lative

re

lative

fre

qu

en

cy

0.0

0.2

0.4

0.6

6 7 8 9 10

GeneralMediumSpecific

High semantic stabilization

Weeks

Collaborative Knowledge

Building

Collaborative Information Search on

the Web

Role of Memory in Convergent and

Divergent Group Processes

Individual Search and

Sensemaking

Discovery Learning in Secondary

School

Role of Working Memory Capacity

COLLABORATIVE KNOWLEDGE BUILDING

How to reach consensus?

How to deal with divergence?

H O W D O E S

C O N S E N S U S

E M E R G E ?

Verbatim tag imitation:

Copying the word(e.g. Dellschaft & Staab, 2008; Halpin et al., 2007;

Rader & Wash, 2008)

Semantic imitation:

Tag-based Topic Inference(e.g. Fu et al., 2009, 2010)

Seitlinger, P., Ley, T., & Albert, D. (2015). Verbatim and semantic imitation in indexing

resources on the Web: A fuzzy-trace account of social tagging. Applied Cognitive

Psychology, 29(1), 32–48. https://doi.org/10.1002/acp.3067

A M O D E L O F S O C I A L T A G G I N G :

F U Z Z Y T R A C E T H E O R Y

Imitation of Tag t (time n+1)

explicit imitation p(t) = D implicit imitation p(t) = R*J

Encoding

verbatim trace (direct access D) gist-trace (reconstruction R)

Tag-based Search for Resources (at time n)

Perception of Tag t

J=Familiarity-based Judgement

(e.g. Brainerd et al., 2010)

BOOK

BOOK

“BOOK”

book

novelread

author

<book>

A N E X P E R I M E N T U S I N G T H E

RT T T- P R O C E D U R E (Brainerd et al., 2002)

N=39, age M=32

Phase R: Incidental Learning of tags

in a decision task

Phase T: Production of tags

(previously learned or not)

recall

processes

tagging

user

social

Imitation

search

semantic

M AT E R I A L U S E D :

P H O T O S A N D TA G S

M E A S U R E M E N T M O D E LMultinomial Processing Tree (adapted from Brainerd et al., 2010)

ImitationRecall state

Recall state

Recall state

Stimulus

Resource

Direct access

Verbatim

No direct access

Recon-struction

Choosing old tag

Semantic

Choosing new tag

Semantic

No recon-struction

Guessing old tag

Idiosyncratic tagging

Not guessing old tag

Idiosyncratic tagging

R E S E A R C H Q U E S T I O N S

- What role do verbatim and semantic processes play when

imitating tags?

- D: Direct access to varbatim trace

- R: Reconstruction of gist trace

- J: Familiarity-based decision

- Can these processes be dissociated using practically significant

variables?

- Tag size

- Semantic Layout of tag clouds (e.g. Lohmann et al., 2009; Rivadeneira et al., 2007;

Schrammel et al., 2009)

- Word connectivity: density of generated associations (Nelson et al., 1998)

PA R A M E T E R E S T I M AT E S

Verbatim and Gist traces (for explicit and semantic imitation) are

represented by independent parameters D and R

A good amount of tag-processing and production is implicit (R and J)

0

0,2

0,4

0,6

0,8

1

E R J G

0

0,2

0,4

0,6

0,8

1

E R J G

0

0,2

0,4

0,6

0,8

1

E R J G

Connectivity

Tag-Size

Semantic Layout

SEMANTIC & VERBATIM IMITATIONSemantic imitation constant at around 13%

Verbatim imitation depends on conditions

(20% vs. 8%)

Other influencing factors

Semantic Layout of Tags → R

Size of Tags → J

Connectivity of Tags → D

F R O M A M E A S U R E M E N T T O A

C O M P U TAT I O N A L M O D E L

3Layers - A tag recommeder

based on verbatim and semantic

processing

Based on ALCOVE – a

connectionist model of category

learning

Seitlinger, P., Kowald, D., Trattner, C., & Ley, T.

(2013). Recommending Tags with a Model of

Human Categorization. In Conference on

Information and Knowledge Management,

CIKM’13, Oct. 27–Nov. 1, 2013, San Francisco,

CA, USA. (pp. 2381–2386).

Encode the resource in terms of

topics or categories

Match the pattern to all previously

encountered examples

Activate the tags from those

examples

Draw tags depending on

their activation

C A N T H E A L G O R I T H M G U E S S

W H I C H T A G S P E O P L E W I L L U S E ?

- Using Wikipedia pages

tagged in Delicious

- Algorithm recommends

tags from previous tags

of that user

- Reinforces convergence

CONVERGENCE& DIVERGENCEIn Social Tagging Research

Convergence is usually highlighted

(consistent indexing of resources)

Divergent Search important for

effective problem solving

(Lazer & Bernstein, 2010)

D I V E R G E N T P R O C E S S E S I N

I N F O R M AT I O N S E A R C H

- Trade-off: Fluency (Flat Associative Hierarchy)

vs. Consistency (Steep Associative Hierarchy)

- Research Questions

- Evidence for the trade-off?

- Supporting information search with recommender?

- Study: Individual and collaborative information

search at the workplace (one month, N=18)

Seitlinger, P., Ley, T., Kowald, D., Theiler, D., Hasani-Mavriqi, I., Dennerlein, S., Albert,

D. (2018). Balancing the Fluency-Consistency Tradeoff in Collaborative Information

Search with a Recommender Approach. International Journal of Human–Computer

Interaction, 34(6), 557–575.

C O L L A B O R AT I V E I N F O R M AT I O N

S E A R C H M A K E S P E O P L E M O R E

C R E AT I V E …

Collaborative

search leads to

higher fluency of

associations …

… B U T L E A D S T O L O W E R

L E V E L S O F C O N S I S T E N C Y

… weaker

relationship between

topic and tags used

H O W T O S U P P O R T

C O N S I S T E N C Y D U R I N G

C R E AT I V E S E R C H ?

- Recommender 1: Most Popular (MPT)

- Recommender 2: Search of Memory (SoMe)

- Similar to the 3Layers Recommender

- Based on Episodic Memory Model MINERVA2

S o M e H A S H I G H E R

A C C E P TA N C E I N

C O L L A B O R AT I V E S E A R C H

CONVERGENCE& DIVERGENCECollaboration leads to higher levels of

creativity

But also to lower levels of consistency

Need a recommendation paradigm

based on semantic processing

Individual Search and

Sensemaking

Discovery Learning in Secondary

School

Role of Working Memory Capacity

SELF-DIRECTED SEARCH AND SENSEMAKING

D I S C O V E R Y L E A R N I N G I N

S C H O O L : S E A R C H A N D

S E N S E M A K I N G

Reflective Search Model

Information

Goal

Search

Find

Information

Sensemaking

Attention

Control

D I S C O V E R Y L E A R N I N G I N

S C H O O L : S E A R C H A N D

S E N S E M A K I N G

- 8th and 9th grade pupils (N = 109)

- Independent Variable

- Working Memory Capacity (WMC): high vs. low

- Dependent Variable

- Learning Curves

- Search and sensemaking

in DinoNimi

Seitlinger, P., Uus, Õ., & Ley, T. (2018). The role of working memory capacity in a self-regulated search

and sense-making task for high school biology. Computers in Human Behavior. Under Review.

… is due to better retrieval of previously learned

categories (sensemaking) …

… and more strategic search.

Group x Practice: F(6,318)

= 2.24, p < .05

Group: F(1,55) =

4.41, p < .05

𝛘(4) = 15.58, p < .01

Performance Difference between

high and low Working Memory

Capacity condition …

L E A R N I N G C AT E G O R I E S I N

S U S TA I N

Inhibition of interference

(Unsworth & Engle)

Empirical

Pattern

Simulated

Pattern

TAKE AWAYS

1.T H E S O C I A L

&

I N D I V I D U A L

- Coupling of social and individual learning

- Convergence in group important for individual learning

- Importance of social cues for creative thought

- Mapping of internal and external (e.g. fluency and tagging)

- Cultural production of patterns and individual learning

- Formal model of Sociocultural Learning Theory

Ley, T., Seitlinger, P., & Pata, K. (2016). Patterns of Meaning in a Cognitive Ecosystem: Modeling

Stabilization and Enculturation in Social Tagging Systems. In U. Cress, H. Jeong, & J. Moskaliuk (Eds.),

Mass Collaboration and Education (pp. 143–163). Heidelberg: Springer.

2.

- Coupling is mediated by our memory processes

- Working memory and long term memory as a

sensemaking machine

- Executive functions control attention and

shield search and retrieval

MEMORY

3.

- Cognitive Models mimic human exploratory learning

- … and allow diagnosis and support in open ended

learning tasks

- “Learning Analytics” made theory-driven rather than

data-driven

ASSI STI NG

LEARNI NG

REFERENCESDennerlein, S., Seitlinger, P., Lex, E., & Ley, T. (2016). Take up My Tags: Exploring Benefits of Meaning Making in a Collaborative Learning Task at the Workplace.

In K. Verbert, M. Sharples, & T. Klobučar (Eds.), Adaptive and Adaptable Learning: 11th European Conference on Technology Enhanced Learning (Vol. 9891, pp.

377–383). Heidelberg: Springer. https://doi.org/10.1007/978-3-319-45153-4_30

Dennerlein, S., Tomberg, V., Treasure-jones, T., Theiler, D., Lex, E., & Ley, T. (2018). The Role of Memory and Sensemaking in Healthcare Professionals’ Informal

Learning at Work: A Design-Based Research Study. Australian Journal of Educational Technology, Under Review

Kopeinik, S., Eskandar, A., Ley, T., Albert, D., & Seitlinger, P. (2018). Adapting an open source social bookmarking system to observe critical information behaviour.

In S. Dietze, M. D’Aquin, D. Gasevic, E. Herder, & J. Kimmerle (Eds.), Workshop Linked Learning 2018, Companion Proceedings of the WebSci 2018. Retrieved

from https://websci18.webscience.org/wp-content/uploads/2018/01/WebSci18_Events_PreProceedings-4-Linked_Learning_2018-lres.pdf

Ley, T., Seitlinger, P., & Pata, K. (2016). Patterns of Meaning in a Cognitive Ecosystem: Modeling Stabilization and Enculturation in Social Tagging Systems. In U.

Cress, H. Jeong, & J. Moskaliuk (Eds.), Mass Collaboration and Education (pp. 143–163). Heidelberg: Springer.

Ley, T., Leoste, J., Poom-Valickis, K., Rodríguez-Triana, M. J., Gillet, D., & Väljataga, T. (2018a). Analyzing Co-Creation in Educational Living Labs using the

Knowledge Appropriation Model. In Workshop on Co-Creation in the Design, Development and Implementation of Technology-Enhanced Learning (CC-TEL’18).

Aachen: CEUR Workshop Proceedings. Retrieved from http://www.ceur-ws.org/Vol-2190/CC-TEL_2018_paper_1.pdf

Ley, T., Maier, R., Thalmann, S., Waizenegger, C., Pata, K., & Ruiz-Calleja, A. (2018b). Adopting and Sustaining Innovation: A Knowledge Appropriation Model to

Connect Knowledge Creation and Workplace Learning. Vocations and Learning, under review.

Seitlinger, P., Ley, T., & Albert, D. (2015). Verbatim and semantic imitation in indexing resources on the Web: A fuzzy-trace account of social tagging. Applied

Cognitive Psychology, 29(1), 32–48. https://doi.org/10.1002/acp.3067

Seitlinger, P., Ley, T., Kowald, D., Theiler, D., Hasani-Mavriqi, I., Dennerlein, S., Albert, D. (2018). Balancing the Fluency-Consistency Tradeoff in Collaborative

Information Search with a Recommender Approach. International Journal of Human–Computer Interaction, 34(6), 557–575.

https://doi.org/10.1080/10447318.2017.1379240

Trattner, C., Kowald, D., Seitlinger, P., Ley, T., & Kopeinik, S. (2016). Modeling Activation Processes in Human Memory to Predict the Reuse of Tags. The Journal

of Web Science, 2(1), 1–18. https://doi.org/10.1561/106.00000004

Ruiz-Calleja, A., Dennerlein, S., Tomberg, V., Pata, K., Ley, T., Theiler, D., & Lex, E. (2015). Supporting Learning Analytics for Informal Workplace Learning with a

Social Semantic Infrastructure. In G. Conole, T. Klobučar, C. Rensing, J. Konert, & E. Lavoué (Eds.), Design for Teaching and Learning in a Networked World, 10th

European Conference on Technology Enhanced Learning. Heidelberg: Springer. https://doi.org/10.1007/978-3-319-24258-3_76

07-11 January 2019

Tallinn, Estonia

http://winterschool.tlu.ee

Educational Innovation – Getting ready for the future

Educational Robotics in Preschool and Primary Education

Tobias Ley

Professor for Learning Analytics

and Educational Innovation

Tallinn University, Estonia

Twitter @tobold

http://ceiter.tlu.ee

http://tobiasley.wordpress.com

H O W D O E S

C O N S E N S U S E M E R G E ?

Verbatim tag imitation: Preferential Attachment

(e.g. Dellschaft & Staab, 2008; Halpin et al., 2007; Rader & Wash, 2008)

Semantic imitation: Tag-based Topic Inference

(e.g. Fu et al., 2009, 2010)

p(t) p(a) p(o)N(t)

N(i)i

tn p(tn+1|tn)copying tag

p(t) p(c | r) p(t |c)

tn p(tn+1|c)

reconstructing tag

BOOK BOOK

BOOK BOOK

NOVELREAD