Embed Size (px)
Citation preview
University of Oulu
WHY PROCESS ANALYSIS?
‒ Learning process is affected by previous
learning experiences, and that these
experiences influence each other
between and within tasks (Molenaar &
Chiu, 2013).
‒ When we want to investigate how
individuals and groups engage to
learning, there is a need to consider both
order (sequential) and temporality
(temporality) of meaningful events, to
better understand how learning is shaped
in a situation.
11.4.2018 Lisää tarvittaessa alatunnisteteksti1
University of Oulu
WHAT TO ANALYSE?
Knowledge construction /Regulation /
Adaptation?
Individual / Group processes?
Variation, progress, fluctuation,
association?
Time frame/duration?
Type of data?
11.4.2018 Lisää tarvittaessa alatunnisteteksti2
University of Oulu
EVENT DATA ‒ How often a certain event occurs in the data?
‒ Time stamped information
1) Need to categorise the data.
- Data driven approach (e.g. logdata, navigational
data)
- Theory driven categories (events under theoretical
interest)
2) Problems
- Some events might occur often
- Categories needs to be reduced
- Often includes multiple rounds of analysis
Sequential and temporal analysis is based on
the events that are recorded or categorised
from the raw data as they occur (involve time
stamps and some event information) and
result often in abstracted models that mirror
reality
University of Oulu
WAYS TO DO PROCESS ANALYSIS?
‒ Multiple methods and techniques to do it,
such as
- Educational datamining from logdata (Malmberg et al 2015;2016).
- Process discovery (Malmberg et al., 2015, Sobocinsky et al., 2017)
- Lag sequential analysis (Malmberg et al., 2017, Kurki et al., 2017)
- T-pattern analysis (Kuvalja & Whitebread, 2014)
11.4.2018 Lisää tarvittaessa alatunnisteteksti4
University of Oulu
Analytical decisions to make
University of Oulu
Ways to present results
‒ How to present results from process
data?
- Quantitative and qualitative descriptions, case
descriptions, data visualization
‒ How to emphasize key results?
Example 1: Temporal analysis
1st Study: Promoting socially shared regulation of learning in CSCL: progress of
SSR among high and low performing groups
University of Oulu
Is there variation in the progress of socially shared regulation
of learning between low – and high performing groups?
University of Oulu
- Aim:
- Exploring socially shared regulation of learning in CSCL, particularly progress
of SSR among high and low performing groups
- Context:
- The first year teacher education students (N = 103, mean age 24,2 years)
- 3-4 member groups, altogether 30 groups
- Nine collaborative on-line learning sessions (1month)
- TASKS: How to utilize technology in future work?
- Data- Situated self-reports
University of Oulu
ANALYSIS PART I
1. Qualitative content analysis of shared challenges (f = 391) and SSRL strategies (f= 383) (Cohen’s k .75 - .83)
1. Challenge – SSRL strategy pairs (f = 27)
University of Oulu
EXAMPLE OF THE CODED CHALLENGES AND SSRL STRATEGIES
University of Oulu
ANALYSIS PART II
1. Composing a process model to identify how groups
progress in their SSRL
(https://fluxicon.com/disco/)
- Fuzzy miner (Günther & Van Der Aals, 2007)
- More significant events are emphasized
- Highlights more important paths
2. Mine less structured processes
University of Oulu
EXAMPLE OF THE COMPOSED PROCESS MODEL (N = 30, 20%)
REGULATING EXTERNAL CHALENGES
REGULATING COGNITIVE AND
MOTIVATIONAL CHALENGES
l
University of Oulu
LOW PERFORMING GROUPS (14)
REGULATING EXTERNAL CHALENGES
REGULATING COGNITIVE AND
MOTIVATIONAL CHALENGES
NO REGULATION
University of Oulu
HIGH PERFORMING GROUPS (8)
REGULATING EXTERNAL CHALENGES
REGULATING COGNITIVE AND
MOTIVATIONAL CHALENGES
REGULATING SOCIAL CHALLENGES
University of Oulu
IS THERE VARIATION IN THE PROGRESS OF SOCIALLY SHARED REGULATION OF LEARNING BETWEEN LOW –
AND HIGH PERFORMING GROUPS?
Example 2: Are we together or not? Temporal interplay of monitoring and physiological synchrony during a collaborative exam
University of Oulu
‒ Aim:
- To explore temporal interplay of monitoring and physiological synchrony during a collaborative exam
‒ Context:
- Four groups of three members, aged 15 to 16 years.
- Advanced physics course – Collaborative exam (28 minutes and 55 seconds (Std = 53s).
- TASK: CALCULATE REFRACTIVE INDEX OF LIGHT FOR WATER
‒ Data:
- Video observations, Electro Dermal Activity (EDA)
University of Oulu
Research questions
- How is physiological synchrony related
on students’ monitoring processes and
physiological arousal during
collaborative exam situation?
University of Oulu
Monitoring
(f)
Monitoring
Duration (Mean)
Monitoring
Duration (Total)
EDA Peaks*
Left student 25 0:00:04 0:01:30 434
Middle student 20 0:00:03 0:01:06 403
Right student 11 0:00:03 0:00:36 343
Group1 Total 56 0:03:12 1180
Left student 21 0:00:05 0:01:48 405
Middle student 26 0:00:05 0:02:09 260
Right student 3 0:00:03 0:00:10 118
Group 2 Total 50 0:04:16 783
Left student 39 0:00:04 0:02:34 601
Middle student 14 0:00:04 0:00:53 507
Right student 12 0:00:04 0:00:49 398
Group 3 Total 65 0:04:16 1506
Left student 23 0:00:04 0:01:20 493
Middle student 10 0:00:03 0:00:33 405
Right student 38 0:00:05 0:02:47 517
Group 4 Total 71 0:04:40 1415
Monitoring activity correlated (r = .663, p < 005) with the number of
EDA peaks
University of Oulu21
GROUP 1 GROUP 2
GROUP 3 GROUP 4
Each student contributed to joint monitoring during each
work phase.
The right student did not contribute to the joint monitoring
during the task interpretation phase.
During the task interpretation phase and the experiment
phase, neither the middle student’s nor the right
student’s monitoring was not followed by for the other.
The right student did not contribute to the joint monitoring
during task interpretation.
During the experiment and reporting phases, each
student’s monitoring was followed by the monitoring of
other group members.
OK..THIS DID NOT WORK..
University of OuluSetting up the experiment Reporting
22 minutes
18 minutes
19 minutes
E
D
A
E
D
A
Mean value of EDA peaks during the collaborative exam
GROUP 4 GROUP 2
GROUP 1
19 minutes
GROUP 3
19 minutes
E
D
A
What do we need to do?
How do we define refractive
index of light for water?
Its a math
Error!But it does not
show error, not
even that minus 1?
Well, then it
is 0.8. What
is the exact
value?
It should be
1.39
It is not even
close enough!
University of Oulu
19 minutes
GROUP 3
How can this be so easy!
We only need to calculate
refractive indef of light for
water?
University of Oulu27
Can how we draw
how did our
experiment?
Do we need to
draw? Maybe
we just explain it
shortly?
If we just explain
it..?
University of Oulu28
- Physiological synchrony
occured with the groups
who struggled with the
task.
- Earlier research has also
indicated, that
physiological synchrony is
correlated with group
tension and negative
expressions (Monster et
al., 2016)
Example 3: Sequential analysis
Children’s use of emotion and behaviour regulation strategies
Kristiina’s research
University of Oulu
‒ Aim:
- To explore how young children’s emotion and behaviour regulation manifests in socio-emotionally challenging situations in authentic educational settings and how the teacher’s presence makes a difference to it
‒ Context:
- Authentic open day-care activities, 2-5 year old children and teachers
‒ Data:
- Video-data of the activities. Focus in socio-emotionally challenging situations
University of Oulu
What can we find out about children’s strategy use from this data?
Frequency of strategy use and their
quality (categorization of strategies)
Which
strategies
occur
together?
Which
strategies occur
before/after
teacher
interference?
University of Oulu
Further research interests → Research questions
How do children respond to other children’s
behaviour in a challenging situation? Do
certain strategies cause another child to
respond in a certain way?
→ Research question How are the strategies
associated sequentially with other strategies?
What kinds of children’s behaviour makes
teachers interfere to children’s activities?
Does teacher interference, in turn, make
children use different strategies?
→ How are the strategies of children
associated sequentially with teacher
interference?
University of Oulu
Next step in analysis: Lag sequential analysis
The occurrences of how many
times each strategy follows one
another in a seven second interval
University of Oulu
2nd behaviour
Teacher
interference
Leaving the
situation/off
task
Physical/
verbal
regulation
Redirecting
activity/
attention
Providing
information
Expressing
emotions/
inhibiting
1st behaviour
Teacher interference Observed/
Expected
0/5.4 4/4 5/11.6 12/5.6 13/6.8 5/5.6
Adjusted Residual -2.6 .0 -2.4 3.1* 2.7* -.3
Leaving the
situation/off task
Observed/
Expected
16/5.3 4/3.9 3/11.3 9/5.4 1/6.6 5/5.4
Adjusted Residual 5.3* .0 -3.1 1.7 -2.5 -.2
Physical/ verbal
regulation
Observed/
Expected
22/22.8 10/17.1 64/49.2 19/23.5 26/28.9 24/23.5
Adjusted Residual -.2 -2.3 3.1* -1.3 -.7 .1
Redirecting
activity/attention
Observed/
Expected
3/4.8 7/3.6 7/10.4 14/5.0 1/6.1 3/5.0
Adjusted Residual -.9 1.9 -1.3 4.5* -2.4 -1.0
Providing information Observed/
Expected
11/17.1 12/12.9 47/37.0 7/17.7 27/21.7 20/17.1
Adjusted Residual -1.9 -.3 2.3* -3.2 1.5 .7
Expressing
emotions/inhibiting
Observed/
Expected
12/8.6 11/6.4 12/18.5 5/8.8 13/10.8 9/8.8
Adjusted Residual 1.4 2.0* -1.9 -1.5 .8 .1
Total Count/
Expected Count
64/64 48/48 138/138 66/66 81/81 66/66
Results
University of Oulu
5.3*
2.7*
What types of sequential associations are there between the children’s (1) strategies and (2) strategies and teacher interference?
Physical
and verbal
regulation
Providing
information
Expressing
emotions
Leaving the
situation/off
task
behaviour
Redirecting
activity/
attention
3.1*
4.5*
2.3* 2.0*Teacher
interference
3.1*
*Adjusted residuals, p =
.05
(z < 1.96)
Example 4: Sequential analysis
Fluctuation of motivation, emotion and cognitive regulation
in collaborative groups
Hanna’s research
University of Oulu
Aim:
‒ To explore how motivation and emotion regulation is associated with cognitive regulation?
Context:
‒ 24 second-year teacher training students (from 44 students, mean age 24. 9 years)
‒ 6 four-members’ groups
‒ Five two-hour time slots for group work
‒ Math didactics course lasting for seven weeks.
‒ A collaborative course assignment - a midterm plan for mathematics in primary school
Data:
‒ working was recorded with 360 degree video camera system.
‒ 22 collaborative group task sessions from 6 groups (3 to 5 sessions, M = 3.67 Std = 1.4).
‒ Altogether 21h 22min of video data
University of Oulu
Step 1 & 2 - How we started?
University of Oulu
Step 3 – Bring coded data for analysis and organise again and again….
University of Oulu
Step 4-n – Explore, trial-error, trial-error… … set conditions… finally a conclusion… or…?
University of Oulu
Step 4-n – Trial, error, trial, error… … finally a conclusion… or…?
University of Oulu42
RESULTS RQ 2 - How motivation regulation is associated with cognitive regulation?
%
3 4 5Paired associations between
regulation of 1) cognition and 2)
motivation & emotions
an occurrence of the regulation
pairs in relation to the time/progress
of the learning session
Meaningful associations between
regulation of 1) cognition and 2)
motivation & emotions
University of Oulu
Step n+1- Dive back in!
Meaningful associations between
regulation of 1) cognition and 2)
motivation & emotions
University of Oulu
Exercise
1. Work in pairs
2. Get to know raw data (excel sheet)
3. Discuss what kind of question you would like to explore/investigatea. What are the relevant variables to for your question?
b. How would you need to reorganise/filter/arrange the data?
c. what programs etc. you need to perform the analysis
4. Start exploring the data
University of Oulu
Contemporary perspective
Chen, B., Knight, S., & Wise, A. F. (2018). Critical Issues in Designing and Implementing Temporal Analytics. Journal of Learning Analytics, 5(1), 1-9.
