UGent MIS research seminar December 2014

1/29

FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION

www.janclaes.info

Jan ClaesTeaching assistant : PhD 2009 – 2015

Supervisors UGent : Geert Poels & Frederik GaillySupervisors TU/e : Paul Grefen & Irene Vanderfeesten

Investigating the process of process modeling and its relation to model quality

2/29


www.janclaes.info

Comprehensive overview PhD

3/29www.janclaes.info

Overview

DESIGN SCIENCEProcess model quality

improving modeling method

BEHAVIOURAL SCIENCELink between process and cognitive

overload

DESIGN SCIENCE

Process visualization

BEHAVIOURAL SCIENCELink between process and model

characteristics

DESIGN SCIENCETool support


Overview




overload

DESIGN SCIENCE



characteristics


P1

Chapter 1• BPM conference paper P1• Explorative research• Hypothesis building• Observational modeling sessions• Structured process modeling relates to

model quality• Correlation, t-test, linear regression


Overview




overload


characteristics


Chapter 2• BPM workshop paper P1• ISEB journal paper A1• PPMChart visualization• Observational modeling sessions• Visualization theories

DESIGN SCIENCE


P1 A1


Overview




characteristics


Chapter 3• CAiSE workshop paper P1• ISF journal paper A1• Explanatory theory development• Structured Process Modeling Theory (SPMT)• Cognitive theories• Observational modeling sessions• Correlation, t-test, PLS-SEM, EGA


overload

DESIGN SCIENCE


P1 A1




Overview


overload

DESIGN SCIENCE



characteristics


A1Chapter 4• Main contribution• Prescriptive theory, modeling method• Structured Process Modeling Method (SPMM)• > measure cognitive style• > determine modeling style


Overview




overload

DESIGN SCIENCE



characteristics

Chapter 5• Tool support• Visualizing structure of model to help

modeler during model creation• Phased modeling


?

9/29


www.janclaes.info

How to test the developed theory (SPMT)?


Research model

A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B

learning style

degree of serialization

adopted serialization style

need for structure

field-dependency

– +

intrinsic cognitive load for modeling phases

intrinsic cognitive load for aggregation phases

cognitive overload

intrinsic cognitive load for strategy building phases

+ + +

serialization style fitstructuredness of serialization

– – – –

1 2 3

process model quality overall construction effort

+–


Cognitive style

Need for structure

Learning styleField dependencySensory

Visual

Inductive

Active

Sequential

Intuitive

Auditory

Deductive

Reflective

Global

I ♥ ︎chaos I #$@&! chaos

DON’T SEE THE FOREST THROUGH THE TREES


Observational modeling sessionJools experimental platformSimple modeling notation

activity – edge – start event – end event – xor gateway – and gateway

Experimental flow Cognitive tests Pre-survey Tool tutorial Modeling sessions Post-survey Interviews


Cognitive testsFour cognitive tests

Operation Span Test Counting Span Test Reading Span Test Hidden Figures Test

Fully-automated in toolTaken at homeAvoiding cheating

Intrinsic motivation Limited cheat detection (score, time, tool focus)


Pre-survey and post-surveyCognitive style

Need for structure scale Learning style scale

Cognitive load Intrinsic, extraneous, germane cognitive load scale

Prior knowledge domain, language, modeling

Cognitive fit language, modeling language, tool

Demographics


Tool tutorial


Modeling session


InterviewsFeedback cognitive profileFeedback used modeling styleFeedback cognitive loadFeedback SPMTGeneral feedback


Modeling style

Flow oriented modeling


Modeling style

pausesnumber of big gaps

distancelines between

consecutive operations

progressnumber of operations

per 3 minutes

overlapnumber of overlapping

operations

alterationsnumber of non-green dots

20/29


www.janclaes.info

Processing empirical data: statistical madness


Modeling effectiveness and efficiencySemantic quality

Completeness & correctness (of content) Number of semantic errors


Modeling effectiveness and efficiencySyntactic quality

Correctness (of syntax) Number of syntactic cognitive errors


Modeling effectiveness and efficiencyTime

Time between start reading case and last operationEffort

Number of operations


Research model

A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B

learning style

degree of serialization

adopted serialization style

need for structure

field-dependency

– +

intrinsic cognitive load for modeling phases

intrinsic cognitive load for aggregation phases

cognitive overload

intrinsic cognitive load for strategy building phases

+ + +

serialization style fitstructuredness of serialization

– – – –

1 2 3

process model quality overall construction effort

+–


CorrelationCognitive style

Mod

elin

g st

yle

Modeling resultCognitive load


T-test

Learning style Need for structure Field dependency

Load

Synt

axTi

me

Effor

t


PLS-SEM


PLS-SEM


Thanks for you attention!

> Do you have feedback on my research plans? <> Do you have questions? <

Jan [email protected]

http://www.janclaes.infoTwitter: @janclaesbelgium

Business

UGent MIS research seminar December 2014