An Empirical Evaluation of 2D and Interactive 3D Terrain ... · An Empirical Evaluation of 2D and Interactive 3D Terrain Visualisations for Cycling Maps – MSc Thesis Defence 1 Introduction

■ Myles Cook

MSc Thesis Defence

Cartography MSc

15th May 2017

An Empirical Evaluation of 2D and Interactive 3D Terrain Visualisations for Cycling Maps

An Empirical Evaluation of 2D and Interactive 3D Terrain Visualisations for Cycling Maps – MSc Thesis Defence

1 Introduction

2 Context & Background

3 Research Questions

4 Methodology

5 Results & Discussion

6 Conclusions

7 Limitations & Future Research

P R E S E N TAT I O N

OUTLINE


1 Introduction

•Howcanwebestshow the hillsonacyclingroute?

•Whichvisualisationsofterrainaremost usableforacyclist?

•Whichvisualisationsaremoreusableforcycleroute-planning–2D or 3D?

What problems stimulated the research?

To assess the relative usability of 2D and 3D elevation visualisations for

cycle route planners

Broad Aim

1. Introduction

2. Context & Background

3. Research

Questions

4. Methodology

5. Results & Discussion

6. Conclusions

7. Limitations & Future Research


1 Introduction

•Growthintheubiquityof3Dandinteractivecartographies,butdotheyofferanyrealbenefits?

•Researchinto3Dcartographyusabilityhasdrawndiffering conclusions,andsuffersfrommethodological flaws.

•Forcyclists,terrain is important.Bettercommunicationofterrainshouldmeanbettercyclingmaps.

•Thereisaneedforbetter design guidelinesforcyclingmaps.

Justification – why do we care?

2D

vs.

3D

1. Introduction


3. Research

Questions

4. Methodology


6. Conclusions



2 Context and BackgroundWhat other work has been done in this field?

Expand on Brügger’s research by looking at 3D visualisations, while addressing that study’s flaws.

CyclingMap

Research

•Maps to make cyclists ‘happy’ (Dickinson, 2012), and ‘universal’ maps to meet the needs of all users (Wessel & Widener, 2015).

•Most relevant research – Brügger et al (2016):

▪︎ Study of the usability of 2D terrain depictions for cycle planners.

▪︎ Results showed that arrow symbolisation was most efficient & effective for determining height, and colour for determining slope.

▪︎ User preference = elevation profile.

▪︎ BUT! Methodological flaws (potential learning effects) & a-typical visualisations...

THIS THESIS:

1. Introduction


3. Research

Questions

4. Methodology


6. Conclusions


2 Context and BackgroundWhat other work has been done in this field?

So, the need for further research into 2D and 3D terrain visualisations for cycling maps is still there!

2Dvs.3Dcarto

research

•A larger body of research has focussed on the usability of 2D and 3D cartographies in general.

•But, it has issues...

▪︎ Results can’t necessarily be applied to cycling maps/route planners.

▪︎ Some research has suggested 3D is more usable than 2D, other research the opposite.

▪︎ Focusses mainly on 3D terrain models.

▪︎ Majority use static rather than interactive 3D depictions.


1. Introduction


3. Research

Questions

4. Methodology


6. Conclusions


3 Specific Research Questions

Toassesstherelativeusabilityof2Dand3Delevationvisualisationsforcyclerouteplanners.

1.Isthereasignificantdifferenceintherelative efficiencyof2Dand3Delevationvisualisationsforcycleroute-planners?

2.Isthereasignificantdifferenceintherelative effectivenessof2Dand3Delevationvisualisationsforcycleroute-planners?

3.Dousersprefer2Dor3Delevationdepictionswhenperformingcycleroute-planningtasks?

Research Questions

Broad Aim

What are we trying to find out?

Efficiency + Effectiveness + User Preference = Usability (ISO 9241-11)


1. Introduction


3. Research

Questions

4. Methodology


6. Conclusions


4 Methodology•Conductedanempirical user-study(repeatedmeasures,36participants),undercontrolledexperimentalconditions.

•Testedsix different visualisation types:

Terrain

Por terBr ook

BlackbrookWood

Nor

thf i

eld

Road

F ulwo od Roa d

Tom Lan e

O ak br oo k Roa d

Ly dg a te Lan e

Tof t s L ane

S lay l e ig hLa ne

Gr a h am R oad

Car si ck

H i llR o ad

Net her g reen Roa d

San dy gateRo ad

Rust l ings Roa d

G l adston eRoa d

Red mir es R oad

Hag

gLa

ne

Hal l am sh ir e Roa d

T ink er

L ane

B ol eHi ll

Roa dH opw oo

dLa

ne

Boleh ill L ane

Bing ley L an e

Lo ng L aneRos coe

Ba nk

Han

gin

gw

ate r

Ro a

d

Cold

well

L a ne

Cr o ok es

Crim

ic arL an e

Ivy Par kRoa d

Ste p he n

Hil l

Ha llam

G

range

Road

Back

L a

ne

Manch ester R oad

Manche ste r Road

Manche ster Road

Rivelin Valley Road

A57

A625

A57

A610 1

Rivelin Glen

WindleHou se Farm

Samehill Farm Hagg LaneCott ages

BingleyVilla

EndcliffeCrescent

Ranmoor

Endcliffe Edge

Carsick

Tapton Hill

Crosspool

Stephen Hill

Lydgate

Den Bank

Crookes

Carsick Hill

Hallam Head

Sandygate

Rivelin Glen

Fulwood

Stumperlowe

Path Slope

Slight Uphill Slope

Moderate Uphill Slope

Steep Uphill Slope

Por terBr ook

BlackbrookWood

Nor

thf i

eld

Road

F ulwo od Roa d

Tom Lan e

O ak br oo k Roa d

Ly dg a te Lan e

Tof t s L ane

S lay l e ig hLa ne

Gr a h am R oad

Car si ck

H i llR o ad


San dy gateRo ad

Rust l ings Roa d

G l adston eRoa d

Red mir es R oad

Hag

gLa

ne


T ink er

L ane

B ol eHi ll

Roa dH opw oo

dLa

ne

Boleh ill L ane

Bing ley L an e

Lo ng L aneRos coe

Ba nk

Han

gin

gw

ate r

Ro a

d

Cold

well

L a ne

Cr o ok es

Crim

ic arL an e

Ivy Par kRoa d

Ste p he n

Hil l

Ha llam

G

range

Road

Back

L a

ne

Manch ester R oad

Manche ste r Road

Manche ster Road

Rivelin Valley Road

A57

A625

A57

A610 1

Rivelin Glen

WindleHou se Farm


BingleyVilla

EndcliffeCrescent

Ranmoor

Endcliffe Edge

Carsick

Tapton Hill

Crosspool

Stephen Hill

Lydgate

Den Bank

Crookes

Carsick Hill

Hallam Head

Sandygate

Rivelin Glen

Fulwood

Stumperlowe

ORANGE

PURPLE

GREY

Path Slope

Por terBr ook

BlackbrookWood

Nor

thf i

eld

Road

F ulwo od Roa d

Tom Lan e

O ak br oo k Roa d

Ly dg a te Lan e

Tof t s L ane

S lay l e ig hLa ne

Gr a h am R oad

Car si ck

H i llR o ad


San dy gateRo ad

Rust l ings Roa d

G l adston eRoa d

Red mir es R oad

Hag

gLa

ne


T ink er

L ane

B ol eHi ll

Roa dH opw oo

dLa

ne

Boleh ill L ane

Bing ley L an e

Lo ng L aneRos coe

Ba nk

Han

gin

gw

ate r

Ro a

d

Cold

well

L a ne

Cr o ok es

Crim

ic arL an e

Ivy Par kRoa d

Ste p he n

Hil l

Ha llam

G

range

Road

Back

L a

ne

Manch ester R oad

Manche ste r Road

Manche ster Road

Rivelin Valley Road

A57

A625

A57

A610 1

Rivelin Glen

WindleHou se Farm


BingleyVilla

EndcliffeCrescent

Ranmoor

Endcliffe Edge

Carsick

Tapton Hill

Crosspool

Stephen Hill

Lydgate

Den Bank

Crookes

Carsick Hill

Hallam Head

Sandygate

Rivelin Glen

Fulwood

Stumperlowe

270m155m160m

Elevation (m)

Por terBr ook

BlackbrookWood

Nor

thf i

eld

Road

F ulwo od Roa d

Tom Lan e

O ak br oo k Roa d

Ly dg a te Lan e

Tof t s L ane

S lay l e ig hLa ne

Gr a h am R oad

Car si ck

H i llR o ad


San dy gateRo ad

Rust l ings Roa d

G l adston eRoa d

Red mir es R oad

Hag

gLa

ne


T ink er

L ane

B ol eHi ll

Roa dH opw oo

dLa

ne

Boleh ill L ane

Bing ley L an e

Lo ng L aneRos coe

Ba nk

Han

gin

gw

ate r

Ro a

d

Cold

well

L a ne

Cr o ok es

Crim

ic arL an e

Ivy Par kRoa d

Ste p he n

Hil l

Ha llam

G

range

Road

Back

L a

ne

Manch ester R oad

Manche ste r Road

Manche ster Road

Rivelin Valley Road

A57

A625

A57

A610 1

Rivelin Glen

WindleHou se Farm


BingleyVilla

EndcliffeCrescent

Ranmoor

Endcliffe Edge

Carsick

Tapton Hill

Crosspool

Stephen Hill

Lydgate

Den Bank

Crookes

Carsick Hill

Hallam Head

Sandygate

Rivelin Glen

Fulwood

Stumperlowe

Arrow Line-Width

Colour ElevationProfile ElevationProfile

2D 3D


1. Introduction


3. Research

Questions

4. Methodology


6. Conclusions


4 Methodology

•Foreachvisualisationtype,userswereaskedtofind:

1.Thehighestofthreepoints.2.Thesteepestofthreeslopes.3.Whichofthreeroutescontainedthehighesttotalamountofclimbing?

•Usedaweb-hostedsurvey(basedonLimeSurveyopensourcesoftware),allowing:

1.Automatedrecordingofresponses(tocalculateeffectiveness).2.Automatedrecordingofresponsetimes(tocalculateefficiency).3.Submissionofuserfeedback(todetermineuserpreference).

•Visualisationandstatisticalanalysisofresultsdatathenidentifiedsignificantdifferencesbetweenvisualisationtypes,foreachofthethreeusabilitycriteria.


1. Introduction


3. Research

Questions

4. Methodology


6. Conclusions


4 Methodology

Exampleuser-studyquestion(heightdetection).


1. Introduction


3. Research

Questions

4. Methodology


6. Conclusions


5 ResultsEfficiency

Box-whiskerplotsshowingresponsetimedifferencesforeachvisualisation(forallquestiontypes).

•Relativeefficiencydependentuponroute-planningtask(height/slope/climbdetection).

•Line-widthwasmostefficientforthemajorityoftasks.

•Forslopetasks,arrowandcolourvisualisationsweresignificantlymoreefficientthanallothers.

•3Dterrainmodel=consistentlyinefficient.

•Nosingledimensionality(i.e.2Dor3D)wasconsistentlymoreefficientthantheother.

Measured using user response

time


1. Introduction


3. Research

Questions

4. Methodology


6. Conclusions


5 ResultsEffectiveness

Visualisation Type

LineColourArrow3D Terrain3D Profile2D Profile

Cor

rect

Ans

wer

s (p

erce

nt)

100

80

60

40

20

0

Page 1

Differencesinthepercentageofcorrectanswersbetweendifferentvisualisations(usingdatapooledfromallquestiontypes)

•Relativeeffectivenessalsodependentontask(height/slope/climbdetection).

•Nosingledimensionality(i.e.2Dor3D)wasconsistentlymoreeffectivethantheother.

•Thelowerthecognitiveloaddemandedbythevisualisation,themoreeffectiveitappearedtobe.

•e.g.pointmatchingorjudginglineangles=highcognitiveload=ineffectiveMeasured

using the proportion of correct answers


1. Introduction


3. Research

Questions

4. Methodology


6. Conclusions


5 ResultsUser Preference

Question Type

Slope TasksHeight TasksClimb Tasks

Par

tici

pant

Cou

nt20

15

10

5

0

Line Width

Colour

Arrow

3D Terrain

3D Profile

2D Profile

Page 1

VisualisationType

Thevariationinuserpreferencefordifferentroute-planningtasks.

•Visualisationpreferencedependedonthetaskathand.

•However,themajorityofuserspreferred2Dvisualisations,foralltasks.

•The2Dprofilewasespeciallypopular.

•Dislikeof3Dvisualisationsmainlystemmed(accordingtomajorityofusercomments)fromtheneedforinteraction.

Solid fill – 2D

Line fill – 3D


1. Introduction


3. Research

Questions

4. Methodology


6. Conclusions


6 ConclusionsWhat can we conclude from these results?

1. The diversity of different 2D and 3D visualisation types, means we cannot state that one dimensionality is ‘more usable’ or ‘better’ than the other.

2. The usability of each visualisation type is inherently tied to the problem it is being used to solve.

3. The greatest impact on usability appears not to be visualisation dimensionality, but the cognitive loadplacedupontheuser.

4. 3D visualisations which presented targeted, abstracted versions of reality appear more useful than those which simply attempt to imitate reality.


1. Introduction


3. Research

Questions

4. Methodology


6. Conclusions


7 Limitations & Future Research

LimitationsFuture

Research

Artificial test environment - indoors, quiet, constant lighting.

Repeat the study in an outdoor setting using mobile devices.

Inconsistent data visualisation: slope/elevation/both.

Only compare like for like, e.g. 2D profile with 3D profile

Participants were all highly educated, and mostly skilled in Cartography.

Include a wider range of participants & use a study design that allows testing of educational influence.

Lack of eye tracking (due to interactivity) = hard to explain usability differences.

Develop/apply eye-tracking techniques that can be used with interactive systems.


1. Introduction


3. Research

Questions

4. Methodology


6. Conclusions



Thankyou!

Any questions?

Documents

An Empirical Evaluation of 2D and Interactive 3D Terrain ... · An Empirical Evaluation of 2D and Interactive 3D Terrain Visualisations for Cycling Maps – MSc Thesis Defence 1 Introduction