Bridging troubled water: Quality of Experience in a mobile media context

Katrien De Moor, Lieven De Marez

MICT-IBBT, Dept. of Communication Sciences, Ghent University

Wout Joseph, Tom Deryckere, Luc Martens

WiCa-IBBT, Dept. of Information Technology, Ghent University

Terena Networking Conference 2008

Introduction and context

Changing ICT environment User experience – Quality of Experience ‘Interactionism’ perspective User-driven innovation Examples

2MICT-WiCaDe Moor et al.

Today’s Innovation Spiral

Technology evolving at a dazzling speed, … intensified competition, … more innovations to distinguish themselves from competitors, …

shortening product life cycles, … increased pressure on NPD processes & introduction strategies, … more failing innovations, …

3

4

5

“More and more companies have recognized the need to learn from

users in order to develop innovative, user experience’-centred products”

(Sleeswijk Visser, Van der Lugt, Stappers, 2007: 35)

”The consumer is king – and needs high QoE” (Kumar, 2005: 39)

”QoE became the ultimate business metric & competitive battleground” (Empirix, 2005: 1)

Increased importance of the integration of user- & user experience oriented research in NPD processess

6

7

8

The era of ‘user experience technology’?

Quality of Experience

= buzzword, USP, ultimate business metric,…

Central question

How to deliver high QoE for end-users?

Problem: ‘Clash’ of visions, fragmented efforts

MICT-WiCaDe Moor et al.

9

Striving for QoE delivery: ‘clash of visions’

“Natuurlijk vonden veel technici het vervelend dat geen enkel technisch detail

genoemd mocht worden, maar in dit geval moesten de emotie en de experience spreken”

(Philips, 06/12/2007)

A: Gaining insight in what users want, expect, feel, think,… as a means to enhance end-user QoE

B: QoS optimisation as a means to optimise end-user QoE

QoE = QoS ?

MICT-WiCaDe Moor et al.

10

QoE = QoS?

Delivering high QoE should be the main goal in NPD

and striving for QoS optimisation a necessary, but unsatisfactory means to achieve that goal

QoS is only one necessary component of QoE!

MICT-WiCaDe Moor et al.

QoE as a multidimensional concept (1)

5 Building blocks ‘Quality of Effectiveness’ (~QoS)

Application/Service Server Network Device/Handset

‘Quality of Efficiency’ (does it work well enough for the user?)

Device/Handset Network Application/Service

11MICT-WiCaDe Moor et al.

QoE as a multidimensional concept (2)

5 Building blocks ‘Usability’ (behavioural and emotional)

‘Expectations’ (degree up to which expectations are met (or eg. exceeded), will determine part of the QoE

‘Context’ Environmental Context (eg. public/private, mobility, usage

context, …) Personal & Social Context (eg. Identity creation, emotions,

empowerment, …) Cultural Context (eg. age, values, …) Technological Context (eg. Tangibles, billing, cost, usage

intensity, compatibility, …) Organisational Context

12MICT-WiCaDe Moor et al.

13

Multidimensional conceptual model

14

Example Conceptual model, based on desk research & expert panel

QoE

QoS

Example GAMING STUDY

Usability

Expectations

Context

Latency, quality of image, speed …

Easy navigation, interface…

gameplay, content, response time…

Community aspect, online, cost, …

MICT-WiCaDe Moor et al.

15

Currently: too much post development measurements (3) and usability and QoS measurements (2)

12

3Prior-to-development & Prior-to-launch

????

Expectations?

(Post)Development & Prior-to-launch

Prototypes, demo’s, ….

Usability testing

QoS optimisation

Post development & Post-launch

User research:

Satisfaction measurements

Good experience?

Delivering high QoE: how?

MICT-WiCaDe Moor et al.

16

New methodology for bridging the QoE-QoS gap

MICT-WiCaDe Moor et al.

Empirical case-study

Study of 2 I-City applications, N=10

Interdisciplinary team

Focus on bridging gap between QoE (user-oriented) and QoS (technology-oriented approaches) in NPD

Translation/transformation of insights from user research

test and refinement of new methodology 5 phases

17MICT-WiCaDe Moor et al.

Some results (1)

18

1. Pre-usage user research: detection of relevant QoE dimensions and user expectations (quali+quantitative)

e.g. price, navigation, speed, display size, …

2. Pre-usage translation workshops social scientists and engineers

e.g. photo-download speed application

3. Monitoring during usage usage scenarios for test users

different reception levels + monitoring of ‘signal strenght’

probe model (cfr. Deryckere, Joseph et al, 2008)

MICT-WiCaDe Moor et al.

Some results (2)

19

4. Post-usage questions on device: after completion of usage scenario questions on device (general experience, frustration,,…)

example: resp. B, male, 30 (Wineguide)

Some results (3)

20

5. Post-usage user research: comparison of expectations vs. experience: gaps? Explanation?

e.g. person B, 30, male (Wineguide)

MICT-WiCaDe Moor et al.

Conclusion

- Integrated approach for QoE-measurement- Both objective and subjective parameters- Future research:

• Larger scale testings: more test users and usage contexts

• Extended probe model: more parameters for monitoring (e.g. contextual probe: GPS data, user-generated information about current activities, mood, …)

• Personalised set of questions on device

21MICT-WiCaDe Moor et al.

22

THANK YOU!

QUESTIONS?

CONTACT US:

http://www.mict.be

http://www.wica.intec.ugent.be/