User-centered Research at FTW, QoE and the ACE Project · QoE for Mobile Broadband Services - QoE...

Forschungszentrum Telekommunikation Wien

Peter Reichl, Raimund Schatz, Sebastian EggerTelecommunications Research Center Vienna (FTW)

QoE Workshop Paris, 18. Sept. 2009

The Human Factor in Telecommunications

User-centered Research at FTW, QoE and the ACE Project

18.09.2009 © ftw. QoE Workshop Paris - 2-

A Brief History of FTW

Founded in 1999 by key industry players

and the Technical University of Vienna

Non-profit R&D Center in communication

technologies with focus on collaborative

application-oriented research projects

Since 2008 K1 center within COMET

competence center funding program

Core team of around 50 international

researchers with international reputation

Holistic approach along the complete value

chain up to the end customer and her needs

18.09.2009 © ftw. QoE Workshop Paris - 3-

Mission: Bridge the Gap betweenAcademic and Industrial Research

18.09.2009 © ftw. QoE Workshop Paris - 4-

Research Areas & Application Sectors

TransportAutomotive

Logistics

MediaContent IndustriesCreative Industries

HealthcareHome NetworksAssisted Living

Telecom OperatorsSystem Integrators

Manufacturers

Cor

e C

ompe

tenc

e A

reas

Application Sectors

Communication Health &Living

IntelligentTransportation

Information &Entertainment

Signal & Information ProcessingArea I:

Networked ServicesArea S:

Communication NetworksArea N:

Markets

Needs

User-centered Interaction & VisualizationArea U:Area U: User-centered Interaction and Communication Economics

18.09.2009 © ftw. QoE Workshop Paris - 5-

User-centered Research @ FTW

Idea: Complement expertise in communication technologies with an interdisciplinary user-centric perspective

Design through Dialogue

Hi-Fi Prototyping

Quality of Experience (QoE)

18.09.2009 © ftw. QoE Workshop Paris - 6-

Design Through Dialogue: The i:lab @ FTW

Interfaces and Interaction Laboratory

18.09.2009 © ftw. QoE Workshop Paris - 7-

i:lab Portfolio and Facilities

18.09.2009 © ftw. QoE Workshop Paris - 8-

Towards User Trials in the Wild: LiLiPUT

Lightweight Lab Equipment for Portable User Testing

Goal: lightweigth, transportable and flexible equipment foruser tests in various real life environments

A typical example: Roto et al. 2004

18.09.2009 © ftw. QoE Workshop Paris - 9-

Towards User Trials in the Wild: LiLiPUT

Our idea: use state-of-the-art telecommunications fully wireless approach

Lightweight Lab Equipment for Portable User Testing

Goal: lightweigth, transportable and flexible equipment foruser tests in various real life environments

18.09.2009 © ftw. QoE Workshop Paris - 10-

Hi-fi Prototyping: WikiVienna and P2D

WikiVienna Point-to-Discover

New Interaction Techniques

18.09.2009 © ftw. QoE Workshop Paris - 11-

Quality of Experience (QoE)

Goal: methods & tools for holistic evaluation of perceived quality

QoE for Voice and Video Services- Impact of network impairment - Focus on conversational interactivity

QoE of Convergence Applications- E.g. Social TV and Mobile iTV- Focus on social presence and emotions

QoE and Telecommunications Economics- Charging architectures for QoE- Pricing schemes for differentiated services

QoE for Mobile Broadband Services- QoE for web browsing, downloads, etc.- QoE prediction in 3G networks via passive monitoring (ACE)

International activities: Dagstuhl, COST 298, 2102, IS0605

Forschungszentrum Telekommunikation Wien

QoE Workshop, 18. Sept 2009, Paris

Raimund SchatzTelecommunications Research Center Vienna (FTW)

18.09.2009 © ftw. QoE Workshop Paris - 13-

Agenda

Background & Motivation

Project Overview

Selected Results

Conclusions

Future Research Interests

18.09.2009 © ftw. QoE Workshop Paris - 14-

Background: QoE estimation as Use Case for Passive Network Monitoring

Increasing capabilities of passive 3G monitoring systems Idea: extend scope from monitoring low-level QoS metrics to user-level QoE

Advantages: Quality assessment on a holistic and realistic basis Continuous monitoring without costly involvement of test users Quality problems can be detected in real-time

Challenge: How to predict QoE on behalf of network-level measurements?

18.09.2009 © ftw. QoE Workshop Paris - 15-

Project ACE: Quick Facts

Multi-firm research project

Start: 1. April 2009

Duration: 18 months

Consortium:

- mobilkom austria

- Kapsch CarrierCom

- ftw. (Telecommunications Research Centre Vienna)

Goal:

- Demonstrate feasibility of automated QoE estimationin 3G networks using passive monitoring

- Focus on Mobile Broadband Scenarios (HSPA)

18.09.2009 © ftw. QoE Workshop Paris - 16-

ACE: QoE Focus

Focus on the mobile network‘s contribution to QoE

18.09.2009 © ftw. QoE Workshop Paris - 17-

Approach

Development of service-specific quality models- Map network-level KPIs to user centric high-level KQIs

Implementation of quality models- Extensions of passive monitoring systems

Validation of results- Live 3G User trials

DefineKQIs

WeighKQIs

MapQoE/QoS

MeasureQoS KPIs Validate

Userfocus

Network focus

Network + Userfocus

User Studies + Technical Analysis Implementation Trials

18.09.2009 © ftw. QoE Workshop Paris - 18-- 18-

ACE QoSQoE modeling framework

Technical Analysis,Traffic Monitoring

User Studies

MAIN TOOL/METHOD

18.09.2009 © ftw. QoE Workshop Paris - 19-- 19-

Targeted Services and Use Cases

VoIPVoice

Remote Folder access

E-Mail

File Download

Web Browsing

ServiceCategory

Mobile Broadband (Data)

Different Services

Different Quality Characteristics

Different Quality Models

have

require

18.09.2009 © ftw. QoE Workshop Paris - 20-- 20-

ACE MBB User Studies: Technical Setup

INTERNET

Metawin Tracer

Client Laptop w. Browser

OPTIONAL: Tablet for forms,ratings, etc.

Test operator front-end

WLAN Access point

„SURF“-LAN

„OPERATOR“-LAN

ACE MBB Tests Hardware & Network SetupVersion 1.4

UMTS Downlink Emulator

Control connections

RT1 RT2

EMU1

AP1

LAP

TRC

OP1

UMTS Uplink Emulator

EMU0

18.09.2009 © ftw. QoE Workshop Paris - 21-

MBB Studies So Far

Application/Use Case Categories:- Web Browsing- File Downloads (zip, mp3)- Progressive Downloads (YouTube)

QoE Impact of: - Bandwidth, Network Delay- UMTS Network States- Fluctuations in Quality (Bandwidth Variance)- Accessibility: Connection Setup Time- Reliability: Outages

Characteristics:- ~85 participants in total- Balanced demographics (gender, age, etc.)- Consistent set of reference tasks and content

Result: a coherent, representative body of data

18.09.2009 © ftw. QoE Workshop Paris - 22-

Application to Passive Monitoring

Practical Challenges- Generalizability of QoE models devised

- Scaleability (network-wide monitoring on a per-user level)

- Traffic classification (differentiation between different services)

Learnings- Thoroughly controlled users experiments are key

- Depth better than breadth focusing enablescreation of ‚QoE Building Blocks‘

- Early validation with live data (i.e. network traces) essential

- QoE modeling requires multiple iterations

18.09.2009 © ftw. QoE Workshop Paris - 23-

Conclusions

Shift from network QoS to QoE monitoring- A step change in complexity …

… but a highly rewarding one!

- First project results demonstrate viability of QoE modelingfor data services such as web browsing

Contribution of the project- Methodology, tooling and reference tasks for QoE lab

studies

- QoE Models (for selected mobile broadband data services) of the impact of key network parameters

- Assessment of the viability of live QoE-monitoring via passive network monitoring

18.09.2009 © ftw. QoE Workshop Paris - 24-

Future Research Directions

Fundamental Questions- Subjective nature of QoE

- Relevant QoE dimensions (user & techie perspective)

- Metrics beyond MOS

Standardized Approach- Methodologies for user studies

- Benchmark tasks for data services (contexts & content)

Economic consequences- Charging for QoE vs charging in QoE

- Translating customer experience into business models

Forschungszentrum Telekommunikation Wien

Thank You Very Much

For Your Attention!

{reichl | schatz | egger} @ftw.at