Intelligent and Adaptive Middleware to Improve User-Perceived QoS in Multimedia Applications

Pedro M. Ruiz, Juan A. Botia,

Antonio Gomez-Skarmeta

University of Murcia

Terena Networking Conference 2004

Rhodes, June 2004

2

Drivers for adaptive applications

• E2E QoS requires local resource management• Terminals are heterogeneous and media adaptation is

needed• Network conditions are unpredictably changing and not

under control (e.g. Ad hoc nets, PLC networks, etc.)• QoS in terms of bandwidth and delay cannot be

guaranteed just with network-layer QoS mechanisms• In these cases, user-perceived QoS can be improved

using applications being able to adapt to:– Network conditions– QoS Violations– Shortage of local resources (eg. CPU, Memory, etc.)– User preferences

3

Adaptive Applications

4

Video Quality vs. Bandwidth

Non-linear

Perception

1190 Kbps

300 Kbps

210 Kbps

140 Kbps

70 Kbps

50 Kbps

30 Kbps

10 FPS, SQCIF both for MJPEG and H.263

5

Towards user-awareness...

• Traditional approaches based on profiles– Simple and easy to implement– Usually are not fine-grained enough– Are not able to capture the perceptual preferences

• Adaptations based on low level parameters (e.g. Bandwidth, packet losses, etc)– Do not really consider user preferences– Perceptual QoS is not linearly related to those low level metrics

• Previous works focused on evaluating the impact of each parameter on the user perception– There is not a real model of the user, particularly when multiple

parameters can be tuned simultaneously (e.g. codecs, frame rates, etc.)

6

App. Layer

Audio Video ... Slides

QoS signaling

Net. Layer Mon Adaptation Logic

QoS signaling

Net.Layer Mon Adaptation Logic

TCP/UDP

IP

MAC

PHY

TCP/UDP

IP

MAC

PHY

Audio Video ... Slides

App. Layer

Architecture for Multimedia Adaptive Applications

Type Seq # Loss % Delay User pref Estimated BW

Audio CodecVideo CodecFrame Rate

SizeQuantization

...

7

Basic Adaptation Logic

8

Why applications aware of the user-perceived QoS?

• There are many ways to adapt data-rates to the available bandwidth– Audio & Video Codecs– Video Quantization factor– Audio sampling rate– Video frame rate– Video size– Component selection– Buffering

• Not trivial to select a new combination of settings satisfying the users– Reduce frame size?, reduce frame rate?, change codec?

• Traditional adaptive applications improve user-perceived QoS but they offer sub-optimal solutions

• Adaptive applications should be able to deal with the user perception of QoS!

Which combination would be preferred by the user?

9

Our proposal for user awareness

• Use of machine learning techniques to help at modeling the user perceived QoS– Number of possible combinations of application

settings is big enough!– Perceptual QoS may change from one indivudual to

another and it is extemely complex to be analitically modelled

– A “black box” model may resemble the user-satisfaction without needing to understand the complex processes involved in user perception

10

Modelling user-perceived QoS

• Difficult to model, due to the subjective aspects involved

• We apply a rule induction machine learning algorithm over learned data

0

20000

40000

60000

80000

100000

120000

140000

160000

0 20 40 60 80

Time(s)

BW 33, 56, 88, 128, 384Kb/sACOD PCM, G.711u, G722, GSMVCOD MJPEG, H263FSIZE CIF, QCIF, 160x128Quant 5, 10, 15, 30, 60FPS 2..24Loss% 0..100%Score 1..5 (according to MOS)

Initial data-set(864 entries)

SLIPPER algorithm with t=5

Set of rules representing user-perceived QoS

11

if matchConfidence { [QFVIDEO >= 60, VIDCOD = MJPEG, FSIZE = QCIF, LOSS <= 10, FPS >= 6] -> 2.8792 [AUDCOD = GSM, BW >= 80, QFVIDEO >= 30, FSIZE = QCIF, FPS <= 6] -> 1.4357 [AUDCOD = GSM, BW >= 128, LOSS = 0, QFVIDEO >= 30, FPS >= 3, VIDCOD = MJPEG] -> 1.7013 [] -> -2.4188} > 0 then 5 else if matchConfidence { [BW >= 384, QFVIDEO >= 40, FSIZE <= 2] -> 2.7121 [QFVIDEO >= 30, VIDCOD = MJPEG, LOSS <= 3, AUDCOD = G722, BW >=80] -> 1.1756 [FSIZE = CIF, QFVIDEO >= 30, LOSS <= 3, AUDCOD = G722, BW >= 80] -> 1.4437 [] -> -1.5044} > 0 then 4 else if matchConfidence { [LOSS >= 30] -> 2.1188 [QFVIDEO <= 5] -> 1.4142 [LOSS >= 16, FPS <= 3] -> 1.5438 [] -> -1.0984207275826066} > 0 then 1 else if matchConfidence { [LOSS >= 16] -> 1.9109 [QFVIDEO <= 10, FSIZE = QCIF] -> 1.5861 [FSIZE = 160X128, QFVIDEO <= 40, VIDCOD = H.263] -> 1.2546 [] -> -0.3953} > 0 then 2 else 3

Rules Generated by SLIPPERSome of the lessons learnt from rules

Higher FR => higher QoS but user’s prefer lower FR (not below 4 FPS) provided that the video is bigger

In most cases PCM audio is not required. The bandwidth savings can be used to improve other components

Audio QoS has greater impact

Etc..

12

Empirical Results

• Scenario– Real MMARP-based ad

hoc network– Path specifically selected

to guarantee variability

• Application– ISABEL-Lite with

extensions

• Trials– Traditional multimedia

application– Adaptive multimedia

application

13

Total Losses

14

Histogram audio/video loss-rate

15

User’s Mean Opinion Scores

16

Conclusions and Future Work

• Adaptive applications have demonstrated to be effective in wireless and mobile scenarios

• The machine learning user’s modelling has shown to be effective

• Applications aware to the user-perceived QoS have demonstrated to offer to better satisfy user’s QoS expectations in a real ad hoc wireless networks

• Optimization on the triggering of the adaptation have demonstrated Future work include among others– Reinforcement learning inside the terminal– Combination with user profiling mechanisms

Intelligent and Adaptive Middleware to Improve User-Perceived QoS in Multimedia Applications

Pedro M. Ruiz, Juan A. Botia,

Antonio Gomez-Skarmeta

University of Murcia

Terena Networking Conference 2004

Rhodes, June 2004