Control-based Quality Adaptation in Data Stream Management Systems (DSMS)

Control-based Quality Adaptation in Data Stream

Management Systems (DSMS)

Data Stream Management System (DSMS)

• Continuous data, discarded after being processed

• Continuously answers queries

• Applications– Financial analysis– Mobile services– Sensor networks– Network monitoring– More …

User

DSMS

User

User

DataQuery

Results

DSMS architecture

• Network of query operators (1 – 12)

• Each operator has its own queue

• Scheduler decides which operator to execute

• Query results pushed to clients

• For our purposes, DSMS can be viewed as a blackbox

4 11

73 10

QueryEngine

S1

S2 III

I

2

5

6

1

9

12

DataStreams

OutputStreamsII

8

DSMS

Data Input Rate

Average DelayLoad

Adaptor

Load Shedding

• Eliminating excessive load by dropping data items less QoS violations

• Basic algorithm (Tatbul et al., 2003):

• Key questions– When?– How much?– Where?

What’s missing?

• Current solutions focus on steady-state performance– Open-loop control ?

• Assuming there inputs reach steady states

• However, arrivals are bursty in practice – always in transient state

• The solution: closed-loop control

Load

Time

CPUcapacity

Why Closed-Loop Control

• Reduce the effects of modeling error, input and output disturbances

• Improve dynamic response• Stabilize unstable systems

a+dm1/ayr

di do

a+dmK_

yr

di do

1( )m m i oy r r d a d d d

a

( ) ( ) 1

1 ( ) 1 ( ) 1 ( )

1 1

m mi o

m m m

i o

K a d a dy r d d

K a d K a d K a d

r d dK K

Identification of Database System

Control of Database System

• Output is delay time• Incoming flow rate fluctuates and unknown• Uncertainties in cost factor

Control

CostFactor

r

fouty

_

1/Sqfin

_

Experiments

• Implemented a controller in a real DSMS – Borealis• With bursty synthetic and real data