© Prof. Dr.-Ing. Wolfgang Lehner |

Processing and Optimization of Forecast Queries

Ulrike Fischer

© Ulrike Fischer | | 2

> Motivation

Time series data appears in many domains

Processing and Optimization of Forecast Queries

High accuracy possible Sophisticated models Sophisticated estimators

Runtime restrictions Large number of time series Short amount of time available

Two Optimization Dimensions: Accuracy and Runtime

Renewable energy ressourcesSales and inventory

© Ulrike Fischer | | 3

> Outline

Motivation

Integration of Forecasting inside a DBMS

Processing of Forecast Queries

Optimization of Forecast Queries in Hierarchies

Summary

Processing and Optimization of Forecast Queries

© Ulrike Fischer | | 4

>

1. Model Creation Model Identification Parameter Estimation

Model-based Time Series Forecasting

3. Model Maintenance Model Evaluation

Threshold-based, time-based … Model Adaption

Parameter Re-estimation

Processing and Optimization of Forecast Queries

!

!

2. Model Usage

Forecasting Model Triple Exponential Smoothing

© Ulrike Fischer | | 5

> Time Series Forecasting in DBMS

Processing and Optimization of Forecast Queries

exportexportM

M

MM

M

SQL

SQL Reuse of models and

results

Transparency and Effienciency

© Ulrike Fischer | | 6

> Project Overview

Processing and Optimization of Forecast Queries

FlexOffers

Scheduling

Forecasting Aggregation

Supply Demand

SELECT date, quantityFROM salesWHERE … FORECAST …

DWH

date quantity2012 34,0002013 38,000

… …

EU FP7 project

© Ulrike Fischer | | 7

> Overview F2DB

Processing and Optimization of Forecast Queries

Model Index

Base Tables

Model Pool

Time Series Time Series Time Series

ModelModel

ModelModel

Query Interface

Forecast Queries Inserts

Query Processing & Optimization

QP in Hierarchies

Publish Subscribe Queries

Model Usage

Hybrid Maintenance

On-Demand Estimation

Model Maintenance

Ensemble Models

Physical Design

Model Creation

© Ulrike Fischer | | 8

> Outline

Motivation

Integration of Forecasting inside a DBMS

Processing of Forecast Queries

Optimization of Forecast Queries in Hierarchies

Summary

Processing and Optimization of Forecast Queries

© Ulrike Fischer | | 9

> Forecast Query Processing

Extension of SQL language Horizon, measure and time column,

model type and parameters, …

Logical query plan Forecast operator Ψ

Processing and Optimization of Forecast Queries

SELECT date, SUM(quantity)FROM salesWHERE product = ‘HTC‘GROUP BY dateFORECAST 3

Physical query plan

σ product= 'HTC'

sales

Ψk=3

πdate, quantity

γdate:AGG(sales)

Scan

Aggregate

BuildModel

Forecast

sales

Forecast

MHTC

© Ulrike Fischer | | 10

> Advanced Forecast Query Processing

Data warehouse contains multidimensional data

Processing and Optimization of Forecast Queries

HTC

HD2

Mobiles

Smart

Nokia

2. Aggregation

3. Disaggregation

Forecast

MMobiles

DisAgg

KeyForecast

MHD2

Forecast

MSmart

Aggregation

SELECT date, SUM(quantity)FROM salesWHERE product = ‘HTC‘GROUP BY dateFORECAST 3 days

1. Direct

© Ulrike Fischer | | 11

> Aggregation vs. Disaggregation

Processing and Optimization of Forecast Queries

Complete(Direct)

Bottom-Up(Aggregation)

Top-Down(Disaggregation)

Efficiency

Accuracy No information lossModel creation easier

Grunfeld and Griliches (1960)Gross and Sohl (1990)

Zellner and Tobias (2000)….

Edwards and Orcuss (1969)Schwarzkopf et. al. (1988)

Hubrich (2005)…

Depends on data set, quality of forecast model, correlation …

© Ulrike Fischer | | 12

> Outline

Motivation

Integration of Forecasting inside a DBMS

Processing of Forecast Queries

Optimization of Forecast Queries in Hierarchies

Summary

Processing and Optimization of Forecast Queries

© Ulrike Fischer | | 13

> Configuration Advisor

Processing and Optimization of Forecast Queries

Problem: Exponential search space Greedy Algorithm (monotonic maintenance costs)

Start one model at the top, add models step-by-step

DWH Model Pool

Query Interface

Updates Forecast Queries

Model Advisor

Workload W Preference α

Analyze

Cost BW + Error EW

Configuration + Strategy

CreateConfiguration CW

© Ulrike Fischer | | 14

> Performance Comparison

Processing and Optimization of Forecast Queries

Complete(C) All models, only direct forecasts Bottom-Up (B) Only models at level one, others use aggregation Top-Down (T) Only one model for top element, others use

disaggregation Greedy (G)

© Ulrike Fischer | | 15

> Extensions

Observation: aggregation (bottom-up) hardly used in real data sets Reason: large number of child time series

Processing and Optimization of Forecast Queries

? ?VirtualGroup

Sample Aggregation Use sample of child models

aggregation + estimation

Group Design Relax fixed aggregation groups

?

Estimate using historical proportion Weighted sampling

support of disjunctive queries

© Ulrike Fischer | | 16

> Outline

Motivation

Integration of Forecasting inside a DBMS

Processing of Forecast Queries

Optimization of Forecast Queries in Hierarchies

Summary

Processing and Optimization of Forecast Queries

© Ulrike Fischer | | 17

> Summary

DBMS Integration Sophisticated models computationally expensive DBMS integration for reuse, transparency and optimization

Forecast Queries New query type with forecast horizon Face two otimization dimensions

Hierarchical Forecasting Reduce maintenance costs with derivation schemes Possible increase of accuracy Large search space

Processing and Optimization of Forecast Queries

© Prof. Dr.-Ing. Wolfgang Lehner |

Processing and Optimization of Forecast Queries

Ulrike Fischer