Upload
doanngoc
View
230
Download
0
Embed Size (px)
Citation preview
Generating Shifting Workloads to Benchmark Adaptability inRelational Database Systems
Tilmann Rabl, Andreas Lang, Thomas Hackl,Bernhard Sick, Harald Kosch
Faculty of Computer ScienceUniversity of Passau
TPC Technology Conference on Performance Evaluation & BenchmarkingAugust 24, 2009
Introduction Benchmark Design Workload Generation Benchmarking Objectives Conclusion
Introduction
Hot database research topics:
Adaptability
Automatic and autonomic tuning
How can they be benchmarked?
Today’s solution
Starting from scratch
Changing the workload completely
2 / 12
Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems
Introduction Benchmark Design Workload Generation Benchmarking Objectives Conclusion
Introduction
Hot database research topics:
Adaptability
Automatic and autonomic tuning
How can they be benchmarked?
Today’s solution
Starting from scratch
Changing the workload completely
2 / 12
Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems
Introduction Benchmark Design Workload Generation Benchmarking Objectives Conclusion
Motivation
How does real-world workload look like?Homogeneous workload:
Daily and weekly patterns
3 / 12
Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems
Introduction Benchmark Design Workload Generation Benchmarking Objectives Conclusion
Motivation
How does real-world workload look like?Special purpose workload:
November December January February11th March 16th April May June21st0
0.5
1
1.5
2
2.5
3
3.5
4x 10
4
Dai
ly A
cces
ses
Daily Website Accesses 24 October 08 − 11 June 09
pluginsexternseminare_mainmeine_seminaredetails
Daily and weekly patterns
Workload classes: working days, holidays, outliers
Trends
3 / 12
Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems
Introduction Benchmark Design Workload Generation Benchmarking Objectives Conclusion
Benchmark Design Challenges
Design Challenges
Realistic data & workload patterns
Scalable
Configurable
Random generators
Basis
Stud.IP eLearning management system
Online application
1 year web server log data
Complete database dump
4 / 12
Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems
Introduction Benchmark Design Workload Generation Benchmarking Objectives Conclusion
Benchmark Design
25 tables (original 200)
30 queries
Only course management
Most important:seminar - seminar user - user
user_idpasswordusernamevornamenachnameemailpermission
users
seminar_idnameweekdaystarttimeendtimestartdateenddateects
seminar
seminar_iduser_idstudiengang_id
seminar_user
course_idseminar_idweekdaystarttimeendtimeroommax_participants
courses
user_idseminar_idcourse_idteam_id
courses_user
course_iduser_id
course_lecturer
studiengang_idname
studiengaengeuser_idstudiengang_id
user_studiengang user_idaddresshobbyphonecellphonepicture_path
user_info
object_iduser_idcontenttimestampvalid_until
objectsuser_idobject_idtimestamp
object_user_visit
institute_idnameaddress
institute
seminar_idinstitute_id
seminar_institute
topic_idseminar_iduser_idroot_idparent_idtimestamplast_editedcontent
topics
seminar_idsem_hierarchy_id
seminar_sem_hierarchy
sem_hierarchy_idparent_idnamepo_idpo_version
sem_hierarchy
dokument_iduser_idseminar_idfilepathtimestampvalid_untilclick_counter
dokumente
folder_idname
folder
permission_iditem_idrange_id
permissions
plugin_idpluginpathpluginnameplugindescenabled
plugins
team_idcourse_idpasswordname
teams
source_iddestination_idtype
eigeneDateien_links
message_idtimestampsender_idreceiver_idsubjectcontent
messages
user_idmessage_id
inbox
user_idmessage_id
outbox
5 / 12
Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems
Introduction Benchmark Design Workload Generation Benchmarking Objectives Conclusion
Data Generation
Based on original data
Maximum likelihood estimation
Standard probability distributions
Scalable
020
4060
80100
020
4060
80100
0
20
40
60
80
100
#Users per Seminar
Reference distribution in table seminar_user
#Seminars per User
#Ent
ries
10 20 30 40 50 60 70 80 90 1000
500
1000
1500
2000
2500
3000
3500
4000
4500Reference Distribution of Users per Seminar in Table seminar_user
Users per Seminar
Ent
ries
User References
Log−Normal Distribution
10 20 30 40 50 60 70 80 90 1000
500
1000
1500
2000
2500
3000
3500
4000
4500Reference Distribution of Seminars per User in Table seminar_user
Seminars per User
Ent
ries
Seminar ReferencesLog−Normal DistributionGamma Distribution
6 / 12
Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems
Introduction Benchmark Design Workload Generation Benchmarking Objectives Conclusion
Workload Modeling
Workload interpreted as time series
Classes of queries (meine seminare)
Classes of days (Monday during lecture period)
Daily approximation by polynomials
0 1 2 3 4 50
2000
4000
6000
8000
10000
12000
Weeks
Pag
e H
its
June 1 − July 6, 2008
pluginsexternseminar_mainmeine_seminare
7 / 12
Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems
Introduction Benchmark Design Workload Generation Benchmarking Objectives Conclusion
Polynomial Approximation
Optimal approximating polynomial of degree K :
pa(x) =K∑
k=0
akpk(x)
Properties of pk :
different ascending degrees
leading coefficient one
pair wise orthogonal (inner product)
6 8 10 12 14 16 18 20 22 0 2 40
200
400
600
800
1000
1200Most Likely Approximating Polynomial
Hours
Pag
e H
its
Most Likely PolynomialAverage MondayMonday 17−11−08Monday 12−01−09
8 / 12
Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems
Introduction Benchmark Design Workload Generation Benchmarking Objectives Conclusion
Workload Generation
Unchanging Behavior
Evaluate polynomial at certain points in time
Random Generation
Time series as point in shape space
Weighting factors ak are normally distributed
Multivariate Gaussian to generate weighting vector a
−5 0 5 10 15 20 25−1
−0.5
0
0.5
1
1.5
2Monomial Coefficients
x0
x1
0 200 400 600 800 1000 1200−40
−30
−20
−10
0Orthogonial Coefficients
average
slop
e
9 / 12
Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems
Introduction Benchmark Design Workload Generation Benchmarking Objectives Conclusion
Benchmarking Objectives
Basic Performance
Shifting workloadsMeasure peak performance
Adaptivity
Different daily patternsMeasure adaptability
Robustness
Introduction of outliersMeasure over adaptation
Energy and Space Efficiency
Shifting workloadsMeasure energy / spaceefficiency
November December January February11th March 16th April May June21st0
0.5
1
1.5
2
2.5
3
3.5
4x 10
4
Dai
ly A
cces
ses
Daily Website Accesses 24 October 08 − 11 June 09
pluginsexternseminare_mainmeine_seminaredetails
10 / 12
Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems
Introduction Benchmark Design Workload Generation Benchmarking Objectives Conclusion
Conclusion
Conclusion
Online eLearning management benchmark
New generator model for query workloads
More realistic benchmarking for automatic / autonomic tuning
Future Work
Tune and test
Apply techniques to standard benchmarks
Trends in workloads
Schema evolution
11 / 12
Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems
Introduction Benchmark Design Workload Generation Benchmarking Objectives Conclusion
Conclusion
Conclusion
Online eLearning management benchmark
New generator model for query workloads
More realistic benchmarking for automatic / autonomic tuning
Future Work
Tune and test
Apply techniques to standard benchmarks
Trends in workloads
Schema evolution
11 / 12
Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems