26
Introduction

Introduction

  • Upload
    melba

  • View
    36

  • Download
    0

Tags:

Embed Size (px)

DESCRIPTION

Introduction. Outline. What is database tuning What is changing The trends that impact database systems and their applications What is NOT changing The principles that underly our approach What these lectures are about. Definition. - PowerPoint PPT Presentation

Citation preview

Page 1: Introduction

Introduction

Page 2: Introduction

Outline

• What is database tuning• What is changing

The trends that impact database systems and their applications

• What is NOT changingThe principles that underly our approach

• What these lectures are about

@ Dennis Shasha and Philippe Bonnet, 2013

Page 3: Introduction

Definition

• Database tuning is the activity of making database applications run faster– Faster means higher throughput or/and lower

response time– Avoiding transactions that create bottlenecks or

avoiding queries that run for hours unnecessarily is a must

@ Dennis Shasha and Philippe Bonnet, 2013

Page 4: Introduction

Why database tuning

@ Dennis Shasha and Philippe Bonnet, 2013

• Troubleshooting:– Make managers and users happy given an

application as well as DBMS/OS/Hardware • Capacity Sizing:– Buy the right DBMS/OS/Hardware given

application requirements• Application Programming:– Code your application for performance given

DBMS/OS/Hardware

Page 5: Introduction

Why do we teach database tuning?

@ Dennis Shasha and Philippe Bonnet, 2013

Simple case study:The following query runs too slowly

select * from Rwhere R.a > 5;

What do you do?

Page 6: Introduction

Trends

• Data– Sense making and the data deluge

• Hardware– Towards dark silicon– The age of semiconductor based persistence– The importance of parallelism

@ Dennis Shasha and Philippe Bonnet, 2013

Page 7: Introduction

Data Growth #1: Volume

@ Dennis Shasha and Philippe Bonnet, 2013Source: http://permabit.com/media-center/blogs/

Page 8: Introduction

Data Growth #2: Data Complexity

@ Dennis Shasha and Philippe Bonnet, 2013Source: https://practicalanalytics.wordpress.com/2011/11/06/explaining-hadoop-to-management-whats-the-big-data-deal/

Page 9: Introduction

Sense making #1: Current Paradigm

@ Dennis Shasha and Philippe Bonnet, 2013Source - http://www.youtube.com/watch?v=2YWvmBtEymE

Questions t

o answer

Build co

nceptual model

Build a lo

gical m

odel

Build a physi

cal model

Load th

e data

(Tune)

Answer th

e questions

t

Time to Insight: Weeks to Months

Collect

the data

Page 10: Introduction

Sense making #2: Paradigm shift

@ Dennis Shasha and Philippe Bonnet, 2013Source – http://www.vldb.org/2011/files/slides/keynotes/campbell_keynote.pptx

Model Scope ofAnalysis

AvailableData

TraditionalSystem

TraditionalSystem

NewSystem

Model

Model

Model

Model

ModelAvailable

Data

Page 11: Introduction

Sense making #3: New Paradigm

@ Dennis Shasha and Philippe Bonnet, 2013

Stru

ctur

e / V

alue

Signal

Data

Information

Knowledge

KnowledgeApplication

Data

Information

Knowledge

KnowledgeApplication

t Time to Insight

Digital Shoebox

Information Production

Monitor, Mine, Manage

Transform & Load

Model Generation

Effort / LatencySource: http://www.vldb.org/pvldb/vol4/p694-campbell.pdf

Page 12: Introduction

Towards Dark Silicon

@ Dennis Shasha and Philippe Bonnet, 2013Source: http://darksilicon.org/horsemen/horsemen_slides.pdf

Page 13: Introduction

The End of Multicore Scaling

• Utilization Wall: With each successive process generation, the percentage of a chip that can actively switch drops exponentially due to power constraints.

@ Dennis Shasha and Philippe Bonnet, 2013Source: http://darksilicon.org/horsemen/horsemen_slides.pdf

65 nm 32nm

4 cores @ 1.8 GHz 4 cores @ 2x1.8 GHz (12 cores dark)

Page 14: Introduction

Hardware Acceleration

@ Dennis Shasha and Philippe Bonnet, 2013Source: http://eecatalog.com/fpga/2012/11/13/xilinx-20nm-all-programmable-portfolio-builds-on-28nm-breakthroughs-to-stay-a-generation-ahead/

Page 15: Introduction

Slotnik’s Law of Effort #1: Heterogeneous Systems

@ Dennis Shasha and Philippe Bonnet, 2013

LOOK UP: Slotnik vs. Amdahl (AFIPS’67), Michael Flynn’s talk on dataflow machines, Ryan Johnson’s paper on bionic databases.

Source: http://www.anandtech.com/show/2933

http://delivery.acm.org/10.1145/1470000/1465559/p477-slotnick.pdf?ip=130.226.133.50&acc=ACTIVE%20SERVICE&CFID=263037015&CFTOKEN=79596176&__acm__=1359029285_553ebd0653823da65a0dadf470dce6d1
http://delivery.acm.org/10.1145/1470000/1465560/p483-amdahl.pdf?ip=130.226.133.50&acc=ACTIVE%20SERVICE&CFID=263037015&CFTOKEN=79596176&__acm__=1359029801_db42a36892c317f829fc99bf0aab2359
http://www.youtube.com/watch?v=ybnOul9jNgE&list=PLB25B1353438707F9&index=10
http://pandis.net/resources/cidr13bionic.pdf
Page 16: Introduction

ReadWrite

Logi

cal a

ddre

ss sp

ace

Scheduling& Mapping

Wear Leveling

Garbage collection

ReadProgram

EraseChipChip

Chip…

ChipChip

Chip…

ChipChip

Chip…

ChipChip

Chip…

Flash memory array

Channels

Phys

ical

add

ress

spac

e

@ Dennis Shasha and Philippe Bonnet, 2013

Slotnik’s Law of Effort #2: The emergence of SSDs

Throughput for 4K read IOs from product specifications

Latency of 5000 random writes on an Intel710 SSD (10 successive passes over 250 KB with 512Brandom writes on a random formatted device).

LOOK UP: The necessary death of the block device interface

http://www.cidrdb.org/cidr2013/Papers/CIDR13_Paper89.pdf
Page 17: Introduction

Warehouse-Scale Computer

@ Dennis Shasha and Philippe Bonnet, 2013Source: http://www.morganclaypool.com/doi/abs/10.2200/S00193ED1V01Y200905CAC006

LOOK UP: Werner Voegels on virtualization.

http://queue.acm.org/detail.cfm?id=1348590
http://queue.acm.org/detail.cfm?id=1348590
http://queue.acm.org/detail.cfm?id=1348590
Page 18: Introduction

Database Appliances

@ Dennis Shasha and Philippe Bonnet, 2013Source: http://www.oracle.com/us/products/database/exadata/overview/index.html

Page 19: Introduction

Trends and Database Systems

@ Dennis Shasha and Philippe Bonnet, 2013Source: http://451research.com/

http://451research.com/
http://451research.com/
Page 20: Introduction

Trends and Database Applications

@ Dennis Shasha and Philippe Bonnet, 2013Source: https://www.facebook.com/notes/facebook-engineering/building-timeline-scaling-up-to-hold-your-life-story/10150468255628920; http://www.vldb.org/pvldb/2/vldb09-938.pdf

https://www.facebook.com/notes/facebook-engineering/building-timeline-scaling-up-to-hold-your-life-story/10150468255628920
https://www.facebook.com/notes/facebook-engineering/building-timeline-scaling-up-to-hold-your-life-story/10150468255628920
https://www.facebook.com/notes/facebook-engineering/building-timeline-scaling-up-to-hold-your-life-story/10150468255628920
http://www.vldb.org/pvldb/2/vldb09-938.pdf
http://www.vldb.org/pvldb/2/vldb09-938.pdf
http://www.vldb.org/pvldb/2/vldb09-938.pdf
Page 21: Introduction

Trends & Database Tuning• Compression is of the essence• Different classes of systems adapted to different classes of

applications• Data outgrows any well-defined model• Time to insight is impacting all applications• Energy is a key metric• Dealing with parallelism requires efforts

– RAM locality is king– Incorporating hardware acceleration– Emergence of utility computing/storage– Vertical integration removes abstraction layers

@ Dennis Shasha and Philippe Bonnet, 2013

Page 22: Introduction

Database Systems Invariants

• The power of transactions– LOOK UP: Virtues and limitations by Jim Gray,

reflections on the CAP theorem by Eric Brewer.• The primacy of data independence

– LOOK UP: System R

• The beauty of declarative queries– LOOK UP: The birth of SQL

• A success story for parallelism– LOOK UP: Parallel Database Systems

@ Dennis Shasha and Philippe Bonnet, 2013

http://research.microsoft.com/en-us/um/people/gray/papers/theTransactionConcept.pdf
http://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed
http://delivery.acm.org/10.1145/330000/320457/p97-astrahan.pdf?ip=130.226.133.50&acc=ACTIVE%20SERVICE&CFID=173886994&CFTOKEN=70370996&__acm__=1359101849_f073f5e211793c45249650d4c1995eac
http://www.mcjones.org/System_R/SQL_Reunion_95/SRC-1997-018.pdf
http://delivery.acm.org/10.1145/130000/129894/p85-dewitt.pdf?ip=130.226.133.50&acc=ACTIVE%20SERVICE&CFID=173886994&CFTOKEN=70370996&__acm__=1359101989_fa610b8da2067fd171c4ac54af39a4d2
Page 23: Introduction

Tuning Invariants

1. Think globally; fix locally2. Partitioning breaks bottlenecks3. Start-up costs are high; running costs are low4. Render unto server what is due unto server5. Be prepared for trade-off

@ Dennis Shasha and Philippe Bonnet, 2013

Page 24: Introduction

Classes of Applications/Systems

• OLAP + OLTP applications• Relational systems:

– Oracle 12g– IBM DB2 10.1– SQL Server 2012– MySQL 6 & InnoDB 5– Exadata

@ Dennis Shasha and Philippe Bonnet, 2013

Page 25: Introduction

Storage Manager

(r-store,SSD)

Storage Manager

(c-store,SSD)

Application ServerApplication Server

Application Server

Web ServerWeb ServerWeb Server

System Architecture

@ Dennis Shasha and Philippe Bonnet, 2013

Hardware

OS

DBMS

Query Processor

Storage Manager

(c-store,HDD)

Application Server

Web ServerWeb ServerWeb ServerWeb ServerWeb Server

instance

Storage Manager

(RAM)

Storage Manager

(r-store,HDD)

Page 26: Introduction

Lectures• Troubleshooting techniques• Tuning the guts• Tuning transactions• Tuning the writes• Index tuning• Schema tuning• Query tuning• Tuning the application interface• Tuning across instances• OLAP tuning• OLTP tuning

@ Dennis Shasha and Philippe Bonnet, 2013


Introduction Section I: Introduction

Introduction Section I: Introduction

Documents
Introduction to Logic Introduction

Introduction to Logic Introduction

Documents
Introduction 1. Introduction

Introduction 1. Introduction

Documents
Introduction to Transducers, Introduction to Transducers ... on... · Introduction to Transducers, Introduction to Transducers, Sensors and ActuatorsSensors and Actuators Introduction

Introduction to Transducers, Introduction to Transducers ... on... · Introduction to Transducers, Introduction to Transducers, Sensors and ActuatorsSensors and Actuators Introduction

Documents
URBAN STREAM REHABILITATION. INTRODUCTION, OBJECTIVES & IMPACTS INTRODUCTION, OBJECTIVES & IMPACTS INTRODUCTION, OBJECTIVES & IMPACTS INTRODUCTION, OBJECTIVES

URBAN STREAM REHABILITATION. INTRODUCTION, OBJECTIVES & IMPACTS INTRODUCTION, OBJECTIVES & IMPACTS INTRODUCTION, OBJECTIVES & IMPACTS INTRODUCTION, OBJECTIVES

Documents
INTRODUCTION SPECIFICATIONS INTRODUCTION

INTRODUCTION SPECIFICATIONS INTRODUCTION

Documents
General introduction 1.1 Introduction

General introduction 1.1 Introduction

Documents
Introduction and Thesis Introduction - …...1 Introduction and Thesis Introduction: The introduction relates the topic to the main point of the essay. The introduction is composed

Introduction and Thesis Introduction - …...1 Introduction and Thesis Introduction: The introduction relates the topic to the main point of the essay. The introduction is composed

Documents
Module: Introduction...Module: Introduction Page: Introduction CC0.1 Introduction Please give a general description and introduction to your organization. Providing electrical, hydraulic

Module: Introduction...Module: Introduction Page: Introduction CC0.1 Introduction Please give a general description and introduction to your organization. Providing electrical, hydraulic

Documents
INTRODUCTIONS (LEADS) Creative introduction Action introduction Dialogue introduction Personal introduction Question introduction Quotation introduction

INTRODUCTIONS (LEADS) Creative introduction Action introduction Dialogue introduction Personal introduction Question introduction Quotation introduction

Documents
INTRODUCTIONS (LEADS) Academic introduction Creative introduction Action introduction Dialogue introduction Overarching societal statements Personal introduction

INTRODUCTIONS (LEADS) Academic introduction Creative introduction Action introduction Dialogue introduction Overarching societal statements Personal introduction

Documents
 · Contents INTRODUCTION Introduction

 · Contents INTRODUCTION Introduction

Documents
Introduction - University of Cambridge · Introduction. Introduction

Introduction - University of Cambridge · Introduction. Introduction

Documents
Topics Introduction 1 / 3 Introduction 2 / 3 Introduction 3 / 3

Topics Introduction 1 / 3 Introduction 2 / 3 Introduction 3 / 3

Documents
Introduction to introduction to introduction to … Optimization

Introduction to introduction to introduction to … Optimization

Documents
Introduction to Elasticsearch and Logstash. - KIT · PDF fileOutline 1 Introduction 2 Logstash Introduction 3 Elasticsearch Introduction 4 Conclusions Introduction Logstash ElasticsearchConclusions

Introduction to Elasticsearch and Logstash. - KIT · PDF fileOutline 1 Introduction 2 Logstash Introduction 3 Elasticsearch Introduction 4 Conclusions Introduction Logstash ElasticsearchConclusions

Documents
voip-iptel-introduction-SRK1.ppt voip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-ip

voip-iptel-introduction-SRK1.ppt voip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-iptel-introduction-SRK1.pptvoip-ip

Documents
Introduction€¦ · Introduction ThisdocumentprovidesinformationabouttheIOSXEsoftwarereleasefortheCiscoNCS4201andCisco …

Introduction€¦ · Introduction ThisdocumentprovidesinformationabouttheIOSXEsoftwarereleasefortheCiscoNCS4201andCisco …

Documents
1. Introduction 1. Introduction 1. Introduction 1 ... · Introduction 1. Introduction 1. Introduction 1. Introduction 6061B0242401. The Phone Screen From the Phone screen, you can

1. Introduction 1. Introduction 1. Introduction 1 ... · Introduction 1. Introduction 1. Introduction 1. Introduction 6061B0242401. The Phone Screen From the Phone screen, you can

Documents
1. INTRODUCTION 1.1 Introduction

1. INTRODUCTION 1.1 Introduction

Documents
Introduction Introduction (ctnd.)

Introduction Introduction (ctnd.)

Documents
Introduction | Introduction | Millennium Serials · 2018. 4. 17. · MARC Display | Print | Introduction | Millennium Serials INTRODUCTION Introduction This tutorial introduces the

Introduction | Introduction | Millennium Serials · 2018. 4. 17. · MARC Display | Print | Introduction | Millennium Serials INTRODUCTION Introduction This tutorial introduces the

Documents
Systems Analysis I introduction I introduction. 2Systems Analysis - Introduction Contents Introduction Introduction 3 classic phases 3 classic phases

Systems Analysis I introduction I introduction. 2Systems Analysis - Introduction Contents Introduction Introduction 3 classic phases 3 classic phases

Documents
An Introduction for An Introduction for Introduction for ......An Introduction for An Introduction for Introduction for TeachersTeachersTeachers . exChange, Blackburn Cathedral’s

An Introduction for An Introduction for Introduction for ......An Introduction for An Introduction for Introduction for TeachersTeachersTeachers . exChange, Blackburn Cathedral’s

Documents
Introduction to Databases: Introduction

Introduction to Databases: Introduction

Documents
Introduction Introduction to Brittle Deformation

Introduction Introduction to Brittle Deformation

Documents
Introduction to Transducers, Introduction to Transducers ...engtech.weebly.com/.../more_on_transducers_sensors_actuators.pdf · Introduction to Transducers, Introduction to Transducers,

Introduction to Transducers, Introduction to Transducers ...engtech.weebly.com/.../more_on_transducers_sensors_actuators.pdf · Introduction to Transducers, Introduction to Transducers,

Documents
introduzione | introduction | einleitung | introduction · introduzione | introduction | einleitung | introduction | introducción | введе́ние | wstęp en Description Operation

introduzione | introduction | einleitung | introduction · introduzione | introduction | einleitung | introduction | introducción | введе́ние | wstęp en Description Operation

Documents
Introduction: Definition Introduction: Philosophy

Introduction: Definition Introduction: Philosophy

Documents
Introduction - daks2k3a4ib2z.cloudfront.net€¦ · Introduction Establishedin1974MarbleHillFireplacesofferoneofthelargestandmostcomprehensiveselectionsof …

Introduction - daks2k3a4ib2z.cloudfront.net€¦ · Introduction Establishedin1974MarbleHillFireplacesofferoneofthelargestandmostcomprehensiveselectionsof …

Documents