Outline - University of Waterlootozsu/courses/CS742/Course Notes/1...Outline !Introduction ... database DBMS Application program 1 (with data semantics) Application ... System? A distributed

Page 1

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.1

Outline n  Introduction & architectural issues

l What is a distributed DBMS l Problems l Current state-of-affairs

q Data distribution q Distributed query processing q Distributed query optimization q Distributed transactions & concurrency control q Distributed reliability q Database replication q Parallel database systems q Database integration & querying q Advanced topics


File Systems

program 1

data description 1

program 2

data description 2

program 3

data description 3

File 1

File 2

File 3

Page 2


Database Management

database

DBMS

Application program 1 (with data semantics)



description manipulation

control


Motivation

Database Technology

Computer Networks

integration distribution

integration

integration ≠ centralization

Distributed Database Systems

Page 3


Distributed Computing

n A number of autonomous processing elements (not necessarily homogeneous) that are interconnected by a computer network and that cooperate in performing their assigned tasks.

n What is being distributed? l Processing logic l Function l Data l Control


What is a Distributed Database System?

A distributed database (DDB) is a collection of multiple, logically interrelated databases distributed over a computer network.

A distributed database management system (D–DBMS) is the software that manages the DDB and provides an access mechanism that makes this distribution transparent to the users.

Distributed database system (DDBS) = DDB + D–DBMS

Page 4


What is not a DDBS?

n A timesharing computer system

n A loosely or tightly coupled multiprocessor system

n A database system which resides at one of the nodes of a network of computers - this is a centralized database on a network node


Centralized DBMS on a Network

Site 5

Site 1

Site 2

Site 3 Site 4

Communication Network

Page 5


Distributed DBMS Environment

Site 5

Site 1

Site 2

Site 3 Site 4



Implicit Assumptions

n Data stored at a number of sites à each site logically consists of a single processor.

n Processors at different sites are interconnected by a computer network à not a multiprocessor system l Parallel database systems

n Distributed database is a database, not a collection of files à data logically related as exhibited in the users’ access patterns l Relational data model

n D-DBMS is a full-fledged DBMS l Not remote file system, not a TP system

Page 6


Data Delivery Alternatives

n Delivery modes l Pull-only l Push-only l Hybrid

n Frequency l Periodic l Conditional l Ad-hoc or irregular

n Communication Methods l Unicast l One-to-many

n Note: not all combinations make sense


Distributed DBMS Promises

� Transparent management of distributed, fragmented, and replicated data

� Improved reliability/availability through distributed transactions

� Improved performance

� Easier and more economical system expansion

Ch.x/12

Page 7


Transparency

n Transparency is the separation of the higher level semantics of a system from the lower level implementation issues.

n Fundamental issue is to provide data independence

in the distributed environment

l  Network (distribution) transparency

l  Replication transparency

l  Fragmentation transparency u  horizontal fragmentation: selection u  vertical fragmentation: projection u  hybrid

Ch.x/13


Example

Page 8


Transparent Access

SELECT ENAME,SAL FROM EMP,ASG,PAY

WHERE DUR > 12

AND EMP.ENO = ASG.ENO

AND PAY.TITLE = EMP.TITLE Paris projects Paris employees Paris assignments Boston employees

Montreal projects Paris projects New York projects with budget > 200000 Montreal employees Montreal assignments

Boston


Montreal

Paris

New York

Boston projects Boston employees Boston assignments

Boston projects New York employees New York projects New York assignments

Tokyo


Distributed Database - User View

Distributed Database

Page 9


Distributed DBMS - Reality

Communication Subsystem

DBMS Software

User Application User

Query

DBMS Software

DBMS Software

DBMS Software

User Query

DBMS Software

User Query

User Application


Types of Transparency

n Data independence n Network transparency (or distribution

transparency) l Location transparency l Fragmentation transparency

n Replication transparency n Fragmentation transparency

Page 10


Reliability Through Transactions

n Replicated components and data should make distributed DBMS more reliable.

n Distributed transactions provide l Concurrency transparency l Failure atomicity

• Distributed transaction support requires implementation of l Distributed concurrency control protocols l Commit protocols

n Data replication l Great for read-intensive workloads, problematic for updates l Replication protocols


Potentially Improved Performance

n Proximity of data to its points of use

l Requires some support for fragmentation and replication

n Parallelism in execution

l  Inter-query parallelism

l  Intra-query parallelism

Page 11


Parallelism Requirements

n Have as much of the data required by each application at the site where the application executes

l Full replication

n How about updates?

l Mutual consistency

l Freshness of copies


System Expansion

n Issue is database scaling

n Emergence of microprocessor and workstation technologies

l Demise of Grosh's law

l Client-server model of computing

n Data communication cost vs telecommunication cost

Page 12


Distributed DBMS Issues

n Distributed Database Design l How to distribute the database

l Replicated & non-replicated database distribution

l A related problem in directory management

n Query Processing l Convert user transactions to data manipulation

instructions

l Optimization problem

u min{cost = data transmission + local processing}

l General formulation is NP-hard


Distributed DBMS Issues

n Concurrency Control l Synchronization of concurrent accesses

l Consistency and isolation of transactions' effects

l Deadlock management

n  Reliability l How to make the system resilient to failures

l Atomicity and durability

Page 13


Directory Management

Relationship Between Issues

Reliability

Deadlock Management

Query Processing

Concurrency Control

Distribution Design


Related Issues

n Operating System Support l Operating system with proper support for database

operations l Dichotomy between general purpose processing

requirements and database processing requirements

n Open Systems and Interoperability l Distributed Multidatabase Systems l More probable scenario l Parallel issues

Page 14


Architecture

n Defines the structure of the system l  components identified

l  functions of each component defined

l  interrelationships and interactions between components defined


ANSI/SPARC Architecture

External Schema

Conceptual Schema

Internal Schema

Internal view

Users

External view

Conceptual view

External view

External view

Page 15


Generic DBMS Architecture


DBMS Implementation Alternatives

Page 16


Dimensions of the Problem

n  Distribution l  Whether the components of the system are located on the same

machine or not

n  Heterogeneity l  Various levels (hardware, communications, operating system) l  DBMS important one

u  data model, query language,transaction management algorithms

n  Autonomy l  Not well understood and most troublesome l  Various versions

u  Design autonomy: Ability of a component DBMS to decide on issues related to its own design.

u  Communication autonomy: Ability of a component DBMS to decide whether and how to communicate with other DBMSs.

u  Execution autonomy: Ability of a component DBMS to execute local operations in any manner it wants to.


Client/Server Architecture

Page 17


Advantages of Client-Server Architectures

n More efficient division of labor

n Horizontal and vertical scaling of resources

n Better price/performance on client machines

n Ability to use familiar tools on client machines

n Client access to remote data (via standards)

n Full DBMS functionality provided to client workstations

n Overall better system price/performance


Database Server

Page 18


Distributed Database Servers


Datalogical Distributed DBMS Architecture

...

...

...

ES1 ES2 ESn

GCS

LCS1 LCS2 LCSn

LIS1 LIS2 LISn

Page 19


Peer-to-Peer Component Architecture

Database

DATA PROCESSOR USER PROCESSOR

USER

User requests

System responses

External Schema

Use

r In

terf

ace

Han

dle

r Global

Conceptual Schema

Sem

anti

c D

ata

Con

trol

ler

Glo

bal

E

xecu

tion

M

onit

or

System Log

Loc

al R

ecov

ery

Man

ager

Local Internal Schema

Ru

nti

me

Su

pp

ort

Pro

cess

or

Loc

al Q

uer

y P

roce

ssor

Local Conceptual

Schema

Glo

bal

Qu

ery

Op

tim

izer

GD/D


Datalogical Multi-DBMS Architecture

...

GCS … …

GES1

LCS2 LCSn …

… LIS2 LISn

LES11 LES1n LESn1 LESnm

GES2 GESn

LIS1

LCS1

Page 20


MDBS Components & Execution

Multi-DBMS Layer

DBMS1 DBMS3 DBMS2

Global User

Request

Local User

Request Global

Subrequest Global

Subrequest Global

Subrequest

Local User

Request


Mediator/Wrapper Architecture

Documents

Outline - University of Waterlootozsu/courses/CS742/Course Notes/1...Outline !Introduction ... database DBMS Application program 1 (with data semantics) Application ... System? A distributed