Database replication policies Database replication policies for dynamic content for dynamic content

applicationsapplications

Gokul Soundararajan, Cristiana Amza, Gokul Soundararajan, Cristiana Amza, Ashvin GoelAshvin Goel

University of TorontoUniversity of Toronto

Presented by Ahmed AtaullahPresented by Ahmed Ataullah

Wednesday, November 22Wednesday, November 22ndnd 2006 2006

22

The planThe plan

Motivation and IntroductionMotivation and Introduction BackgroundBackground Suggested TechniqueSuggested Technique Optimization/Feeback ‘loop’Optimization/Feeback ‘loop’ Summary of ResultsSummary of Results DiscussionDiscussion

33

The ProblemThe Problem

3-Tier Framework3-Tier Framework– Recall: The benefits of partial/full database Recall: The benefits of partial/full database

replication in dynamic content (web) replication in dynamic content (web) applicationsapplications

– We need to address some issues in the We need to address some issues in the framework as presented in ‘Ganymed’framework as presented in ‘Ganymed’

Problem:Problem:– How many replicas do we allocate?How many replicas do we allocate?– How do we take care of overload situations How do we take care of overload situations

while maximizing utilization and minimizing while maximizing utilization and minimizing the number of replicas?the number of replicas?

Solution:Solution:– Dynamically (de)allocate replicas as neededDynamically (de)allocate replicas as needed

44

AssumptionsAssumptions

Query Load:Query Load:– Write queries are short, sweet and simpleWrite queries are short, sweet and simple– Read queries are complex, costly and Read queries are complex, costly and

more frequentmore frequent Infrastructure:Infrastructure:

– Replica addition is time consuming and Replica addition is time consuming and ‘expensive’‘expensive’

– Machines are flexible in natureMachines are flexible in nature Replica Allocation vs. Replica MappingReplica Allocation vs. Replica Mapping

– Assume an intelligent middleware is Assume an intelligent middleware is presentpresent

55

Replica AllocationReplica Allocation

Full overlapping allocationFull overlapping allocation– All databases replicated across all All databases replicated across all

machines in the clustermachines in the cluster Disjoint allocation (no overlapping)Disjoint allocation (no overlapping)

(A two database Scenario: Global replica pool not shown)

RS = Read Sets, WS=Write Sets : Ratio of WS to RS may be misleading in the above diagram

A B

66

Partial Overlapping Partial Overlapping AllocationAllocation

Only share write sets. Read sets do not overlapOnly share write sets. Read sets do not overlap

A B

(A two database scenario)

RS = Read Sets, WS=Write Sets

77

Dynamic Replication Dynamic Replication (Eurosys 2006 Slides)(Eurosys 2006 Slides)

Assume a cluster hosts 2 applicationsAssume a cluster hosts 2 applications– App1 (Red) using 2 machinesApp1 (Red) using 2 machines– App2 (Blue) using 2 machinesApp2 (Blue) using 2 machines

Assume App1 has a load spikeAssume App1 has a load spike

1 2 3 4

Replica SetApp1 = {1,2} Replica SetApp2 = {3,4}

88

Dynamic ReplicationDynamic Replication

Choose # of replicas to allocate to App1Choose # of replicas to allocate to App1– Say, we adapt by allocating one more replicaSay, we adapt by allocating one more replica

Then, two optionsThen, two options– App2 still uses two replicas (overlap replica sets)App2 still uses two replicas (overlap replica sets)– App2 loses one replica (disjoint replica sets)App2 loses one replica (disjoint replica sets)

1 2 3 4

Replica SetApp1 = {1,2} Replica SetApp2 = {3,4}

99

Dynamic ReplicationDynamic Replication

Choose # of replicas to allocate to App1Choose # of replicas to allocate to App1– Say, we adapt by allocating one more replicaSay, we adapt by allocating one more replica

Then, two optionsThen, two options– App2 still uses two replicas (overlap replica sets)App2 still uses two replicas (overlap replica sets)– App2 loses one replica (disjoint replica sets)App2 loses one replica (disjoint replica sets)

1 2 4

Replica SetApp1 = {1,2,3} Replica SetApp2 = {3,4}

3

1010

Dynamic ReplicationDynamic Replication

Choose # of replicas to allocate to App1Choose # of replicas to allocate to App1– Say, we adapt by allocating one more replicaSay, we adapt by allocating one more replica

Then, two optionsThen, two options– App2 still uses two replicas (overlap replica sets)App2 still uses two replicas (overlap replica sets)– App2 loses one replica (disjoint replica sets)App2 loses one replica (disjoint replica sets)

1 2 3 4

Replica SetApp1 = {1,2,3} Replica SetApp2 = {4}

1111

ChallengesChallenges

Adding a replica can take timeAdding a replica can take time– Bring replica up-to-dateBring replica up-to-date– Warm-up memoryWarm-up memory

Can avoid adaptation with fully-overlapped Can avoid adaptation with fully-overlapped replica setsreplica sets

Disjoint Full OverlapSLOW Adaptation

NO Adaptation

1212

ChallengesChallenges

However, overlapping applications However, overlapping applications compete for memory causing interferencecompete for memory causing interference

Can avoid interference with disjoint Can avoid interference with disjoint replica setsreplica sets

Disjoint Full OverlapNOInterference

HIGH Interference

1313

ChallengesChallenges

However, overlapping applications However, overlapping applications compete for memory causing interferencecompete for memory causing interference

Can avoid interference with disjoint Can avoid interference with disjoint replica setsreplica sets

Disjoint Full OverlapNOInterference

HIGH Interference

Tradeoff between adaptation delay and interference

1414

Our Solution – Partial Our Solution – Partial OverlapOverlap

Reads of applications sent to disjoint replica Reads of applications sent to disjoint replica setssets– Avoids interferenceAvoids interference

Read-SetRead-Set– Set of replicas where reads are sentSet of replicas where reads are sent

1 2 3 4

Read-SetApp1 = {1,2} Read-SetApp2 = {3,4}

1515

Our Solution – Partial Our Solution – Partial OverlapOverlap

Writes of apps also sent to overlapping Writes of apps also sent to overlapping replica setsreplica sets– Reduces replica addition timeReduces replica addition time

Write-SetWrite-Set– Set of replicas where writes are sentSet of replicas where writes are sent

1 2 3 4

Read-SetApp1 = {1,2}Write-SetApp1 = {1,2,3}

Read-SetApp2 = {3,4}Write-SetApp2 = {2,3,4}

1616

OptimizationOptimization

For a given application,For a given application,– Replicas in Write-Set – Fully Up-to-DateReplicas in Write-Set – Fully Up-to-Date– Other Replicas – Periodic Batch UpdatesOther Replicas – Periodic Batch Updates

1 2 3

Fully Up-to-date

4PeriodicUpdates

1717

Secondary Secondary Implementation Implementation

DetailsDetails Scheduler(s):Scheduler(s):

– Separate read-only from read/write Separate read-only from read/write queriesqueries

– One copy serializability is guaranteedOne copy serializability is guaranteed Optimization:Optimization:

– Scheduler also stores some cached Scheduler also stores some cached information (queries, write sets etc,) information (queries, write sets etc,) to reduce warm-up/ready time.to reduce warm-up/ready time.

– Conflict awareness at the scheduler Conflict awareness at the scheduler layerlayer

1818

Replica Allocation Replica Allocation LogicLogic

Measure Average Query Latency by solving: WL= (alpha) L + (1 – alpha) WL

L is the current query latency and alpha a constant. Note: Responsiveness/stability both depend on alpha

Stability

Delay

?

1919

ResultsResults

It works…

2020

One last interesting One last interesting issueissue WL= (alpha) L + (1 – alpha) WL

– L is the current query latency and alpha a constant

2121

DiscussionDiscussion Questionable AssumptionsQuestionable Assumptions

– Are write requests really (always) simple?Are write requests really (always) simple?– Scalability beyond 60 replicas (is it an issue?)Scalability beyond 60 replicas (is it an issue?)

How closely does this represent a real data center How closely does this represent a real data center situation?situation?– Load contention issuesLoad contention issues– Overlap assignmentOverlap assignment– Determination of Determination of alpha(s)alpha(s)

Actual cost savings vs. Implied cost savingsActual cost savings vs. Implied cost savings– Depends on SLA etc.Depends on SLA etc.– Depends on hardware leasing agreementsDepends on hardware leasing agreements

The issue of readiness in secondary replicasThe issue of readiness in secondary replicas– What level of ‘warmth’ is good enough for each What level of ‘warmth’ is good enough for each

application. Can some machines be turned off?application. Can some machines be turned off?– What about contention in many databases trying to stay What about contention in many databases trying to stay

warm.warm. Management concernsManagement concerns

– Can we truly provide strong guarantees for keeping our Can we truly provide strong guarantees for keeping our end of the SLA promised?end of the SLA promised?