cs4432 recovery 1

CS4432: Database Systems II

Database Consistency and Violations?

cs4432 recovery 2

Transactions, etc.

• Crash recovery Chapter 17• Concurrency control Chapter 18• Transaction processing Chapter 19

cs4432 recovery 3

Integrity or correctness of data ?

• We would like data in our database to be “accurate” ( “correct” ) at all times.

EMP

• How DBMS decides if data is consistent?

Name

WhiteGreenGray

Age

523421

1

cs4432 recovery 4

Integrity or consistency constraints• Utilize predicates data must

satisfy• Examples:

- x is key of relation R- x y holds in R- Domain(x) = {Red, Blue, Green}- no employee should make more than

twice the average salary

cs4432 recovery 5

Definitions:

• Consistent state: satisfies all constraints

• Consistent DB: DB in consistent state

cs4432 recovery 6

Such constraints may not capture “full

correctness”

Example 1 : Transaction constraints• When salary is updated,

new salary > old salary• When account record is deleted,

balance = 0

cs4432 recovery 7

Example 2 Database should reflect real world

DBReality

Constraints (as we use here) may not capture “full

correctness”

cs4432 recovery 8

in any case, continue with constraints ...

Observation: DB cannot be consistent always

Example: Constraint : a1 + a2 +…. an = TOT Action:

Deposit $100 in a2: a2 a2 + 100TOT TOT + 100

cs4432 recovery 9

a2

TOT

.

.50

.

.1000

.

.150

.

.1000

.

.150

.

.1100

Example: a1 + a2 +…. an = TOT (constraint)Deposit $100 in a2: a2 a2 + 100

TOT TOT + 100

cs4432 recovery 10

Transaction: a collection of actions

that preserve consistency

Consistent DB Consistent DB’T

cs4432 recovery 11

Big assumption:

If T starts with consistent state ANDT executes in isolation

T leaves consistent state

cs4432 recovery 12

Correctness (informally)

• If we stop running transaction(s),DB left consistent

• Each transaction sees a consistent DB

cs4432 recovery 13

How can constraints be violated?

• Transaction bug• DBMS bug• Hardware failure

e.g., disk crash alters balance of account

• Data sharinge.g.: T1: give 10% raise to programmers

T2: change programmers systems analysts

cs4432 recovery 14

Will not consider:

• How to write correct transactions• How to write correct DBMS system• Constraint checking & repair

cs4432 recovery 15

How can we prevent/fix violations?

• Chapter 17: due to failures only• Chapter 18: due to data sharing only• Chapter 19: due to failures and

sharing

cs4432 recovery 16

Chapter 17: Recovery

• First : Failure Model

cs4432 recovery 17

Events Desired Undesired Expected

Unexpected

cs4432 recovery 18

Our failure model

processor

memory disk

CPU

M D

cs4432 recovery 19

Desired events: see product manuals….

Undesired expected events:System crash

- memory lost- cpu halts, resets

Undesired Unexpected: Everything else!

that’s it!!

cs4432 recovery 20

Examples:• Disk data is lost• Memory lost without CPU halt• CPU implodes wiping out

universe….• You name it …

Undesired Unexpected: Everything else!

cs4432 recovery 22

Operations reStorage Hierarchy:• Input (x): block containing x memory• Output (x): block containing x disk

• Read (x,t): do input(x) if necessary t value of x in

block• Write (x,t): do input(x) if necessary

value of x in block t

Memory Disk

x x

cs4432 recovery 23

Key problem : Unfinished transaction

Example: Constraint: A=B T1: A A 2 B B 2

cs4432 recovery 24

T1: Read (A,t); t t2Write (A,t);Read (B,t); t t2Write (B,t);Output-to-Disk (A);Output-to-Disk (B);

A: 8B: 8

A: 8B: 8

memory disk

1616

16

failure!

cs4432 recovery 25

• Need atomicity: – execute all actions of a

transaction or none at all

cs4432 recovery 26

One solution: undo logging (immediate

modification)

A la Hansel and Gretel recording their navigation through forest via bread crumbs …

Must have durable undo logging !!!

cs4432 recovery 27

T1: Read (A,t); t t2 A=B

Write (A,t);Read (B,t); t t2Write (B,t);Output-to-disk (A);Output-to-disk (B);

A:8B:8

A:8B:8

memory disk log

Undo logging (Immediate modification)

1616

<T1, start><T1, A, 8>

<T1, commit>16 <T1, B, 8>

16

cs4432 recovery 28

One “complication” : First disk, then log• Log is first written in memory• Not written to disk on every action

memory

DB

Log

A: 8 16B: 8 16Log:<T1,start><T1, A, 8><T1, B, 8>

A: 8B: 8

16BAD STATE

# 1

cs4432 recovery 29

One “complication” : first log, then disk.• Log is first written in memory• Not written to disk on every action

memory

DB

Log

A: 8 16B: 8 16Log:<T1,start><T1, A, 8><T1, B, 8><T1, commit>

A: 8B: 8

16BAD STATE

# 2

<T1, B, 8><T1, commit>

...

cs4432 recovery 30

Undo logging rules

(1)For every action generate undo log record (containing old value)

(2) Before x is modified on disk, log records pertaining to x must be

on disk (write ahead logging)

(3) Before commit is written to log on disk, all writes of transaction must be

reflected on disk

cs4432 recovery 31

Recovery rules: Undo logging

(1) Let S = set of transactions with<Ti, start> in log, but no<Ti, commit> (or <Ti, abort>) record

in log(2) For each <Ti, X, v> in log,

in reverse order (latest earliest) do: - write old value from log back to disk:

- if Ti S then - write (X, v) - output (X)

(3) For each Ti S do- write <Ti, abort> to log

cs4432 recovery 32

What if failure during recovery?No problem! Undo idempotent

cs4432 recovery 33

To discuss next :

• Redo logging• Undo/redo logging, why both?• Checkpoints

cs4432 recovery 34

Redo logging (deferred modification)

T1: Read(A,t); t t2; write (A,t); Read(B,t); t t2; write (B,t);

Output(A); Output(B)

A: 8B: 8

A: 8B: 8

memory DB LOG

1616

<T1, start><T1, A, 16><T1, B, 16>

<T1, commit>

output16

cs4432 recovery 35

Redo logging rules

(1) For every action, generate redo logrecord (containing new value)

(2) Before X is modified on disk (DB),all log records for transaction that

modified X (including commit) must be on disk

(3) Flush log at commit

cs4432 recovery 36

(1) Let S = set of transactions with<Ti, commit> in log

(2) For each <Ti, X, v> in log, in forward order (earliest latest) do:

- if Ti S then Write(X, v) Output(X)

optional

Recovery rules: Redo logging

cs4432 recovery 37

Recovery is very, very

SLOW !Redo log:

First T1 wrote A,B LastRecord Committed a year ago

Record(1 year ago) --> STILL, Need to redo after crash!!

... ... ...

Crash

cs4432 recovery 38

Solution: Checkpoint (simple version)

Periodically:(1) Do not accept new transactions(2) Wait until all transactions finish(3) Flush all log records to disk (log)(4) Flush all buffers to disk (DB) (do not discard buffers)

(5) Write “checkpoint” record on disk (log)

(6) Resume transaction processing

cs4432 recovery 39

Example: what to do at recovery?

Redo log (disk):

<T1

,A,1

6>

<T1

,com

mit

>

Ch

eck

poin

t

<T2

,B,1

7>

<T2

,com

mit

>

<T3

,C,2

1>

Crash... ... ... ...

...

...

cs4432 recovery 40

Key drawbacks:

• Undo logging: Too many disk IOs• Redo logging: need to keep all

modified blocks in memory

until commit

cs4432 recovery 41

Solution: undo/redo logging!

Combine Undo/Redo Logging, namely:

Update <Ti, Xid, New X val, Old X val>page X

cs4432 recovery 42

Rules

• Page X can be flushed before orafter Ti commit

• Log record flushed before corresponding updated page (WAL)

• Flush at commit (log only)

cs4432 recovery 43

Non-quiescent checkpoint

LOG

for undo dirty buffer

pool pagesflushed

Start-ckptactive TR:

Ti,T2,...

endckpt

.........

...

cs4432 recovery 44

Examples what to do at recovery time?

no T1 commit

LOG

T1,-a

...CkptT1

...Ckptend

...T1-b

...

Undo T1 (undo a,b)

cs4432 recovery 45

Example

LOG

...T1

a... ...

T1

b... ...

T1

c...

T1

cmt...

ckpt-end

ckpt-sT1

Redo T1: (redo b,c)

cs4432 recovery 46

Recovery process:• Backwards pass (end of log latest checkpoint

start)

– construct set S of committed transactions– undo actions of transactions not in S

• Undo pending transactions– follow undo chains for transactions in

(checkpoint active list) - S

• Forward pass (latest checkpoint start end of log)

– redo actions of S transactions

backward pass

forward passstart

check-point

cs4432 recovery 47

Summary

• Consistency of data• One source of problems: failures

- Logging- Redundancy

• Another source of problems: Data Sharing..... next