Lectures 5 & 7: Design Theory Lectures 5 & 7. Announcements Homework #1 due today! Homework was not easy You learned a new, declarative way of programming!

Lectures 5 & 7:Design Theory

Lectures 5 & 7

Announcements

Homework #1 due today! Homework was not easy• You learned a new, declarative way of programming!

• Hope: you got the concept, so you can pick up a book on SQL whenever you need.

• Some issues with iPython+SQLite versions. Ugh!• You were great about this issues! Constructive questions and feedback! Thanks!• You will never need iPython to submit homework • Setting Expectations: cf. simply text assignments with no expected output.

• Searching on piazza is problem, we’ll try to do a better job aggregating posts…• Lots of stuff was there, but hard to find!• Thank you to those who aggregated posts! (Candy?)

Lecture 5

Announcements #2

Office hours last night was filled with nice people!• Queue management had a hiccup (our mistake?) Resulted in “fake” long

queue times... Don’t be scared away!

• We need a better way to group and identify problems (Luke is on it! Thanks!)

• We will be better about triaging issues to get help and make groups quickly.

• Thanks to CAs who came in for extra time!

Lecture 5

Announcements #3

• How to use activity time: Review slides, ask questions, or do the activity!• We designed these to cope with different backgrounds and pace requirements.

If you don’t get the activity in class YOU ARE DOING FINE!

• Thank you! We appreciate the “thank you”s for new material!• Be aggressive about giving feedback, we want you to have best class possible!• There will be hiccups in new material, please start early. • We’re trying to make class more fun and `teach you more material!!!

• Guest lecture on Thursday from GOOGLE. HAVE FUN!

Lecture 5

Lecture 5:Design Theory I

Lecture 5

6

Today’s Lecture

1. Normal forms & functional dependencies• ACTIVITY: Finding FDs

2. Finding functional dependencies• ACTIVITY: Compute the closures

3. Closures, superkeys & keys• ACTIVITY: The key or a key?

Lecture 5

7

1. Normal forms & functional dependencies

Lecture 5 > Section 1

8

What you will learn about in this section1. Overview of design theory & normal forms

2. Data anomalies & constraints

3. Functional dependencies

4. ACTIVITY: Finding FDs


Design Theory

• Design theory is about how to represent your data to avoid anomalies.

• It is a mostly mechanical process• Tools can carry out routine portions

• We have a notebook implementing all algorithms!• We’ll play with it in the activities!

Lecture 5 > Section 1 > Overview

Normal Forms• 1st Normal Form (1NF) = All tables are flat

• 2nd Normal Form = disused

• Boyce-Codd Normal Form (BCNF)

• 3rd Normal Form (3NF)

• 4th and 5th Normal Forms = see text books

DB designs based on functional dependencies, intended to prevent data anomalies

Our focus in this lecture + next one


1st Normal Form (1NF)

Student CoursesMary {CS145,CS229}Joe {CS145,CS106}… …

Violates 1NF.

1NF Constraint: Types must be atomic!

Student CoursesMary CS145Mary CS229Joe CS145Joe CS106

In 1st NF


Data Anomalies & Constraints

Lecture 5 > Section 1 > Data anomalies & constraints

Constraints Prevent (some) Anomalies in the Data

Student Course RoomMary CS145 B01Joe CS145 B01Sam CS145 B01.. .. ..

If every course is in only one room, contains redundant information!

A poorly designed database causes anomalies:



Student Course RoomMary CS145 B01Joe CS145 C12Sam CS145 B01.. .. ..

If we update the room number for one tuple, we get inconsistent data = an update anomaly




Student Course Room.. .. ..

If everyone drops the class, we lose what room the class is in! = a

delete anomaly





Similarly, we can’t reserve a room without students = an insert anomaly


… CS229 C12



Student CourseMary CS145Joe CS145Sam CS145.. ..

Course RoomCS145 B01CS229 C12

Today: develop theory to understand why this design may be better and how to find

this decomposition…

Is this form better?

• Redundancy? • Update

anomaly? • Delete

anomaly?• Insert anomaly?


Functional Dependencies

Lecture 5 > Section 1 > Functional dependencies

Functional Dependency

A->B means that “whenever two tuples agree on A then they

agree on B.”

Def: Let A,B be sets of attributesWe write A B or say A functionally determines B if, for any tuples t1 and t2:

t1[A] = t2[A] implies t1[B] = t2[B]

and we call A B a functional dependency


A Picture Of FDs

A1 … Am B1 … Bn

Defn (again):Given attribute sets A={A1,…,Am} and B = {B1,…Bn} in R,


A1 … Am B1 … Bn

A Picture Of FDs

ti

tj

Defn (again):Given attribute sets A={A1,…,Am} and B = {B1,…Bn} in R,

The functional dependency A B on R holds if for any ti,tj in R:


A Picture Of FDsDefn (again):Given attribute sets A={A1,…,Am} and B = {B1,…Bn} in R,


ti[A1] = tj[A1] AND ti[A2]=tj[A2] AND … AND ti[Am] = tj[Am]

A1 … Am B1 … Bn

ti

tj

If t1,t2 agree here..


A Picture Of FDsDefn (again):Given attribute sets A={A1,…,Am} and B = {B1,…Bn} in R,


if ti[A1] = tj[A1] AND ti[A2]=tj[A2] AND … AND ti[Am] = tj[Am]

then ti[B1] = tj[B1] AND ti[B2]=tj[B2] AND … AND ti[Bn] = tj[Bn]

A1 … Am B1 … Bn

ti

tj

If t1,t2 agree here.. …they also agree here!


FDs for Relational Schema Design

• High-level idea: why do we care about FDs?

1. Start with some relational schema

2. Find out its functional dependencies (FDs)

3. Use these to design a better schema1. One which minimizes the possibility of anomalies


Functional Dependencies as Constraints


Note: The FD {Course} -> {Room} holds on this instance

A functional dependency is a form of constraint

• Holds on some instances not others.

• Part of the schema, helps define a valid instance.


Recall: an instance of a schema is a multiset of tuples conforming to that schema, i.e. a table

Functional Dependencies as Constraints


However, cannot prove that the FD {Course} -> {Room} is part of the schema

Note that:• You can check if an FD is

violated by examining a single instance;

• However, you cannot prove that an FD is part of the schema by examining a single instance. • This would require checking

every valid instance


27

More ExamplesAn FD is a constraint which holds, or does not hold on an instance:

EmpID Name Phone Position

E0045 Smith 1234 Clerk

E3542 Mike 9876 Salesrep

E1111 Smith 9876 Salesrep

E9999 Mary 1234 Lawyer


28

{Position} {Phone}






More Examples


29

EmpID Name Phone PositionE0045 Smith 1234 ClerkE3542 Mike 9876 SalesrepE1111 Smith 9876 SalesrepE9999 Mary 1234 Lawyer

but not {Phone} {Position}

More Examples


30

ACTIVITY

Lecture 5 > Section 1 > ACTIVITY

A B C D E

1 2 4 3 6

3 2 5 1 8

1 4 4 5 7

1 2 4 3 6

3 2 5 1 8

Find at least three FDs which hold on this instance:{ } { }{ } { }{ } { }

31

2. Finding functional dependencies


32

What you will learn about in this section1. “Good” vs. “Bad” FDs: Intuition

2. Finding FDs

3. Closures

4. ACTIVITY: Compute the closures


33

“Good” vs. “Bad” FDs

We can start to develop a notion of good vs. bad FDs:






Intuitively:

EmpID -> Name, Phone, Position is “good FD”• Minimal

redundancy, less possibility of anomalies

Lecture 5 > Section 2 > Good vs. Bad FDs

34


We can start to develop a notion of good vs. bad FDs:






Intuitively:

EmpID -> Name, Phone, Position is “good FD”

But Position -> Phone is a “bad FD”• Redundancy!

Possibility of data anomalies



Given a set of FDs (from user) our goal is to:1. Find all FDs, and 2. Eliminate the “Bad Ones".

Returning to our original example… can you see how the “bad FD” {Course} -> {Room} could lead to an:• Update Anomaly• Insert Anomaly• Delete Anomaly• …



FDs for Relational Schema Design

• High-level idea: why do we care about FDs?

1. Start with some relational schema

2. Find out its functional dependencies (FDs)

3. Use these to design a better schema1. One which minimizes possibility of anomalies

Lecture 5 > Section 2 > Finding FDs

This part can be tricky!

Finding Functional Dependencies

• There can be a very large number of FDs…• How to find them all efficiently?

• We can’t necessarily show that any FD will hold on all instances…• How to do this?

We will start with this problem:Given a set of FDs, F, what other FDs must hold?


Equivalent to asking: Given a set of FDs, F = {f1,…fn}, does an FD g hold?

Inference problem: How do we decide?




1. {Name} {Color}2. {Category} {Department}3. {Color, Category} {Price}

Name Color Category Dep PriceGizmo Green Gadget Toys 49Widget Black Gadget Toys 59Gizmo Green Whatsit Garden 99

Which / how many other FDs do?!?

Provided FDs:Products

Given the provided FDs, we can see that {Name, Category} {Price} must also hold on any instance…

Example:


Equivalent to asking: Given a set of FDs, F = {f1,…fn}, does an FD g hold?

Inference problem: How do we decide?

Answer: Three simple rules called Armstrong’s Rules.

1. Split/Combine,2. Reduction, and3. Transitivity… ideas by picture



1. Split/Combine

A1 … Am B1 … Bn

A1, …, Am B1,…,Bn


1. Split/Combine

A1 … Am B1 … Bn

A1, …, Am B1,…,Bn

… is equivalent to the following n FDs…

A1,…,Am Bi for i=1,…,n


1. Split/Combine

A1 … Am B1 … Bn

A1, …, Am B1,…,Bn

… is equivalent to …

And vice-versa, A1,…,Am Bi for i=1,…,n


Reduction/Trivial A1 … Am

A1,…,Am Aj for any j=1,…,m


3. Transitive Closure

A1 … Am B1 … Bn C1 … Ck

A1, …, Am B1,…,Bn andB1,…,Bn C1,…,Ck


3. Transitive Closure

A1 … Am B1 … Bn C1 … Ck

A1, …, Am B1,…,Bn andB1,…,Bn C1,…,Ck

impliesA1,…,Am C1,…,Ck



1. {Name} {Color}2. {Category} {Department}3. {Color, Category} {Price}

Name Color Category Dep PriceGizmo Green Gadget Toys 49Widget Black Gadget Toys 59Gizmo Green Whatsit Garden 99

Which / how many other FDs hold?

Provided FDs:Products

Example:



1. {Name} {Color}2. {Category} {Dept.}3. {Color, Category} {Price}

Which / how many other FDs hold?

Provided FDs:Inferred FDs:

Example:

Inferred FD Rule used

4. {Name, Category} -> {Name} ?5. {Name, Category} -> {Color} ?6. {Name, Category} -> {Category} ?7. {Name, Category -> {Color, Category} ?8. {Name, Category} -> {Price} ?



1. {Name} {Color}2. {Category} {Dept.}3. {Color, Category} {Price}

Can we find an algorithmic way to do this?

Provided FDs:Inferred FDs:

Example:

Inferred FD Rule used

4. {Name, Category} -> {Name} Trivial5. {Name, Category} -> {Color} Transitive (4 -> 1)6. {Name, Category} -> {Category} Trivial7. {Name, Category -> {Color, Category} Split/combine (5 + 6)8. {Name, Category} -> {Price} Transitive (7 -> 3)


Closures

Lecture 5 > Section 2 > Closures

51

Closure of a set of Attributes

Given a set of attributes A1, …, An and a set of FDs F:Then the closure, {A1, …, An}+ is the set of attributes B s.t. {A1, …, An} B

{name} {color}{category} {department}{color, category} {price}

Example: F =

Example Closures:

{name}+ = {name, color}{name, category}+ = {name, category, color, dept, price}{color}+ = {color}


52

Closure Algorithm

Start with X = {A1, …, An} and set of

FDs F.

Repeat until X doesn’t change; do:

if {B1, …, Bn} C is entailed by F

and {B1, …, Bn} X

then add C to X.

Return X as X+


53

Closure AlgorithmStart with X = {A1, …, An}, FDs F.Repeat until X doesn’t change; do: if {B1, …, Bn} C is in F and {B1, …, Bn} X: then add C to X.Return X as X+

{name} {color}

{category} {dept}

{color, category} {price}

F =

{name, category}+ = {name, category}


54


{name} {color}

{category} {dept}


F =


{name, category}+ = {name, category, color}


55


{name} {color}

{category} {dept}


F =



{name, category}+ = {name, category, color, dept}


56

Closure AlgorithmStart with X = {A1, …, An}, FDs F.Repeat until X doesn’t change; do: if {B1, …, Bn} C is in F and {B1, …, Bn} X: then add C to X.Return X as X+F =


{name, category}+ = {name, category, color, dept, price}


{name, category}+ = {name, category, color, dept}{name} {color}

{category} {dept}



57

Example

Compute {A,B}+ = {A, B, }

Compute {A, F}+ = {A, F, }

R(A,B,C,D,E,F) {A,B} {C}{A,D} {E}{B} {D}{A,F} {B}


58

Example

Compute {A,B}+ = {A, B, C, D }

Compute {A, F}+ = {A, F, B }



59

Example

Compute {A,B}+ = {A, B, C, D, E}

Compute {A, F}+ = {A, B, C, D, E, F}



60

Activity-5-2.ipynb


61

3. Closures, Superkeys & Keys


62

What you will learn about in this section1. Closures Pt. II

2. Superkeys & Keys

3. ACTIVITY: The key or a key?


63

Why Do We Need the Closure?

• With closure we can find all FD’s easily

• To check if X ® A

1. Compute X+

2. Check if A Î X+

Note here that X is a set of attributes, but A is a single attribute. Why does considering FDs of this form suffice?Recall the Split/combine rule:X A1, …, X An

impliesX {A1, …, An}

Lecture 5 > Section 3 > Closures Pt. II

64

Using Closure to Infer ALL FDs{A,B} C{A,D} B{B} D

Example:Given F =Step 1: Compute X+, for every set of attributes X:

{A}+ = {A}{B}+ = {B,D}{C}+ = {C}{D}+ = {D}{A,B}+ = {A,B,C,D}{A,C}+ = {A,C}{A,D}+ = {A,B,C,D}{A,B,C}+ = {A,B,D}+ = {A,C,D}+ = {A,B,C,D} {B,C,D}+ = {B,C,D}{A,B,C,D}+ = {A,B,C,D}

No need to compute these- why?


65


Example:Given F =Step 1: Compute X+, for every set of attributes X:

{A}+ = {A}, {B}+ = {B,D}, {C}+ = {C}, {D}+ = {D}, {A,B}+ = {A,B,C,D}, {A,C}+ = {A,C}, {A,D}+ = {A,B,C,D}, {A,B,C}+ = {A,B,D}+ = {A,C,D}+ = {A,B,C,D}, {B,C,D}+ = {B,C,D}, {A,B,C,D}+ = {A,B,C,D}

Step 2: Enumerate all FDs X Y, s.t. Y X+ and X Y = :

{A,B} {C,D}, {A,D} {B,C},{A,B,C} {D}, {A,B,D} {C},{A,C,D} {B}


66


Example:Given F =




“Y is in the closure of X”


Step 1: Compute X+, for every set of attributes X:

67


Example:Given F =




The FD X Y is non-trivial


Step 1: Compute X+, for every set of attributes X:

Superkeys and Keys

Lecture 5 > Section 3 > Superkeys & Keys

Keys and Superkeys

A superkey is a set of attributes A1, …, An s.t. for any other attribute B in R,we have {A1, …, An} B

A key is a minimal superkey

I.e. all attributes are functionally determined by a superkey

Meaning that no subset of a key is also a superkey


Finding Keys and Superkeys

• For each set of attributes X

1. Compute X+

2. If X+ = set of all attributes then X is a superkey

3. If X is minimal, then it is a key Do we need to check all sets of

attributes? Which sets?


Example of Finding Keys

Product(name, price, category, color)

{name, category} price{category} color

What is a key?


Example of Keys

Product(name, price, category, color)

{name, category} price{category} color

{name, category}+ = {name, price, category, color}= the set of all attributes this is a superkey this is a key, since neither name nor category alone is a superkey


73

Activity-5-3.ipynb


Lecture 7:Design Theory II

Lecture 7

75

Today’s Lecture

1. PS#1 Review (via AJ RATNER!!!!)

2. Decompositions• ACTIVITY

3. Online Course Feedback• Give you some time in class to fill it out.• Want feedback about new format (20+10/notebooks/heavy ps

feedback). Whatever else we’re doing wrong

Lecture 7

PS1: What you learned

• This was a tough problem set- congratulations on doing so well!

• You used a declarative programming language (SQL) to• do linear algebra• answer questions about data• do graph operations

• Cool stuff! However the point is not these specific applications…

• Less tricky versions of these same types of queries will be fair game for exams

PS1 Recap

Linear Algebra, Declaratively

• Matrix multiplication & other operations = just joins!

• The shift from procedural to declarative programming

PS1 Recap > Problem 1

𝐶𝑖𝑗=∑𝑘=1𝑚 𝐴𝑖𝑘𝐵𝑘 𝑗

C = [[0]*p for i in range(n)]

for i in range(n): for j in range(p): for k in range(m): C[i][j] += A[i][k] * B[k][j]

Proceed through a series of instructions

SELECT A.i, B.j, SUM(A.x * B.x)FROM A, BWHERE A.j = B.iGROUP BY A.i, B.j;

Declare a desired output set

Common SQL Query Paradigms

GROUP BY / HAVING + Aggregators + Nested queries


SELECT station_id, COUNT(day) AS nbd

FROM precipitation, (SELECT day, MAX(precip) FROM precipitation GROUP BY day) AS mWHERE day = m.day AND precip = m.precipGROUP BY station_idHAVING COUNT(day) > 1ORDER BY nbd DESC;

Think about order*!

*of the semantics, not the actual execution

Common SQL Query ParadigmsGROUP BY / HAVING + Aggregators + Nested queries




GROUP BY

SELECT Get the max precipitation by day





GROUP BY

Get the station, day pairs where / when this happened

SELECT

SELECT

WHERE

Get the max precipitation by day





GROUP BY


SELECT

SELECT

WHERE


GROUP BY Group by stations





GROUP BY


SELECT

SELECT

WHERE


GROUP BY Group by stations

HAVING Having > 1 such day

Common SQL Query Paradigms

Complex correlated queries


SELECT x1.p AS medianFROM x AS x1WHERE

(SELECT COUNT(*) FROM X AS x2

WHERE x2.p > x1.p)=(SELECT COUNT(*) FROM X AS x2 WHERE x2.p < x1.p);

This was a tricky problem- but good practice in thinking about things declaratively

Common SQL Query ParadigmsNesting + EXISTS / ANY / ALL


SELECT sid, p3.precipFROM (

SELECT sid, precipFROM precipitation AS p1WHERE precip > 0 AND NOT EXISTS (

SELECT p2.precipFROM precipitation AS p2WHERE p2.sid = p1.sid AND p2.precip > 0 AND p2.precip < p1.precip)) AS p3

WHERE NOT EXISTS (SELECT p4.precipFROM precipitation AS p4WHERE p4.precip – 400 > p3.precip);

More complex, but again just think about order!

Graph traversal & recursion


A

B

For fixed-length pathsSELECT A, B, dFROM edges



A

B

For fixed-length pathsSELECT A, B, dFROM edgesUNIONSELECT e1.A, e2.B,

e1.d + e2.d AS dFROM edges e1, edges e2WHERE e1.B = e2.A AND e2.B <> e1.A



A

B

For fixed-length pathsSELECT A, B, dFROM edgesUNIONSELECT e1.A, e2.B,

e1.d + e2.d AS dFROM edges e1, edges e2WHERE e1.B = e2.A AND e2.B <> e1.AUNIONSELECT e1.A, e3.B,

e1.d + e2.d + e3.d AS dFROM edges e1, edges e2, edges e3WHERE e1.B = e2.A AND e2.B = e3.A AND e2.B <> e1.A AND e3.B <> e2.A AMD e3.B <> e1.A



AB

For variable-length paths on trees

WITH RECURSIVE paths(a, b, b_prev, d) AS (

SELECT A, B, AFROM edgesUNIONSELECT p.a, e.B, e.A, p.d + e.dFROM paths p, edges eWHERE p.b = e.A AND s.B <> p.b_prev)

SELECT a, b, MAX(d)FROM paths;



B

For variable-length paths on trees

WITH RECURSIVE paths(a, b, b_prev, d) AS (

SELECT A, B, AFROM edgesUNIONSELECT p.a, e.B, e.A, p.d + e.dFROM paths p, edges eWHERE p.b = e.A AND e.B <> p.b_prev)

SELECT a, b, MAX(d)FROM paths;

p.a

pp.b

p.bprev

A

90

Today’s Lecture

1. Boyce-Codd Normal Form• ACTIVITY

2. Decompositions & 3NF• ACTIVITY

3. MVDs• ACTIVITY

Lecture 7

91

1. Boyce-Codd Normal Form


92

What you will learn about in this section1. Conceptual Design

2. Boyce-Codd Normal Form

3. The BCNF Decomposition Algorithm

4. ACTIVITY


Conceptual Design

Lecture 7 > Section 1 > Conceptual Design

94

Back to Conceptual Design

Now that we know how to find FDs, it’s a straight-forward process:

1. Search for “bad” FDs

2. If there are any, then keep decomposing the table into sub-tables until no more bad FDs

3. When done, the database schema is normalized

Recall: there are several normal forms…

Lecture 7 > Section 1 > Conceptual Design

Boyce-Codd Normal Form (BCNF)

• Main idea is that we define “good” and “bad” FDs as follows:

• X A is a “good FD” if X is a (super)key• In other words, if A is the set of all attributes

• X A is a “bad FD” otherwise

•We will try to eliminate the “bad” FDs!

Lecture 7 > Section 1 > BCNF

Boyce-Codd Normal Form (BCNF)

• Why does this definition of “good” and “bad” FDs make sense?

• If X is not a (super)key, it functionally determines some of the attributes

• Recall: this means there is redundancy• And redundancy like this can lead to data anomalies!







97

Boyce-Codd Normal Form

BCNF is a simple condition for removing anomalies from relations:

In other words: there are no “bad” FDs

A relation R is in BCNF if:

if {A1, ..., An} B is a non-trivial FD in R

then {A1, ..., An} is a superkey for R

Equivalently: sets of attributes X, either (X+ = X) or (X+ = all attributes)


98

Example

What is the key?{SSN, PhoneNumber}

Name SSN PhoneNumber City

Fred 123-45-6789 206-555-1234 Seattle

Fred 123-45-6789 206-555-6543 Seattle

Joe 987-65-4321 908-555-2121 Westfield

Joe 987-65-4321 908-555-1234 Westfield

{SSN} {Name,City}

Not in BCNF

This FD is bad because it is not a superkey


99

Example

Name SSN CityFred 123-45-6789 SeattleJoe 987-65-4321 Madison

SSN PhoneNumber

123-45-6789 206-555-1234

123-45-6789 206-555-6543

987-65-4321 908-555-2121

987-65-4321 908-555-1234

Let’s check anomalies:• Redundancy ?• Update ?• Delete ?

{SSN} {Name,City}

Now in BCNF!

This FD is now good because it is the key


100

BCNF Decomposition Algorithm

BCNFDecomp(R): Find X s.t.: X+ ≠ X and X+ ≠ [all attributes]

if (not found) then Return R let Y = X+ - X, Z = (X+)C decompose R into R1(X Y) and R2(X Z) Return BCNFDecomp(R1), BCNFDecomp(R2)


101


BCNFDecomp(R): Find a set of attributes X s.t.: X+ ≠ X and X+ ≠ [all attributes]


Find a set of attributes X which has non-trivial “bad” FDs, i.e. is not a superkey, using closures


102




If no “bad” FDs found, in BCNF!


103




Let Y be the attributes that X functionally determines (+ that are not in X)

And let Z be the other attributes that it doesn’t


104




X ZY

R1 R2

Split into one relation (table) with X plus the attributes that X determines (Y)…


105




X ZY

R1 R2

And one relation with X plus the attributes it does not determine (Z)


106




Proceed recursively until no more “bad” FDs!


R(A,B,C,D,E)BCNFDecomp(R): Find a set of attributes X s.t.: X+ ≠ X and X+ ≠ [all attributes]


Example

{A} {B,C}{C} {D}


108

Example

R(A,B,C,D,E) {A}+ = {A,B,C,D} ≠ {A,B,C,D,E}

R1(A,B,C,D) {C}+ = {C,D} ≠ {A,B,C,D}

R2(A,E)R11(C,D) R12(A,B,C)

R(A,B,C,D,E)

{A} {B,C}{C} {D}


109

Activity-7-1.ipynb


110

2. Decompositions


111

What you will learn about in this section1. Lossy & Lossless Decompositions

2. ACTIVITY


112

Recap: Decompose to remove redundancies1. We saw that redundancies in the data (“bad FDs”) can lead to data

anomalies

2. We developed mechanisms to detect and remove redundancies by decomposing tables into BCNF1. BCNF decomposition is standard practice- very powerful & widely used!

3. However, sometimes decompositions can lead to more subtle unwanted effects…

Lecture 7 > Section 2 > Decompositions

When does this happen?

113

Decompositions in General

R1 = the projection of R on A1, ..., An, B1, ..., Bm

R(A1,...,An,B1,...,Bm,C1,...,Cp)

R1(A1,...,An,B1,...,Bm) R2(A1,...,An,C1,...,Cp)


R2 = the projection of R on A1, ..., An, C1, ..., Cp

114

Theory of Decomposition

Name Price Category

Gizmo 19.99 Gadget

OneClick 24.99 Camera

Gizmo 19.99 Camera

Name Price

Gizmo 19.99

OneClick 24.99

Gizmo 19.99

Name Category

Gizmo Gadget

OneClick Camera

Gizmo Camera

I.e. it is a Lossless

decomposition

Sometimes a decomposition is “correct”


115

Lossy Decomposition

Name Price Category

Gizmo 19.99 Gadget

OneClick 24.99 Camera

Gizmo 19.99 Camera

Name Category

Gizmo Gadget

OneClick Camera

Gizmo Camera

Price Category

19.99 Gadget

24.99 Camera

19.99 Camera

What’s wrong here?

However sometimes it isn’t


Lossless Decompositions

What (set) relationship holds between R1 Join R2 and R if lossless?

Hint: Which tuples of R will be present?

It’s lossless if we have equality!

R(A1,...,An,B1,...,Bm,C1,...,Cp)

R1(A1,...,An,B1,...,Bm) R2(A1,...,An,C1,...,Cp)



A decomposition R to (R1, R2) is lossless if R = R1 Join R2

R(A1,...,An,B1,...,Bm,C1,...,Cp)

R1(A1,...,An,B1,...,Bm) R2(A1,...,An,C1,...,Cp)



118

BCNF decomposition is always lossless. Why?

Note: don’t need {A1, ..., An} {C1, ..., Cp}

If {A1, ..., An} {B1, ..., Bm}

Then the decomposition is lossless

R(A1,...,An,B1,...,Bm,C1,...,Cp)

R1(A1,...,An,B1,...,Bm) R2(A1,...,An,C1,...,Cp)


A problem with BCNF

Note: This is historically inaccurate, but it makes it easier to explain

Problem: To enforce a FD, must reconstruct original relation—on each insert!


120

A Problem with BCNF{Unit} {Company}{Company,Product} {Unit}

We do a BCNF decomposition on a “bad” FD:{Unit}+ = {Unit, Company}

We lose the FD {Company,Product} {Unit}!!

Unit Company Product

… … …

Unit Company

… …

Unit Product

… …

{Unit} {Company}


121

So Why is that a Problem?

No problem so far. All local FD’s are satisfied.

Unit Company

Galaga99 UW

Bingo UW

Unit Product

Galaga99 Databases

Bingo Databases

Unit Company Product

Galaga99 UW Databases

Bingo UW Databases

Let’s put all the data back into a single table again:

{Unit} {Company}

Violates the FD {Company,Product} {Unit}!!


122

The Problem

• We started with a table R and FDs F

• We decomposed R into BCNF tables R1, R2, …with their own FDs F1, F2, …

• We insert some tuples into each of the relations—which satisfy their local FDs but when reconstruct it violates some FD across tables!

Practical Problem: To enforce FD, must reconstruct R—on each insert!


123

Possible Solutions

• Various ways to handle so that decompositions are all lossless / no FDs lost• For example 3NF- stop short of full BCNF decompositions. See Bonus Activity!

• Usually a tradeoff between redundancy / data anomalies and FD preservation…

BCNF still most common- with additional steps to keep track of lost FDs…


124

Activity-7-2.ipynb


125

3. MVDs


126

What you will learn about in this section1. MVDs

2. ACTIVITY


Multiple Value Dependencies (MVDs)

MVD Ex: For each fixed course (e.g. CS145),Every staff member in that course and every student in that course occur in a tuple in that table.

Course Staff Student

CS949 Amy Bob

CS145 Chris Deb

CS145 Chris Eli

CS145 Firas Deb

CS145 Firas Eli

Write: Course ↠ Staff or Course ↠ Student

Lecture 7 > Section 3 > MVDs

Formal Definition of MVDCourse Staff Student

CS949 Amy Bob

CS145 Chris Deb

CS145 Chris Eli

CS145 Firas Deb

CS145 Firas Eli

We write A↠ B if for any tuples t1,t2 s.t. t1[A] = t2[A] there is a tuple t3 s.t.• t3[A] = t1[A]• t3[B] = t1[B]

• and t3[C] = t2[C]where C = (AB)C, i.e. the attributes of R not in A or B.

t1

t2

t3

Course Staff ↠


Formal Definition of MVD


CS949 Amy Bob

CS145 Chris Deb

CS145 Chris Eli

CS145 Firas Deb

CS145 Firas Eli



t1

t2

Course Staff ↠

A




CS949 Amy Bob

CS145 Chris Deb

CS145 Chris Eli

CS145 Firas Deb

CS145 Firas Eli



t1

t2

t3

Course Staff ↠

A




CS949 Amy Bob

CS145 Chris Deb

CS145 Chris Eli

CS145 Firas Deb

CS145 Firas Eli



t1

t2

t3

Course Staff ↠

A B




CS949 Amy Bob

CS145 Chris Deb

CS145 Chris Eli

CS145 Firas Deb

CS145 Firas Eli



t1

t2

t3

Course Staff ↠

A B C


Does Course ↠ Staff hold now?


CS949 Amy Bob

CS145 Chris Deb

CS145 Chris Eli

CS145 Firas Eli


Connection to FDs

If A B does A B ?↠

Hint: sort of like multiplying by one…


Comments on MVDs

• MVDs have “rules” too! • Experts: Axiomatizable

• 4th Normal Form is “non-trivial MVD”

• For AI nerds: MVD is conditional independence in graphical models!


136

Activity-7-3.ipynb


Summary

• Constraints allow one to reason about redundancy in the data

• Normal forms describe how to remove this redundancy by decomposing relations• Elegant—by representing data appropriately certain errors are essentially

impossible• For FDs, BCNF is the normal form.

• A tradeoff for insert performance: 3NF

Lectures 5,7 > SUMMARY

Feedback!

• https://ctleval.stanford.edu/auth/evaluation.php?id=9451

• We’re trying a new format (the notebooks, the 20+activity lecture format).

• Constructive feedback (of any aspect) is really appreciated!• We’ve tried to be responsive (when we can get out of our own way!)

https://ctleval.stanford.edu/auth/evaluation.php?id=9451



Documents

Lectures 5 & 7: Design Theory Lectures 5 & 7. Announcements Homework #1 due today! Homework was not easy You learned a new, declarative way of programming!