Joins And Subqueries

JoinsAnd

Subqueries

Multi-table queries(JOIN) 2 tablesRetrieve the part name for all parts shipped in quantities equal to 200SELECT DISTINCT PART.PartNameFROM PART, SHIPMENTWHERE PART.PartNumber = SHIPMENT.PartNumber AND SHIPMENT.quantity = 200;

Multi-table queries(JOIN) 2 tablesSELECT DISTINCT PART.PartNameFROM PART, SHIPMENTWHERE PART.PartNumber = SHIPMENT.PartNumber AND SHIPMENT.quantity = 200;

So how does this work?

Conjunct #1 is a PK FK comparison

Conjunct #2 qualifies retrieval


Conceptually, we can understand the execution of the query in terms of row scans.

A Scan is a sequential inspection of many rows for the purpose of returning rows that meet a criteria.

The following demonstrations are a simplified version of the Nested Loop technique (as opposed to “merge sort” and “hashed join”)


PK/FK comparison requires one scan of the FK set for each PK in the parent table.Row is included in the view where PK=FK and Qty = 200


(Scan 1)

.

.

.(FK Scan)

(No Matches: TT)

(P1=P1 300=200)(P1=P2 200=200)

(P1=P5 100=200)


(Scan 2)

.

.

.(FK Scan)

(Two Matches: TT)

(P2=P1 300=200)(P2=P2 200=200)

(P2=P5 100=200)

This removes duplicates in results


(Scan 3)

.

.

.(FK Scan)

( No Matches: TT)

(P3=P1 300=200)(P3=P2 200=200)

(P3=P5 100=200)


(Scan 4)

.

.

.(FK Scan)

( Two Matches: TT)

(P4=P1 300=200)(P4=P2 200=200)

(P4=P5 100=200)

(And So On...)


(Scan 6)

How many row comparisons?

rows(PK-table) X rows(FK-table)

6 X 12 = 72

Multi-table queriesJOIN Evaluation Techniques

• Three common methods used by DBMS optimizers to evaluate Joins– Nested Loop– Merge Scan– Hash Join


• Nested Loop– Essentially the preceding demo– One table defined as external and one

table defined as internal– (External:Internal)– (1:M)– (Parent:Child)

– If there isn’t an index on the FK, the internal table has to be opened for a scan for every row of the external table


• Merge Scan– Both tables have to be ordered by PK/FK– Parallel scans are executed on both

tables– Qualifying rows are found by merging

the order lists into groups or partitions


• Hash Join– Both tables are stored using hash

function on join attributes (PK/FK)– Execute join on each partition

Multi-table queries(JOIN) 3 tablesRetrieve supplier name and part name for parts shipped in quantities less than 400.

SELECT DISTINCT SUPPLIER.SupplierName, PART.PartNameFROM SUPPLIER, PART, SHIPMENTWHERE PART.PartNumber = SHIPMENT.PartNumber AND SUPPLIER.SupplierNumber = SHIPMENT.SupplierNumber AND SHIPMENT.Quantity < 400;

Multi-table queries(JOIN) 3 tablesSELECT DISTINCT SUPPLIER.SupplierName, PART.PartNameFROM SUPPLIER, PART, SHIPMENTWHERE PART.PartNumber = SHIPMENT.PartNumber AND SUPPLIER.SupplierNumber = SHIPMENT.SupplierNumber AND SHIPMENT.Quantity < 400;

(Scan 1)

(One Match: TT T)

.

.

.(FK Scan)

(P1=P1 S1=S1 400>300)(P1=P2 S1=S1 400>200)

(P1=P5 S1=S4 400>100)


(Scan 2)

(No Matches: TT T)

.

.

.(FK Scan)

(P1=P1 S2=S1 400>300)(P1=P2 S2=S1 400>200)

(P1=P5 S2=S4 400>100)


(Scan 2-5)

(No Matches: TT T)

.

.

.

So we finish out the “P1” Supplier scan with no matches


(Scan 6)

(One Match: TT T)

.

.

.(FK Scan)

(P2=P1 S1=S1 400>300)(P2=P2 S1=S1 400>200)

(P2=P5 S1=S4 400>100)


(Scan 7)

(One Match: TT T)

.

.

.(FK Scan)

(P2=P1 S2=S1 400>300)(P2=P2 S2=S1 400>200)

(P1=P5 S2=S4 400>100)


(Scan 8)

(One Match: TT T)

.

.

.(FK Scan)

(P2=P1 S3=S1 400>300)(P2=P2 S3=S1 400>200)

(P1=P5 S3=S4 400>100)

(And so on...)


How many row comparisons?

6x5x12 = 360

Multi-table queries(JOIN) 3 tablesRetrieve the supplier name and part name for all shipments with quantity greater than or equal to 200 for which the warehouse is located in the same city as the supplier

SELECT DISTINCT SUPPLIER.SupplierName, PART.PartNameFROM SUPPLIER, PART, SHIPMENTWHERE (PART.PartNumber = SHIPMENT.PartNumber AND SUPPLIER.SupplierNumber = SHIPMENT.SupplierNumber) AND (SHIPMENT.Quantity >= 200 AND SUPPLIER.SupplierCity = PART.PartCity);

Multi-table queries(JOIN) Let’s recap...For each row in the outer table (Parent), every row in the inner table (Child) is scanned for a PK/FK match and any qualifying predicate.

In your WHERE clause, you must have a PK=FK statement for every Parent/Child relationship involved in the query.

(PK=FK Scan)

Data

Data

Data

Data

Data

FK Value

FK Value

FK Value

FK Value

FK Value

PK Value

PK Value

PK Value

Data

Data

Data

(Outer Table) (Inner Table)

OuterTable

InnerTable

Sub-Queries (nested)UncorrelatedSo... Joins are really just the combination of the PRODUCT and SELECT relational operators.

But in the case where you have a qualifying predicate such as PartNumber=‘P2’, why do a PRODUCT of the outer table with the entire inner table?Why not eliminate some of the rows from the inner table first with a simple query and then do a PRODUCT of the resulting (and smaller) table?Recall, that the result of a relational operation is a table. So we can Join the outer table with the intermediate table resulting from the inner or sub-query.

Data

Data

Data

Data

Data

FK Value

FK Value

FK Value

FK Value

FK Value

PK Value

PK Value

PK Value

Data

Data

Data

(Outer Table) (Inner Table)Scan inner table

for PartNumber = ‘P2’

(Intermediate Table)

FK valueData

FK ValueData

FK ValueData

Remove non-

qualifying rows

Then Join Outer with

Intermediate

Sub-Queries (nested)Introduction

• Nested queries can be either correlated or uncorrelated– correlated:inner query depends on row that is

currently being examined in the outer query– uncorrelated: inner query performed

independently of outer query

• The nested or sub-query usually appears in the WHERE clause of a query

• Sub-queries can also appear in the FROM and HAVING clause


• General structure of uncorrelated sub-query

SELECT Item FROM table1 WHERE Item IN [NOT IN] (SELECT Item FROM table2 WHERE predicate);

(Inner Query)

(Outer Query)

Sub-Queries (nested)UncorrelatedRetrieve supplier names for suppliers who supply part P2

SELECT DISTINCTROW SUPPLIER.SupplierNameFROM SUPPLIERWHERE SUPPLIER.SupplierNumber IN (SELECT SHIPMENT.SupplierNumber FROM SHIPMENT WHERE SHIPMENT.PartNumber = 'P2');

Sub-Queries (nested)UncorrelatedSELECT DISTINCTROW SUPPLIER.SupplierNameFROM SUPPLIERWHERE SUPPLIER.SupplierNumber IN (SELECT SHIPMENT.SupplierNumber FROM SHIPMENT WHERE SHIPMENT.PartNumber = 'P2');

Intermediate

Now join outer table to intermediate result

Outer Result

Execute inner query: Scan SHIPMENT for ‘P2’


Intermediate

Now join outer table to intermediate result

Outer Result


How many row comparisons?12 + (5 x 4) = 32

Intermediate

If formulated as a join?(5 x 12) = 60


Formulated as a Join

SELECT DISTINCT SUPPLIER.SupplierNameFROM SUPPLIER, SHIPMENTWHERE SUPPLIER.SupplierNumber = SHIPMENT.SupplierNumber AND SHIPMENT.PartNumber=‘P2’;

Use a join if you need columns from multiple tables. If, as in the example above, you need columns from only one table, use either a join or a subquery.

A subquery may include the GROUP BY and HAVING clauses, but not the ORDER BY and UNION.

Sub-Queries (nested)Uncorrelated (3 tables)Retrieve supplier name and city for all suppliers who supply at least one galvanized part.SELECT DISTINCTROW SUPPLIER.SupplierName, SUPPLIER.SupplierCityFROM SUPPLIERWHERE SUPPLIER.SupplierNumber IN (SELECT SHIPMENT.SupplierNumber FROM SHIPMENT WHERE SHIPMENT.PartNumber IN (SELECT PART.PartNumber FROM PART WHERE METAL = 'GALV'));

Sub-Queries (nested)Uncorrelated (3 tables)SELECT DISTINCTROW SUPPLIER.SupplierName, SUPPLIER.SupplierCityFROM SUPPLIERWHERE SUPPLIER.SupplierNumber IN (SELECT SHIPMENT.SupplierNumber FROM SHIPMENT WHERE SHIPMENT.PartNumber IN (SELECT PART.PartNumber FROM PART WHERE METAL = 'GALV'));

PART is scanned for Metal=‘GALV’

Inner-most query first

Inner-most Intermediate

result


Join SHIPMENT with Intermediate result of inner query

Inner-most Intermediate

result

Join

Intermediate SHIPMENT Result


Join SUPPLIER with Intermediate result of inner query

Intermediate SHIPMENT Result

Join


In Summary...(Join)

(Join)


SELECT DISTINCTROW SUPPLIER.SupplierName, SUPPLIER.SupplierCityFROM SUPPLIER, SHIPMENT, PARTWHERE SUPPLIER.SupplierNumber = SHIPMENT.SupplierNumber AND SHIPMENT.PartNumber = PART.PartNumber AND PART.METAL = 'GALV';

SUPPLIER

SHIPMENT

PART

Written as a Join


How many row comparisons for subquery?

(Join)

6 + (2*12) + (3*5) = 45

As a Join?

6*5*12 = 360

(Join)

Sub-Queries (nested)An alternative formulation

SELECT T1.Item …, T2.Item… FROM table1 T1, (SELECT Item FROM table2 WHERE predicate) As T2 WHERE T1.PK = T2.FK;

In this case, the inner table is filtered by the predicate and then joined to the outer table using the alias T2. This overcomes the shortcoming of the IN method by allowing attributes from the inner table to be included in the outer SELECT. See handout for examples from class.


• General structure of correlated sub-query• Using Exists

SELECT Item FROM table1 WHERE EXISTS [NOT EXISTS] (SELECT Item FROM table2 WHERE PK = FK AND predicate);

(Inner Query)

(Outer Query)


• General structure of correlated sub-query• Using IN

SELECT Item FROM table1 WHERE predicate IN (SELECT Item FROM table2 WHERE PK = FK);

(Inner Query)

(Outer Query)

Sub-Queries (nested)CorrelatedRetrieve supplier names for suppliers who supply part P1

SELECT SUPPLIER.SupplierNameFROM SUPPLIERWHERE EXISTS (SELECT SHIPMENT.PartNumber FROM SHIPMENT WHERE SUPPLIER.SupplierNumber = SHIPMENT.SupplierNumber AND SHIPMENT.PartNumber = 'P1');



This is what makes it correlated.

The inner query is executed once for every row in the SUPPLIER table. That is, the value for SupplierNumber is passed by value into the sub-query.



Tests if intermediate result is nonempty (empty set)

For each nonempty intermediate result, the SupplierName is selected.The EXISTS test will be performed for each row in the SUPPLIER table.

Sub-Queries (nested)CorrelatedSELECT SUPPLIER.SupplierNameFROM SUPPLIERWHERE EXISTS (SELECT SHIPMENT.PartNumber FROM SHIPMENT WHERE SUPPLIER.SupplierNumber = SHIPMENT.SupplierNumber AND SHIPMENT.PartNumber = 'P1');

Intermediate

EXISTS?

Yes

Execute nested query


EXISTS?

Yes


Intermediate


EXISTS?

No


Intermediate


EXISTS isFalse for S4 and S5

Intermediate

Sub-Queries (nested)CorrelatedRetrieve supplier names for suppliers who DO NOT supply part P1

SELECT SUPPLIER.SupplierNameFROM SUPPLIERWHERE NOT EXISTS (SELECT SHIPMENT.PartNumber FROM SHIPMENT WHERE SUPPLIER.SupplierNumber = SHIPMENT.SupplierNumber AND SHIPMENT.PartNumber = 'P1');

Sub-Queries (nested)CorrelatedUsing “IN”

SELECT DISTINCTROW SUPPLIER.SupplierNameFROM SUPPLIERWHERE 'P1' IN (SELECT SHIPMENT.PartNumber FROM SHIPMENT WHERE SUPPLIER.SupplierNumber =SHIPMENT.SupplierNumber);

It’s still correlated.

But now, the intermediate result will be a list of PartNumbers for the current Supplier.If ‘P1’ is IN that resulting set, the SupplierName is selected.

Sub-Queries (nested)CorrelatedSELECT DISTINCTROW SUPPLIER.SupplierNameFROM SUPPLIERWHERE 'P1' IN (SELECT SHIPMENT.PartNumber FROM SHIPMENT WHERE SUPPLIER.SupplierNumber =SHIPMENT.SupplierNumber);

S5

S3 S4S1 S2

Sub-Queries (nested)Correlated (in HAVING clause)Find the average Weight of PARTs for each City that has at least two parts.SELECT P1.PartCity, AVG (Weight) AS AverageFROM PART P1GROUP BY P1.PartCityHAVING 1 < (SELECT COUNT(*) FROM PART P2 WHERE P1.PartCity=P2.PartCity);

Only has 1

Sub-Queries (nested)Correlated (in HAVING clause)Find the average Weight of PARTs for each City that has at least two parts.SELECT P1.PartCity, AVG (Weight) AS AverageFROM PART P1GROUP BY P1.PartCityHAVING 1 < (SELECT COUNT(*) FROM PART P2 WHERE P1.PartCity=P2.PartCity);

A sub-query on the same table

Correlated with P1 cursor in the PART table

Sub-Queries (nested)Correlated (DIVISION)Retrieve all the suppliers that have shipped all parts in the PART table.

• As stated in the definition of the DIVISION operator, this is a general operation for any case where you want to know if a particular instance in one table corresponds to every instance in another table.

• For example– List employees that have passed all the exams

for MS certification– List students that have taken all courses for

graduation

Division

Page 206


• How would you do it procedurally?


An outer loop that scrolls through Suppliers

Supplier Loop

End Loop

An inner loop that scrolls through Parts for every row in Suppliers

End Loop

Part Loop An inner-most loop that scrolls through Shipment for every row in Part looking for PK=FK matchesEnd Loop

Shipment Loop

PK=FK?

Sub-Queries (nested)Correlated (DIVISION)Priming read (Supplier)While NOT EOF Supplier

Initialize exit condition to TRUEPriming read (Part)While (NOT EOF Part) AND (PK=FK found)

Set exit condition to FALSEPriming read (Shipment)While (NOT EOF Shipment) AND (NOT PK=FK found)

Evaluate PK=FKSet found booleanMove cursor (Shipment)

End While Move cursor (Part)

End WhileIf supply all parts, print Supplier Info Move cursor (Supplier)

End While

Pseudo-code

Sub-Queries (nested)Correlated (DIVISION)

rstSupplier.MoveFirstDo While Not rstSupplier.EOF FoundPart = True rstParts.MoveFirst Do While (Not rstParts.EOF) And FoundPart FoundPart = False rstShipment.MoveFirst Do While Not rstShipment.EOF And (Not FoundPart) FoundPart = (rstShipment.Fields("PartNumber") = rstParts.Fields("PartNumber")) _ And (rstShipment.Fields("SupplierNumber") = rstSupplier.Fields("SupplierNumber")) rstShipment.MoveNext Loop rstParts.MoveNext Loop If FoundPart Then With rstSupplier picBox.Print .Fields("SupplierNumber"), .Fields("SupplierName"), .Fields("SupplierCity") End With End IfLoop

Visual BasicCourtesy Dr. Landry*

* His original code was well documented...


SELECT SUPPLIER.SupplierNumber, SUPPLIER.SupplierName

FROM SUPPLIER

WHERE SUPPLIER.SupplierNumber IN

(SELECT SupplierNumber

FROM SHIPMENT S1

WHERE NOT EXISTS

(SELECT PART.PartNumber

FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2

WHERE S2.SupplierNumber = S1.SupplierNumber AND

S2.PartNumber = PART.PartNumber)));

Starting from the inner most query and working up...

SHIPMENT, using cursor ‘S2’, ...

is scanned once for every row in PART.

SQL



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



PART is scanned once, ...

For every row in SHIPMENT, ...

using cursor ‘S1’.



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



The resulting set of SupplierNumbers from SHIPMENT using ‘S1’, ...

is joined with SUPPLIERto retrieve SupplierName


Uses two cursors into SHIPMENT

SELECT SUPPLIER.SupplierNumber

FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2

(TRUE)(S1=S1) AND (P1=P1)

P1

Intermediate result



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2

(FALSE)(S1=S1) AND (P2=P1)

P1

Intermediate result



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2


P1

Intermediate result

and so on...



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2

P1

Intermediate result


Is Intermediate result empty?

NOT(True) = False



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2


So we don’t retrieve the PartNumber from PART



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2


and so on...

Now we re-run the inner-most query for the next value in PART



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2

P2

Intermediate result



NOT(True) = False



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2


So we don’t retrieve the PartNumber from PART And this

cycle gets repeated for every row in

PART



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2


So we don’t retrieve the PartNumber from PART And this

cycle gets repeated for every row in

PART



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2


Once we’ve searched every row in PART,



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2for the first row in SHIPMENT, we evaluate the WHERE clause



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2Did the inner query return an empty set?

Yes. So retrieve the SupplierNumber

S1

Intermediate result



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2Now we move to the next row in SHIPMENT

And repeat the entire sub-query

again.

(TRUE)(S1=S1) AND (P1=P1)



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2However... when we get to this point in the execution




FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2there will be no instances of (S2=S2) AND (P3=P3)

Intermediate result

So the innermost query does not return a PartNumber



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2

Intermediate result

Yes, so...


NOT(False) = True



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2the PART sub-query will return the PartNumber that S2 doesn’t supply

P3

Intermediate result



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2At this point, the PART sub-query will have returned the set of Parts not supplied by S2.

P3

Intermediate result

P4

P5

P6



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2Once we’ve searched every row in PART,

P3

Intermediate result

P4

P5

P6



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2

P3

Intermediate result

P4

P5

P6

for the Supplier ‘S2’, we evaluate the WHERE clause



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2

P3

Intermediate result

P4

P5

P6

Did the PART sub-query return an empty set?

No. So don’t retrieve the SupplierNumber ‘S2’



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE NOT EXISTS

(SELECT PartNumber

FROM SHIPMENT S2



S1 S2

S1

And so finally... the SupplierNumber result is joined with SUPPLIER

to get SupplierName



FROM SUPPLIER



FROM SHIPMENT S1

WHERE NOT EXISTS


FROM PART

WHERE PART.PartNumber NOT IN

(SELECT PartNumber

FROM SHIPMENT S2

WHERE S2.SupplierNumber = S1.SupplierNumber)));

An alternative formulation using IN

Sub-Queries (nested)Correlated (DIVISION)List the companyname from the Suppliers table in the NorthWind database for those suppliers who supply ALL products in Category 7.

Documents

Joins And Subqueries