A U Business Data Design Professor J. Alberto Espinosa Business Analysis and Data Design ITEC-630 Fall 2008

AUBusiness Data DesignProfessor J. Alberto Espinosa

Business Analysis and Data DesignITEC-630 Fall 2008

2

Agenda

• Introduction to database concepts• Data modeling & relational database design• Transitional artifacts: the CRUD matrix –

linking requirements to data design• Normalization• Database queries

3

Database Concepts

4

Definitions

Database:An organized collection of “logically related” data

that can be retrieved on demand

Database Management System (DBMS):Software that manages databases

(i.e., define, create, update, and query databases)

Acts as intermediary between business applications and physical data files

“Most powerful, scalable, flexible and effective business applications rely on a well designed database and a powerful underlying DBMS”

5

HARDWAREHARDWARE

System SoftwareSystem Software

Application ProgramsApplication Programs

APIAPI11

INSTRUCTION SETINSTRUCTION SET

The Old Way: Programs and Data files

WindowsUnix, Linux

PC, Mainframe

Accounting, Human Resources

Examples:

Data and program files were separate. You had to write individual programs to: define the data; upload it; update it; manipulate it; and or retrieve it

Data Files

6

HARDWAREHARDWARE

System SoftwareSystem Software

DBMSDBMS

Database ApplicationDatabase Application

APIAPI22

APIAPI11

INSTRUCTION SETINSTRUCTION SET

A Better Way: Using a DBMS

WindowsUnix, Linux

PC, Mainframe

Oracle, Access, MS SQL Server

Accounting, Human Resources,

ERP, CRM

Examples:

A business application passes high level instructions to the DBMS. The DBMS has capabilities to do all the necessary data management: data definition, manipulation, and retrieval. So, the business application does not have to worry about low level data management functions

Database

7

Advantages of Using Databases & DBMSs

• Programs independent of data structure• Less data redundancy• Better consistency in the data• More flexibility & scalability• Easier to integrate & share data• Easier to develop business applications• Easier to enforce business rules/constraints• Easier access to data by users

(e.g., queries, reports, forms, etc.)

8

DBMS and database work in the same computer: the user’s computer OK for personal productivity

Stand-alone DBMS

Database

Stand-aloneDBMS

(e.g., MS Access)

9

DBMS Server: runs the “back-end” part of the DBMS and performs most of the data management functions – e.g., queries, updates, etc.

DBMS Client: runs “front-end” part of the DBMS that provides the user interface (e.g., data entry, screen displays or presentation, report formatting, query building tools)

Data Request (e.g., query)

DBMS in a Client/Server Environment:Better for corporate use the DBMS has two components

Database

DBMSClient

DBMSServer

Response(e.g., query result)

Retrieve, add, delete and/or update data

10

DBMS in a Web Server Environment:Very common when there are large numbers of users and would be impractical to deploy and

install a DBSM client access to the database is done through a browser(e.g., on-line purchases)

Request (ex. get a price quote, place an order)

Response (ex. query results with HTML-formatted product price or order confirmation notice)

11

Business to Business E-Commerce Example

using XML

Internet

e.g., supplier

e.g., buyerDBMS

(e.g., Oracle)SELECT

query

XML Processor

XML Document (e.g., Purchase

Order)

DBMS(e.g., MS

SQL Server)

INSERT query

XML Processor

XML Document (e.g., Purchase

Order)

12

Data Warehouse“A database that stores and consolidates current and historical data from various systems

(internal and external) with tools for management reporting and sophisticated analysis—i.e., Datamining”

Business Intelligence

13

Most Common Database Models

• Hierarchical (of historical interest only)

• Network (of historical interest only)

• Relational

• Object Oriented (new)

14

• For a database to be truly relational, it must comply with 12 rules defined by its inventor (Dr. E. F. Codd).

• No commercially available database complies with the full set of rules, but the 12 rules are used as guidelines for sound database design.

• Rule 1 states that data should be presented in tables• Rule 2 states that data must be accessible without

ambiguity• We will talk more about other rules later (i.e., about

entity integrity and referential integrity – stay tuned).

Relational Database

15

A relational database must have:• Tables: or “entities”

Every table has a unique nameEx. Students, Courses

• Fields: or “columns”, “attributes”Every field has a unique name within the tableEx. Students (StudentID, StudentName, Major, Address)Ex. Courses (CourseNo, CouseName, CreditPoints,

Description)• Records: or “rows”, “tuples”, “instances”

Every record is unique (has a unique field that identifies it)Ex. {“jdoe”, “John Doe”, “CS”, 5000 Forbes Ave.)Ex. {“MGMT-352-001”, “MIS”, Fall 2002, “A great course”}

Implications about Rule 1

16

Unambiguous reference Table.Field WHERE Record Search

Ex. Students.StudentID refers to the StudentID field/column of the Students table

Ex. Courses.CourseName WHERE CourseNo = “ITEC-630” more specifically, refers to the value in the CourseName field/column of the Courses table in the record/row in which the field/column CourseNo is “ITEC-630”, that is “System Requirements”

Implications about Rule 2

17

Object Oriented (OO) Databases

• OO languages + added database functionality, or• Database products + added OO programming facilities• Similar to relational databases• “Classes” (a grouping of similar objects -- like tables)• “Objects” (an instance of a class -- like records)• “Object properties” (object attributes -- like fields)

• Plus:– Methods (i.e., procedures or programs)

Programs embedded in classes and objects– Other OO Properties (inheritance, encapsulation, etc.)

18

Terminology Equivalence

ERD or Data Model

OO Database RelationalDatabase

OtherTerms Used

Entity Class Table

Instances Objects Records Rows, Tuples

Relationship Relationship Relationship

Attributes Properties Fields Columns

19

Software that manages databases

i.e., define, create, update, and query databases

e.g., MS Access, MS SQL Server, Oracle

Database ManagementSystems (DBMS)

20

DBMS Functions and Tools

• Performs 3 main functions:– Data definition (define, create databases)– Data manipulation (data entry, updates)– Data retrieval (extraction, reports, displays)

• Plus additional database tools:– Data dictionary: data about the database– Visual tools: report & form design– Data modeling & database design tools– Macros and programming languages– Internet/web features, etc.

• Examples:– Oracle, DB2, Visual FoxPro, MS Access & MS SQL

21

DatabaseDesign

22

Database Design Goals

• Data integrityAvoid anomalies in the data

• No data redundancyRecord the data in one place only

• Efficient data entryDuplicate data means having to enter the same data more than once

• ConsistencyDuplicate data can lead to inconsistencies when the data changese.g., 2 different addresses for same client

• Flexibility and easy evolutionEast to maintain, update and add new tables

23

Database Design Issue #1:

Enforce Entity Integrity

24

Entity Integrity

• Is ensuring that every record in each table in the database can be addressed (i.e., found) – this means that there each record has to have a unique identifier that is not duplicate or null (i.e., not blank)

• Examples: every student has an AU ID; every purchase order has a unique number; every customer has an ID

Primary key (PK) helps enforce Entity Integrity:• Field(s) that uniquely identifies a record in a table

(e.g., AU user ID)• Entity integrity = PK is not duplicate & not blank• PK can be:

– A single field (e.g., UserID), or– Or more than one field (e.g., OrderNo, LineItem)

25


Enforce Referential Integrity

26

• Is ensuring that the data that is entered in one table is consistent with data in other tables

• Examples: purchase orders can only be placed by valid customers; accounting transactions can only be posted to valid company accounts

Foreign key (FK) helps enforce referential Integrity:• A field in a table that is a PK in another table• That is, a field that “must” exist in another table• This is how referential integrity is maintained

Referential Integrity

27

Illustration: Primary and Foreign Keys

PK

PK

FK

28

Entity, Referential Integrity

PK

FKPK

PK

PK, FKPK, FK

Database Schema: The structure of the database, which contain tables, views, constraints, relations, etc. – just about everything, except the data itself

29

Other Important Keys

• Candidate Keys:– Often there are more than one keys that could serve as a primary

key– Example: Order, LineItem vs. Order, ProdID– Example: AU ID, SSN, AU Login ID– These are called candidate– Any candidate can be selected as the primary key

• Alternative Keys:– Once a primary key has been selected from the choice of

candidate keys, the other keys (not used as PKs) are referred to as “alternative keys”

30


Develop a Data Modelor Entity-Relationship Diagram (ERD)

31

Data Model Example (Entity Relationship Diagram--ERD): Course Registration System

Instructors

InstructorID

LastNameFirstNameTelephoneEMailAddr

Courses

CourseNo

CourseDescriptionInstructorIDCreditPointsPreRequisitesClassroomNo

Teach

Enrollments

StudentIDCourseNo

Comments

Students

StudentID

LastNameFirstNameSSNDepartmentCollegeMajorEMailAddr

Enrolls

Includes

1Many

Many

Many

1

1

Entities

Relationships

32

Data Model Example (Entity Relationship Diagram--ERD): Course Registration System

Teaches

Enrolls

Includes

1 toMany

EntitiesRelationships

Cardinality

33

The Textbook’s ERD Notation

LastName FirstName

Telephone EMail

InstructorID

InstructorID(FK) CourseDescr

CreditPoints PreReqs

CourseNo

Instructors CoursesTeach

Entities

Relationships

34

Peter Chen’s ERD Notation

Instructors

PK InstructorID

LastNameFirstNameTelephoneEMail

Course

PK CourseNo

CourseDescriptionFK1 InstructorID

CreditPointsPreRequisites

Teaches

35

Entity-Relationship Diagrams (ERDs)i.e., Conceptual Data Modeling

• Similar to a class diagram, but without methods and generalizations

• Data-oriented modeling method that describes the data and relationships among data entities

• Goal: capture meaning of the data

• 2 main ERD or data model constructs:– Entities and its attributes– Relationships between entities

36

Entity

“An object, person, place, event or thing or which we want to record data”

• Equivalent to a table in a database• Examples: instructors, students, classrooms, invoices,

registration, machines, countries, states, etc.

• Entity instance: a single occurrence of an entityExample: Espinosa, Kogod 39, ITEC 630

• Entities can be identified in a requirements analysis description by following the use of NOUNS

37

Relationships

• Relationships describe how two entities relate to each other

• Relationships in a database application can be identified following the VERBS that describe how entities are associated with one another

• Examples: students enroll in courses countries have cities, etc.

38

Cardinality• Cardinality is an important database concept to describe how two

entities are related

• The Cardinality of a relationship describes how many instances of one entity can be associated with another entity

• The cardinality of a relationship between two entities has two components:– Maximum Cardinality: is the maximum number of instances that

can be associated with the other entity – usually either 1 or many (the exact number is rarely used)

– Minimum Cardinality: is the minimum number of instances that can be associated with the other entity – usually either 0 or 1

– Symbols:

0

1

Many

39

Cardinality (cont’d.)

• A relationship is fully described by describing the cardinality in both directions of the relationship: e.g., a client places zero (i.e., optional) or many orders and each order must relate to only one (i.e., mandatory) client.

• Examples:

1 student can only park 1 (or 0) cars 1 to (0 or) 1

1 client can place (0 or ) many orders 1 to (0 or) many

1 student can enroll in (at least 1 or) many courses anda course can have (0 or) many students (0 or) many to (1 or) many

40

Example: 2 Entities, 1 Relationship

Instructors

PK InstructorID


Course

PK CourseNo



Teaches

Peter Chen’s notation& MS Visio software

One and only one

Zero or many

41

ERD SYMBOLS (cont’d.)Note: high level conceptual models don’t show attributes, just entities

1 to 1MaximumCardinality(outer symbol)

Minimum Cardinality (inner symbol)

Mandatory Optional

Employee BioData

Employee FamilyData

Has

Has

Peter Chen’s notationusing Systems Architect software

42

ERD SYMBOLS (cont’d.)

1 to Many

1 to Many (or None)

MaximumCardinality

Minimum Cardinality

Mandatory Optional

Advisor Student

Faculty CourseTeaches

Peter Chen’s (“crow’s feet”) notationusing Systems Architect software

→ Advises← Have

43

Many to Many Relationships?

Many to Many

1 to Many

1 to Many (or None)

Convert a Many-to-Many into 2 One-to-Many’s

Orders Products

ProductsOrders

LineItems

Intersection Table

44

Cardinality: 1 to 1 (MS Access notation)

45

Cardinality: 1 to many(MS Access notation)

46

Steps in data modeling Modeling

1. Identify and diagram all ENTITIES

2. Add PK attributes – i.e., implement entity integrityEnsure PK’s are non-null & non-duplicates

3. Identify and diagram all RELATIONSHIPSNote CARDINALITIES (1 to 1, 1 to n, n to n)

4. Add FK attributes – i.e., implement referential integrity (this is automatic in some tools—MS Access)

5. Add remaining attributes

47

ERD Example:Course Registration System

Courses (CourseNo (PK), CourseDescripition, InstructorID, CreditPoints, ClassroomNo)

PreRequisites (CourseNo (PK), PreRequisiteNo (PK), Comments)

Students (StudentID (PK), LastName, FirstName, SSN, Department, College, Major, EMail)

Enrollment (StudentID (PK), CourseNo (PK), Comments)

Instructors (InstructorID (PK), LastName, FirstName, Telephone, EMail)

Classrooms (ClassroomNo (PK), ClassroomName, Building, BuildingRoomNo, Equipment, Capacity)

Note: PK denotes a primary key

48

Example: Course Registration SystemStep 1. Draw Entities

InstructorsCoursePreRequisites

ClassRooms Enrollment Students

49

Instructors

InstructorID

Course

CourseNo

PreRequisites

CourseNoPreRequisiteNo

ClassRooms

ClassroomNo

Enrollment

StudentIDCourseNo

Students

StudentID

Example: Course Registration System

Step 2. Add PK’s (undeline/separate with a line)

50

Instructors

PK InstructorID

Course

PK CourseNo

TeachesPreRequisites

PK,FK1 CourseNoPK PreRequisiteNo

ClassRooms

PK ClassroomNo

Enrollment

PK,FK1 StudentIDPK,FK2 CourseNo

Students

PK StudentID

has

Enrolls

IncludesAssigned


Step 3. Add Relationships (w/Cardinalities)

51


Step 4. Add FK’s

Instructors

PK InstructorID

Course

PK CourseNo

FK1 InstructorIDFK2 ClassroomNo

TeachesPreRequisites


ClassRooms

PK ClassroomNo

Enrollment


Students

PK StudentID

has

Enrolls

IncludesAssigned

52

Instructors

PK InstructorID


Course

PK CourseNo


CreditPoints

FK2 ClassroomNo

Teaches

PreRequisites


Comments

ClassRooms

PK ClassroomNo

ClassroomNameBuildingBuildingRoomNoEquipmentCapacity

Enrollment


Comments

Students

PK StudentID

LastNameFirstNameSSNDepartmentCollegeMajorEMail

Has

Enrolls

Includes

Assigned


Step 5. Add Remaining Attributes

53

Example:Course Registration System

54

EXAMPLE:Package Delivery Tracking System

ClientsPackages

Trucks Drivers

DeliveriesClients

PK ClientID

Packages

PK PackageNo

Trucks

PK TruckNo

Drivers

PK DriverNo

Deliveries

PK DeliveryNo

Clients

PK ClientID

Packages

PK PackageNo

Trucks

PK TruckNo

Drivers

PK DriverNo

Deliveries

PK DeliveryNo

Clients

PK ClientID

Packages

PK PackageNo

FK4 DeliveryNo

Trucks

PK TruckNo

Drivers

PK DriverNo

FK1 TruckNo

Deliveries

PK DeliveryNo

FK4 ClientIDFK5 DriverNo

Clients

PK ClientID

LastName FirstName Address Telephone

Packages

PK PackageNo

FK4 DeliveryNo Size Charge

Trucks

PK TruckNo

Make Model Year

Drivers

PK DriverNo

FK1 TruckNo DriverName LicenseNo

Deliveries

PK DeliveryNo

FK4 ClientIDFK5 DriverNo Date Status

55

Example:Package Delivery Tracking System

56

EXAMPLE:Airline Reservation System

57

Example:Airline Reservation System

58


Implement Important Rules: Update, Delete and

Business Rules

59

Referential Integrity:Update Rules

What can be updated/modified in the database and when?

1. It is OK to update values in any non-PK fields, provided that referential integrity and business rules are respected

2. It is OK to update values in the PK in one table if it is not linked to a FK in another table, provided that entity integrity, referential integrity and business rules are respected

3. If a PK is linked to a FK in another table, we need to ensure that referential integrity is maintained. Depending on what makes business sense, the update rule can be either:• U:R (Update:Restrict) – i.e., Disallow updates of values in the PK, or• U:C (Update:Cascade) – i.e., Allow updates,

but cascade changes to all related FKs in other tables

60

Referential Integrity:Delete Rules

What can be deleted in the database and when?

1. It is OK to delete records in a table [only] if its PK is not linked to a FK in another table

2. If its PK is linked to a FK in another table, we need to ensure that referential integrity is maintained. Depending on what makes business sense, the delete rule can be either:• D:R (Delete:Restrict) – i.e., Disallow deletion of records, or• D:C (Delete:Cascade) – i.e., Allow deletion of records,

but cascade deletions in all related tables that contain a FK linked to this table

61

Illustration of Update and Delete Rules

Instructors

PK InstructorID


Course

PK CourseNo



What happens if (how is referential integrity affected):• We change an instructor’s last name?• We change an InstructorID in the Course table?• We change an InstructorID in the Instructors table?• We delete a course?• We delete an instructor

62

Other Data Integrity:Business Rules

Most DBMS have features that allow you to impose constraints in the data to meet rules imposed by a company when conducting business: i.e., “business rules” – examples:

• CustomerAge >= 18• OrderQty >= 100• ProductPrice <= 1000• PaymentDate <= PurchaseDate + 90

63


“Normalize” Your Design(we will discuss this later)

64

DatabaseQueries

65

DBMS Functions and Tools

• Performs 3 main functions:– Data definition (create databases)– Data manipulation (enter & update data)– Data retrieval (data extraction)

• Plus additional database tools:– Data dictionary: data about the database– Visual tools: report & form design– Data modeling & database design tools– Macros and programming languages– Internet/web features, etc.

UsingQueries

(or proprietary features)

66

Queries

Often thought of as a method to retrieve data, but queries can also be used to define and manipulate data

Databases can be queried in many ways:

•Proprietary DBMS commands and languages, which are unique to the particular DBMS product, or

•Standard query methods/languages (QBE, SQL, etc.), which most DBMS products support

67

Standard Query Methods

Query by Example (QBE) (Design View in MS Access)• Visual interface using examples of data requested• Similar to how you do searches in the library

Structured Query Language (SQL)• Popular with power users• Works in most DBMS• Can embed SQL commands in programs, web scripts,

etc.• English-like commands, practical• Exact, mathematical: relational algebra & matrix math

68

Query by Example (QBE)

• Called Query “Design View” in MS Access• Column labels are the fields we want to retrieve• In table cells we enter “examples” of the info we want

69

Structured Query Language

(SQL)

70

SQL Commands TypesOnly 8 Commands!!

• Data Definition: Create, Drop

• Data Manipulation: Insert, Update, Delete, Union, Join

• Data Retrieval: Select

71

One SQL Command

(from CREATE to ;)

SQL Commands: Data Definition Example: Create & Delete Table called “Employees”

CREATE TABLE Employees (EmployeeID integer, LastName char(24), FirstName char(24), Birthday date, Phone char(10), Notes memo);

; = End ofSQL Command

DROP TABLE Employees;

Fields created in

Employees table

72

SQL Commands: Data Manipulation

• INSERT: Add new records

• UPDATE: Modify existing records

• DELETE: Delete records

• UNION: Combine records from two tables

• JOIN: Combine columns from two tables

73

SQL Commands: Data ManipulationAdd & Update Records

Insert (add) a complete record (values in all fields):

INSERT INTO Employees

VALUES (“ae”, “Espinosa”, “Alberto”, 12/12/2002, “885-1958”, “Looks tired, needs a vacation”);

Update (modify) record with new values:

UPDATE Employees SET LastName=“Espinosa”WHERE EmployeeID = “ae”;

Insert (add) partial record (values in some fields only):

INSERT INTO Employees (EmployeeID, LastName, FirstName) VALUES (“ae”, “Espinosa”, “Alberto”);

74

Data Extraction Queries: The Idea

• Organize database (design, create):– In the most efficient & consistent way (internally) – Not based on how you want the data to look

• Produce the “virtual” temporary tables the way you want them to look using queries How we store

the data

How we display the data

75

Data Extraction in SQL:The “SELECT” CommandNote BOLDFACE denotes SQL Keywords

SELECT field1, field2, etc. – columns to retrieve and displayFROM table1, table2, etc. – tables that contain the dataWHERE condition1 – which records to retrieve AND [OR] condition2 ……. – further conditionsGROUP BY field2, ….. – to group results

HAVING condition3 – like WHERE but after groupingORDER BY field1, field2, etc.... [DESC] [ASC] – to sort the query results

SELECT can be followed by:DISTINCT (SELECT DISTINCT eliminates duplicate rows from result)

TOP n (lists only the top n rows of result – e.g. SELECT Top 5)

* (lists all fields in the table – e.g., SELECT * FROM …)

76

Complexity of SELECT Queries

• Simple Queries:Involve a single table

• Queries with Aggregate Functions:When we only want averages, totals, etc.

• Queries with Aggregate Functions and Grouping:When we want averages, totals, etc. categorized by groups

• Complex Queries with Joins:Involve more than one table

• Complex Nested Queries:Sub-queries (which compute something) within queries

77

Simple SQL SELECT Queries

SELECT ProdID, ProdName, Type, Price (a list of fields) FROM Products (the table where the data resides) WHERE Price>=300; (which rows to display)

SELECT ProdName, Price FROM Products WHERE Price>=120 AND Type=“Percussion”;

Note: the SQL DELETE command works identically to the SELECT command, but instead of displaying the results, it deletes them. For example, the following DELETE command deletes all the records displayed with the previous query (it is not a bad idea to view the records before you delete them)

DELETE ProdName, Price FROM Products WHERE Price>=120 AND Type=“Percussion”;

Note: an SQL query can not only contain a list of fields, but also any expression involving on or more fields. For example:

SELECT Labor, Parts, Labor+Parts AS Charges (column name) FROM Repairs WHERE Labor+Parts>=300;

78

Delimiters

When writing WHERE conditions and similar statements in SQL, one often needs to compare a field with a particular value. The values need to be written within delimiters that match the data type. These are the delimiters:

• Text – quotes:ex. WHERE UserID = “alberto”

• Date/time – pound signex. WHERE OrderDate > #01/07/2008#

• Number – nothingex. WHERE Amount > 200

79

SQL Queries With Aggregate Functions

• These queries yield a single number result(i.e., a table with 1 column and 1 row)

• The only thing you can include in the SELECT line are the fields you are aggregating

• Aggregate functions you can use: Avg, Sum, Min, Max, Count

• These functions aggregate vertically a column of (usually numeric) values (e.g., salaries, payment amounts, etc.)

Note: the Count function counts how many rows meet the Where criteria, so it you can use any column you wish to count and you will get the same results – the easiest thing is to use Count(*)

Note: the AS clause is optional; it does not change the query results; it only changes the column label in the results

Note: you can use more than one aggregate function in one SELECT command

SELECT Avg(Price) AS AvgPrice FROM Products WHERE Price>=120 AND Type=“Percussion”;

SELECT Max(Price) AS MaxPrice, Avg(Price) AS AvgPrice FROM Products WHERE Type=“Guitars”;

SELECT Count(*) as TotOrders FROM Orders WHERE OrderStatus = “Top Priority”

80

SQL Queries With Aggregate Functions and Grouping

• The ONLY things you can include in the SELECT line are:(1) the fields you are aggregating [e.g., Avg(Price)](2) and the fields you are using to group [e.g, Type]

SELECT Type, Avg(Price) AS AvgPrice, Max(Price) AS MaxPrice FROM Products WHERE Price>=1000 GROUP BY Type

SELECT Type, Avg(Price) AS AvgPrice, Max(Price) as MaxPrice FROM Products GROUP BY Type HAVING Avg(Price)>1000

Note: the WHERE clause is evaluated BEFORE the grouping

Note: the HAVING clause is evaluated AFTER the grouping

81

Complex SELECT Queries with Joins

Tables: Orders (OrderNo, ClientID, OrderDate, OrderStatus)

LineItems (OrderNo, LineItem, ProdID, Qty)

Table Join (2 ways):

SELECT Orders.OrderNo, OrderStatus, ClientID, LineItem, ProdID, Qty FROM Orders, LineItems WHERE Orders.OrderNo = LineItems.OrderNo;

Table Product (WRONG!! Don’t forget the join condition):

SELECT Orders.OrderNo, ClientID, LineItem, ProdID, Qty

FROM Orders, LineItems;

Join Condition

82

Complex SELECT Queries with Joins: TIPS

COMPLEX queries that JOIN 2 tables are identical to SIMPLE queries, except for 2 additional rules you MUST ALWAYS apply:

1. The two tables need to be JOINED through the common field that links theme.g., WHERE Orders.OrderNo = LineItems.OrderNo

2. ANY time you refer to a COMMOND FIELD that exists in both tables, you must use a TABLE PREFIX to eliminate the ambiguity e.g., SELECT Orders.OrderNo; WHERE Orders.OrderNo = 990001

For complex queries that JOIN 3 or more tables apply rule 1 for EACH link, and always apply rule 2 – e.g.,

SELECT Clients.ClientID, ClientName, Orders.OrderNo,

OrderStatus, LineItem, ProdID, Qty

FROM Clients, Orders, LineItems

WHERE Clients.ClientID = Orders.ClientID

AND Orders.OrderNo = LineItems.OrderNo

83

Mapping QBE to SQL

SELECT

WHERE

JOIN CONDITION

FROM

ORDER BY

84

Display above average quantities in line items – i.e., order number, line item, product ID and quantity for any line item in which the quantity ordered is above the average quantity ordered in all orders:

SELECT OrderNo, LineItem, ProdID, Qty FROM LineItems WHERE Qty>(SELECT Avg(Qty) FROM LineItems)

Tip: prepare the sub-query first: SELECT Avg(Qty) FROM LineItems

Nested Queries w/Aggregates

85

Display product ID’s and total quantities ordered for that product for totals exceeding 1000 units (2 solutions):

SELECT ProdID, TotQty FROM (SELECT ProdID, Sum(Qty) AS TotQty FROM LineItems GROUP BY ProdID) WHERE TotQty > 1000

Tip: again, prepare the sub-query first:

SELECT ProdID, Sum(Qty) AS TotQty FROM LineItems GROUP BY ProdID

Alternative solution without sub-query:

SELECT Prod ID, Sum(Qty) AS TotQty FROM LineItems GROUP BY ProdID HAVING Sum(Qty) > 1000

Nested Queries w/Aggregates

86

Nested Queries w/Lists

Produce Sub-Query First – e.g., a single-column table (list)

SELECT PartNumber FROM ShipmentsWHERE SupplierID = "S5“GROUP BY PartNumberHAVING Avg(Qty)>200;

Then enclose the Sub-Query in parenthesis and use with IN keyword

SELECT DISTINCT PartNameFROM PartsWHERE PartNumber IN

(SELECT PartNumber FROM Shipments WHERE SupplierID = "S5" GROUP BY PartNumber HAVING AVG(Qty)>200);

Another example:

SELECT DISTINCT PartNameFROM PartsWHERE PartNumber IN (352, 353, 354)

89


“Normalize” Your Design

90

Database Design Goals

• Data integrity (Entity and Referential Integrity – ERD’s)

Avoid anomalies in the data

• No data redundancy

Record the data in one place only

• Efficient data entry

Duplicate data means having to enter the same data more than once

• Consistency

Duplicate data can lead to inconsistencies when the data changes

e.g., 2 different addresses for same client

• Flexibility and easy evolution

East to maintain, update and add new tables

Normalization

91

Why Normalization?

• Question: if a data model/ERD is sound and all entity integrity, referential integrity, update/delete and business rules have been well implemented, does this guarantee a good database design?

Answer: not necessarily. If your design is not “normalized”, you could have redundant data, and that would be a BAD thing (design)

• Normalization should yield the most efficient way to organize and record the data internally—not necessarily how users want to see the data, but what makes more sense for non-redundant data storage

• We can later build user table views (i.e., what the user wants or needs to see) by querying these normalized tables.

• Redundancy: only PK and FK (e.g., client ID’s) values should appear in multiple tables (because they are needed to link tables)

Non-key data (e.g., client last name) that appears in multiple tables is “redundant”

92

ExampleYou gather requirements from users and one user gives you this table and tell you that she would like the system to collect this data.

How would you organize this data internally in the database?

93

Normalization

• Normalization = The systematic process of “decomposing” a set of unorganized tables with redundant data into smaller, simpler, and more organized tables with only minimal data redundant in key fields and no data redundancy on non-key fields — i.e., from chaos to order

Decomposition

Query

Decompose to most efficient internal organization

You can always recover the original data format with a query

94

Degree of Normalization

• Normalization is a matter of degree -- the more normalized your design is, the lower the chances of having redundant data

• Normal Forms (NF) (higher NF designs are more normalized):1NF 2NF 3NF BCNF PJNF DKNF 4NF 5NF

• The process of normalizing a design to 3NF may seem complex, but the concept is very simple:

(1) Minimize data redundancy in key attributes -- i.e., data in key fields can be entered in more than one table

(2) Eliminate data redundancy in non-key attributes -- i.e., data in non-key fields should be entered only in one table

(3) Ensure that every piece of data (each non-key attribute) can be unambiguously located by its PK

(4) Each incremental NF gets us a step closer in this direction

95

To what extent is a database normalized?

• Normalization is a matter of degree• Measured in what is called “normal forms” (NF)

• 1NF, 2NF, 3NF, etc., higher NF = more normalized• 3NF Good enough for most applications• BCNF Boyce-Codd NF (more robust version of 3NF)

Mostly of academic interest (and complex applications):

• 4NF, 5NF or PJNF (Project Join), DKNF (Domain-Key) More advanced theoretically, little practical use Useful for research and formal methods only

Normal Forms

96

Q: What’s wrong with this table?

A: Data in PayDate & Amount fields not single-valued—i.e., they have repeating values

97

Similar Table, Same Problem

A: repeating values for a PK value PK is duplicate

98

First Normal Form (1NF)

• A “TABLE” is in 1NF if there are no multi-valued attributesand no PK is duplicated

• i.e., attributes are “atomic”

• A “DATABASE” is in 1NF if ALL its tables are in 1NF

99

Decomposition to 1NF:

Create a separate table where the repeating values can be recorded as rows

100

Decomposition

101


A: Some data in the Client and OrderDate fields are entered twicei.e., some non-key data are redundant

i.e., there are “partial dependencies” in the table (see next slide)

102

Functional Dependencies

• An attribute B is functionally dependent on attribute A if the value of a valid instance of attribute A uniquely determines the value of attribute B

• Represented as:

A B

103

Functional Dependency Examples

StudentID StudentName

StudentID StudentMajor

What are the functional dependencies in this relations?

Clients (ClientID, ClientName, City, State, Zip)

LineItems (OrderNo, LineItem, ClientID, ProdID, Qty)

104

Second Normal Form (2NF)

• Applies to tables with “composite” PKs (i.e., PK has more than one attribute)

• A “TABLE” is in 2NF if(1) it is in 1NF, and (2) non-key attributes are functionally dependent on the whole PK, not on just part of it (i.e., no partial dependencies)

• Note: we only need to worry about 2NF when PK contains more than one attribute (i.e., “composite”)

• That is: if a table is in 1NF and has a single PK, it is automatically in 2NF

• A “DATABASE” is in 2NF if ALL its tables are in 2NF

105

Decomposition to 2NF

Move the partial key (e.g., OrderNo) and the fields that are functionally dependent on only that part of the key (e.g., ClientID, OrderDate) to a separate table and make that partial key the PK in that new table

106

Decomposition

107


A: Some of the data in the ClientCity field is redundant, because once we know who the ClientID is, we know the city where they live

i.e., there are “transitive dependencies” in the table

108

Transitive Dependencies

• If a non-key attribute C is functionally dependent on another non-key attribute B (BC) and B is in turn dependent on the PK attribute A (AB)this implies C is transitively dependent on A (AC) (through B or ABC), which will cause redundancies

• In 2NF, all non-key attributes are functionally dependent on the PK

• Thus, in a 2NF table, a transitive dependency will occur every time there is a functional dependency between any two non-key attributes.

109

Transitive Dependency Examples

OrderNo ClientID ClientName

CourseNo InstructorID InstructorName

Are there transitive dependencies in these relations?

LineItems (OrderNo, LineItem, ProdID, Qty)

LineItems (OrderNo, LineItem, ProdID, ProdName, Qty)

110

Third Normal Form (3NF)

• A “TABLE” is in 3NF if (1) it is in 2NF and (2) non-key attributes depend on the PK and nothing else

• That is, non-key attributes are NOT functionally dependent on other non-key attributes (just on the PK)

• In other words, there are no transitive dependencies• A “DATABASE” is in 3NF if ALL its tables are in 3NF

111

Decomposition to 3NF:

Move the fields with transitive dependencies to a separate table

112

Decomposition

113

In Summary

• 1NF = no multi-value attributes (or no PK duplicates)

• 2NF = 1NF + the “whole” PK, not just part of it

• 3NF = 2NF + the PK and “nothing but” the PK

• Important! it is OK to have non-normalized designs, and some database applications may actually require a non-normalized design, but you must have an understanding of which normalization form you are violating and a good reason for doing it

114

Exercises

Text p.203, problem 3: Indicate the normal form (PK underlined) and decompose to 3NF

Class (CourseNo, SectionNo, RoomNo)Class (CourseNo, SectionNo, RoomNo, Capacity)Class (CourseNo, SectionNo, CourseName, RoomNo, Capacity)

115

Exercises

POS System:

Indicate the normal form (PK underlined) and decompose to 3NF

Sales (SaleNo, ClientID, ClientName, SaleDate, SaleAmount)

SalesDetails (SaleNo, LineItem, SaleDate, ProdID, ProdName, Qty)

Other Systems:

VideoRental (VideoNo, Date, MovieID, MovieName, ClientID)

VideoRental (VideoNo, Date, ClientID, CheckoutDate, RentalDays)

Videos (VideoNo, MovieID, MovieName, MovieType)

Videos (VideoNo, MovieID, VideoCondition)

Movies (MovieID, MovieName, MovieType, Producer, ReleaseDate)

116

ExerciseIndicate the normal form and decompose to 3NF

117

FYI,

Conceptually, normalization can be thought of the opposite of a SELECT SQL query. When you normalize, you decompose a large table into simpler, smaller tables without redundancies. In contrast, when you query several small tables, the result is a larger table in which redundancies don’t matter.

For example, the decomposed tables of the exercise in the prior page can be reconstructed by querying the normalized tables as follows:

SELECT Companies.CompanyID, CompanyName,

Employees.EmployeeID, EmployeeName,

Departments.DeptID, DeptName

FROM Departments, Companies, Employees

WHERE Companies.CompanyID = Employees.CompanyID

AND Departments.DeptID = Employees.DeptID

118

ExerciseIndicate the normal form and decompose to 3NF

(and then try to write an SQL query to re-construct the original table)

119

FYI Only

• Boyce-Codd Normal Form (BCNF): – A more robust version of 3NF– A database is in BCNF when the database is in 3NF when you

substitute the PK with any other Alternative Key– That is, the database is in 3NF for all Candidate Keys

• Domain-Key Normal Form (BKNF): – All values entered in an attribute satisfy the constraints defined in

the domain of that attribute– An attribute’s domain is the pool of data from which the attribute

can draw its values– Example: if we define a constraint for the OrderID attribute

(e.g., 6 digits, from 000001 to 999999) in general (i.e., the domain), the OrderID attribute in every table that uses this attribute, must satisfy the same constraints.

AUTransitional

Artifact:The CRUD Matrix

Connecting Data Objects to

Use Cases

121

Data Objects

• A data object is a person or thing you want to collect data for:

• In a database application a data object is a table

Examples: courses, students, clients, invoices, orders, deliveries

122

Identifying Data Objects

To identify data objects, refer to the Use Cases

(or other requirements artifacts) and:

• Identify and highlight (or bold face) all nouns• Inspect these nouns to see if they represent possible system data

objects• But be careful, a noun may not refer to a data object, but simply to

an attribute of a data object• A data object maps to a class (in a class diagram), entity (in a data

model) or table (in a database)• A data object has attributes (and behaviors if object is for a class)• An attribute is something you want to record about a data object• For example, in Students (StudentID, Name, SSN, Email)—Students

represents a data object and the data inside the parenthesis represents attributes of that data object

123

The CRUD Matrix

• A “transitional artifact” is one that helps establish a relationship or cross reference between artifacts

• A CRUD matrix is a transitional artifact between Use Cases and Data Objects

• Helps ensure that the Use Cases specified have all the necessary Data Objects to handle the data needs of the application and, conversely, that the collection of Data Objects identified cover the entire functionality specified in the requirements.

• The Use Cases, if properly specified, must describe all the actions necessary to maintain all data objects

• A CRUD matrix is a table that cross references which Use Cases: (C)reate, (R)ead, (U)pdate and/or (D)elete data in these objects

124

Developing a CRUD Matrix

• The CRUD matrix has one row for every data object identified and one column for every Use Case specified (or the other way around)

• So, first create a column (or row) for every Use Case in your model

• Every noun highlighted in the Use Cases will suggest the need for data object to store the respective data you, so you need to create a row (or column) for each of these data objects.

• Then go through every cell in the first Use Case and enter a C, R, U and/or D on the cell depending on whether the Use Case is creating, reading, updating or deleting records in the respective data object.

• The data objects should give you an indication of the entities (i.e., database tables) that you will need in your Data Model (and database)

• And the C’s, R’s, U’s and D’s should give you an idea of the SQL queries that your application will need

125

Illustration

UC-101 UC-102 UC-103

Table 1 C R

Table 2 U

Table 3 D

• UC-102 reads data from Table 1 It will require an SQL SELECT query

• UC-101 creates a record in Table 1 It will require an SQL INSERT query

• UC-103 deletes records data from Table 3 It will require an SQL DELETE query

• UC-102 updates data in Table 2 It will require an SQL UPDATE query

126

CRUD Matrix Example for a Loan Processing Application

Use Case

Data Object

Submit a Loan Request

Evaluate a Loan Request

Book a Loan

Applicant C

Loan Application C R

Credit Score C R

Credit Report C R

Account History C R

Loan Request C R,U R

Loan Officer R

Evaluation C R

Loan Agreement R

Loan Account C

Loan Clerk R

In a database application, these are tables and these are queries

127

ATM Application Example

128

ATM Use Case

Use Case ID UC-100

Use Case Withdraw Funds

Actors (P) Customer

Description The customer inserts card in the ATM, logs in with a pass code, and makes a selection from the available choices to withdraw funds. Once in the funds withdrawal screen, the customer is prompted to enter the amount to withdraw. After the amount is entered, the system will check for availability of funds for that customer. Provided that funds are available, the system will dispense the amount requested in cash and then debit that amount from the customer’s bank account. The system will record the last withdrawal date in customer’s file and record transaction in ATM transaction log .

Priority

Non-Functional Requirements

Assumptions

Source

129

ATM Use Case

Use Case ID UC-101

Use Case Deposit Funds

Actors (P) Customer

Description The customer inserts card in the ATM, logs in with a pass code, and makes a selection from the available choices to deposit funds. Once in the funds deposit screen, the customer is prompted to enter the amount to deposit. After the amount is entered, deposit slot door opens, customer places deposit envelop in slot, deposit slot door closes. The system credits the customer’s account accordingly, records the last deposit date in the customer’s file and record the transaction in ATM transaction log.

Priority


Assumptions

Source

130

ATM Use Case

Use Case ID UC-102

Use Case Transfer Funds

Actors (P) Customer

Description The customer inserts card in the ATM, logs in with a pass code, and makes a selection from the available choices to transfer funds. Once in the funds transfer screen, the customer is prompted to enter the amount to transfer, from account and to account. After the information is entered, the checks for availability of funds. If funds are available, it displays the transaction and asks for confirmation. The customer confirms transaction and the customer’s account gets adjusted accordingly. The system records the last funds transfer date in the customer’s file and records the transaction in ATM transaction log.

Priority


Assumptions

Source

131

ATM Use Case

Use Case ID UC-103

Use Case Balance Inquiry

Actors (P) Customer

Description The customer inserts card in the ATM, logs in with a pass code, and makes a selection from the available choice to inquire balances. The machine prints balances, records the last balance inquiry date in the customer’s file and records the transaction in ATM transaction log .

Priority


Assumptions

Source

132

ATM System’s CRUD Matrix

Use Case

Data Object

Withdraw Funds

Deposit Funds

Transfer Funds

Inquire Balances

ATM R,U

ATM Transaction Log C U U U

Customer File R,U R,U R,U R,U

Customer Account R,U U R,U R

Customer Transactions C U U

Documents

A U Business Data Design Professor J. Alberto Espinosa Business Analysis and Data Design ITEC-630 Fall 2008