46
SQL Tuning Briefing Null is not equal to null but null is null

SQL Tuning Briefing Null is not equal to null but null is null

Embed Size (px)

Citation preview

Page 1: SQL Tuning Briefing Null is not equal to null but null is null

SQL Tuning Briefing

Null is not equal to null but null is null

Page 2: SQL Tuning Briefing Null is not equal to null but null is null

What are “Best Practices” ?• A set of guiding principles proposed by industry leading experts• Acknowledgement that the application is part of an evolving lifecycle

which will continue to iterate over a period of time• Determination to deliver a quality product or what we used to call “taking pride in your work”!

• Objectively and consistently leads to: – better structured, more robust code– more accurate code– better performing code– better maintained code– more readable code

• Much less reliance on individual developers• Repeatability and measurability

Page 3: SQL Tuning Briefing Null is not equal to null but null is null

Single Developer Best Practices

Page 4: SQL Tuning Briefing Null is not equal to null but null is null

Development team best practice

One-tenth of a second is about the limit for having the user feel that the system is reacting instantaneously

Team Developers ProcessPrepare

Write & Compile

Test Code

Debug

Code Review

Optimize SQL

USDevelopment

Team

HKDevelopment

Team

ZHA/SHADevelopment

Team

Team Review

ScheduledCode Review Benchmark test& SQL Scan

Manager

ManagementReporting

Bad Code or Problematic SQL Detected:

Return to Development

Pre-deploymentQA: Regression

Testing and Scalability/

Performance Tuning

Code Review

Successful

ProductionDatabase

VersionControl

(Team Coding)

Page 5: SQL Tuning Briefing Null is not equal to null but null is null

ChallengesChallengesHigher speed means more business

Less resource consumption means more concurrent users

ProblemsProblems

The challenges and problems of application

A Constantly-changing Environment

Page 6: SQL Tuning Briefing Null is not equal to null but null is null

What affects performance? Hardware

– Disk space– RAM– Network– CPU

Operating System settings Database Server parameter settings Database Design Indexes on Database Tables SQL statement

Page 7: SQL Tuning Briefing Null is not equal to null but null is null

Why SQL statement affecting performance

90%

60%

Page 8: SQL Tuning Briefing Null is not equal to null but null is null

40%

60%

Performance Problems

Application

Others

According to industrial experts 60% of the database’sperformance problems are caused by applications

Page 9: SQL Tuning Briefing Null is not equal to null but null is null

DBMS Optimize - Find the best way to process your SQL Statements

Very Good

(sometime for simple SQL)

Good

(Normally, but still be improved)

Very Bad

(most of time for complex SQL)

Page 10: SQL Tuning Briefing Null is not equal to null but null is null

Why does SQL need tuning?

It is easy to write functional SQL.

It is harder to write efficient, high performing SQL.

Page 11: SQL Tuning Briefing Null is not equal to null but null is null

Why is it so hard?

your database structure how SQL statements work in general how Database executes SQL database changes over time

You must know

Then it’s trial and error until you get a satisfactory result.

Page 12: SQL Tuning Briefing Null is not equal to null but null is null

Why don’t people tune their SQL?

Too busy now. I’ll do it later.

Is that what the Oracle optimizer is for?

I’m a Java, not an SQL, programmer.

I don’t know how.

Generated by Toplink, not my business.

It’s works. I’ve got my data. I’m happy.

Page 13: SQL Tuning Briefing Null is not equal to null but null is null

Generate SQL code based on its own object-oriented functions

Toplink will transform the object-oriented requests into object-liked SQL statements

Relational based SQL engine like Oracle to

execute an object-liked SQL statements may cause unprecedented large system resources utilization and bad system performance

Top Link problem

Page 14: SQL Tuning Briefing Null is not equal to null but null is null

When is the best time to tune SQL?

At the time it is written

As the database changes

Page 15: SQL Tuning Briefing Null is not equal to null but null is null

Server receives SQL

Parses

Oracle optimizer determines the execution path

Binds variables

Executes

How does Oracle process SQL?

Page 16: SQL Tuning Briefing Null is not equal to null but null is null

How Oracle Optimizer works

Internal rewrite & generation of

multiple execution plans

Plan 1 cost=1000

Plan 2 cost=3000

Plan 3 cost=4000

Plan 4 cost= 500 Execution

How accurate isthe cost estimation?

Plan 1

Plan 2

Plan 3

Plan 4

Cost estimation

Does it try every possible way to

rewrite your SQL?SQL

Page 17: SQL Tuning Briefing Null is not equal to null but null is null

Limitations of Optimizer

Limited SQL rewrite Resource limitation Cost estimation is only an

estimation Problems too complex for optimizer

Page 18: SQL Tuning Briefing Null is not equal to null but null is null

Basic Concept of SQL Execution - Driving Path

SELECT * FROM A, B, C WHERE A.key1 = B.key1 AND B.key2 = C.key2 AND C.f2 = 0 AND A.f1 = 0

Driving Path : C→B→A

Page 19: SQL Tuning Briefing Null is not equal to null but null is null

Driving Path : C→B→A

Driving Path : A→B→C

Page 20: SQL Tuning Briefing Null is not equal to null but null is null

Why join path matter

Path from table A to table B: which means that we open table A, looking at each row to then use an index to search for matching rows in table B:

select * from A, B /* table A has 100,000 records */ where A.key = B.key /* table B has 1,000 records */

Path from B→A is around 59 times faster than the speed of A→B

Path from table B to table A: which means that we open B table, looking at each row to then use an index to search for matching rows in table A:

– Number of Operations (A→B) = 100,000 * RoundUp(LN(1000) / LN(2)) / 2 = 100,000 * 10 / 2 = 500,000

– Number of Operations (B→A) = 1000 * RoundUp(LN(100,000) / LN(2)) / 2 = 1000 * 17 / 2 = 8,500

Page 21: SQL Tuning Briefing Null is not equal to null but null is null

Why join path matter

500,000 operations 8500 operations

FullTable Scan

A

BB Index

ScanAIndex Scan Full Table

Scan

Page 22: SQL Tuning Briefing Null is not equal to null but null is null

Specific Tips for writing SQL

Page 23: SQL Tuning Briefing Null is not equal to null but null is null

Use table join in place of sub-queryIf A,B is many to one or one to one relationship

Replace

Select * from A

where A.CITY in

(select B.City from B)

With

Select A.* from A, B

where A.CITY = B.CITY

Page 24: SQL Tuning Briefing Null is not equal to null but null is null

Select the smallest table and/or smallest result set first.

SELECT * FROM A,B WHERE A.STATE = “CA”

AND B.CITY =“CONCORD”

Joins

Page 25: SQL Tuning Briefing Null is not equal to null but null is null

If A is a large table and B is small. Small table should drive the large table. Disable index on B. This changes the table driving path.

Replace SELECT * FROM A,B

WHERE A.STATE = B.STATE

With

SELECT * FROM B,AWHERE A.STATE = B.STATE || ''

Joins—Change table driving path

Page 26: SQL Tuning Briefing Null is not equal to null but null is null

For some complex SQL(Group function, Set function), View’s SQL is normally cannot be merged to your upper layer SQL.

So, Oracle normally processes the view first, creates a temp table which does not have any indexes, and you cannot tune the view because Oracle always handles the view in the same manner.

Avoid Views

Page 27: SQL Tuning Briefing Null is not equal to null but null is null

Explain Plan

Analyze your Explain Plan. Always test the SQL statement to find

its actual time.

Page 28: SQL Tuning Briefing Null is not equal to null but null is null

Indexed Fields

Know your indexes and use them to your advantage.

Page 29: SQL Tuning Briefing Null is not equal to null but null is null

If you want the index used, don’t perform an operation on the field.Replace

SELECT * from A

where SALARY +1000 = :NEWSALARY

with

SELECT * from A

where SALARY = :NEWSALARY -1000

Indexed Fields

Page 30: SQL Tuning Briefing Null is not equal to null but null is null

Index will not be used when a function is used.

SELECT * from A

where substr(name, 1, 3) = 'Wil'

Indexed Fields

Page 31: SQL Tuning Briefing Null is not equal to null but null is null

WHERE clauseAvoid using

!= (not equal to)

Like '%SA%'

Indexed Fields

Page 32: SQL Tuning Briefing Null is not equal to null but null is null

Sometimes DO disable the index

SELECT * FROM A

WHERE SALARY + 0 = '10000'

AND DEPT = 'IT'

SELECT * FROM A

WHERE EMP_SEX || '' = 'm'

Indexed Fields

Page 33: SQL Tuning Briefing Null is not equal to null but null is null

Do not have default value set to NULL. If it is a number field and lowest value is 0, then:Replace

SELECT * FROM A

WHERE NUMBER IS NOT NULL

with (normally faster response time)

SELECT * FROM A

WHERE NUMBER >0

Indexed Fields

Page 34: SQL Tuning Briefing Null is not equal to null but null is null

Index FieldsReplace Outer Join with Union.If both A.State and B.State have a unique indexed:

Replace SELECT A.CITY, B.CITY FROM A,B

WHERE A.STATE=B.STATE(+)

WithSELECT A.CITY, B.CITY FROM A,B

WHERE A.STATE=B.STATE

UNION

SELECT NULL, B.CITY FROM B WHERE NOT EXISTS(SELECT 'X' FROM A

Where A.STATE=B.STATE)

Page 35: SQL Tuning Briefing Null is not equal to null but null is null

Learn to use optimization hints

Rule StarFirst_Row Star_TransformationAll_Rows Index_CombineOrdered MergeUse_Hash No_MergeUse_ConcatNo_ExpandParallelDriving_Site

Optimization Hints

Page 36: SQL Tuning Briefing Null is not equal to null but null is null

Use concatenated index.If A table is indexed with lastname, firstname.

Replace

SELECT * FROM A

WHERE lastname = 'Smith'

ORDER BY firstname

With

SELECT * FROM A

WHERE lastname = 'Smith'

ORDER BY lastname, firstname

ORDER BY clause

Page 37: SQL Tuning Briefing Null is not equal to null but null is null

Assume table A,B relationship is one to many.

The following statements have the same results.

SELECT * FROM AWHERE A.CITY IN

(SELECT B.CITY FROM B)

SELECT * FROM AWHERE EXISTS

(SELECT CITY FROM B WHERE A.CITY = B.CITY)

EXIST and IN Sub-query

Page 38: SQL Tuning Briefing Null is not equal to null but null is null

Use IN Sub-query

A.CITY is indexed, B.CITY is not indexed, and table B has much less rows than A.

SELECT * FROM AWHERE A.CITY IN

(SELECT B.CITY FROM B)

SELECT * FROM AWHERE A.CITY IN

(SELECT B.CITY||’’ FROM B)

A.CITY is indexed, B.CITY is indexed, and table B has much less rows than A.

Page 39: SQL Tuning Briefing Null is not equal to null but null is null

Use EXISTS Sub-query

A.CITY is not indexed and B.CITY is indexed, and B has more rows than A.

SELECT * FROM AWHERE EXISTS

(SELECT CITY FROM B WHERE A.CITY = B.CITY)

A.CITY is indexed and B.CITY is indexed, and table B has more rows than A.

SELECT * FROM AWHERE EXISTS

(SELECT CITY FROM B WHERE A.CITY||’’ = B.CITY)

Page 40: SQL Tuning Briefing Null is not equal to null but null is null

Variables Problemselect * from Employee

where

(Emp_id>:range_lower or :range_lower is null)

and (A.key<:range_upper or :range_upper is null)

To enable index range search with a input range boundaries.

select * from Employee

where

Emp_id>=nvl(:range_lower,min , Emp_id)

and A.key<=nvl(:range_upper, A.key)

Page 41: SQL Tuning Briefing Null is not equal to null but null is null

TransitivitySelect * from A,B,Cwhere A.key=B.keyand B.key=C.keyand C.key=‘text’

To make Oracle’s optimizer more intelligent by increasing the transitivity(as Oracle’s Optimizer cannot do very complicated transitivity improvement internally)

Select * from A,B,Cwhere A.key=B.keyand B.key=C.keyand A.key=C.keyand C.key=‘text’and A.key=‘text’and B.key=‘text’

Page 42: SQL Tuning Briefing Null is not equal to null but null is null

select * from Awhere (A.key1,A.key2) not in(select B.key1,B.key2 from B)

select * from Awhere (A.key1,A.key2) in

(select B.key1,B.key2 from B minus select A.key1,A.key2 from A)

Complex SQL Transformation

Sometimes the following transformation may give you surprise !

Page 43: SQL Tuning Briefing Null is not equal to null but null is null

Which SQL is best?

Number of Records

Re

sp

on

se

Tim

e

SQL1

SQL2

Page 44: SQL Tuning Briefing Null is not equal to null but null is null

select emp_name, dpt_name, grd_desc from employee, department DEPARTMENT1, grade where emp_grade = grd_id and emp_dept = dpt_id and EXISTS (SELECT 'X' from department DEPARTMENT2 WHERE dpt_avg_salary in (select min(dpt_avg_salary) from department DEPARTMENT3) AND dpt_id = EMPLOYEE.emp_dept)

How many ways can you rewrite this statement?

Page 45: SQL Tuning Briefing Null is not equal to null but null is null

1261 semantically equivalent SQL statements

Page 46: SQL Tuning Briefing Null is not equal to null but null is null

Thank You