SQL Tuning Ohio Oracle User’s Group October 2002 © Copyright, Kris T. Mason, 2002

SQL TuningOhio Oracle Users GroupOctober 2002 Copyright, Kris T. Mason, 2002

Kris T. MasonInformation Control Corp (ICC) Consultant10+ years with OracleOracle 7x, 8x, 9i70% Application 30% ProductionCSCC Adjunct FacultyOracle ProgrammingComputer Information Systems Concepts

IntroductionOverview of the SQL Tuning processDatabase InternalsSchema Objects involved with queriesSQL Optimizer and Execution PlansApplication construction

Why Tune?

PhilosophyBe a Good NeighborConsume only those resources you needRelease resources you are not usingThe more you know ...

Reality CheckApplications can not treat the database as a black-box and expect optimal performance. They must contribute to efficiency.Overall efficiency can be gained by tuning small, frequently executing queries as well as large infrequent onesTuning cant overcome bad designTune SQL before tuning the DB InstancePolitics, Balance, Judgement

Preparing to tuneProgrammingLanguagesApplication segmentationBind VariablesServer ConfigurationHardware (CPUs, Memory, Disks, Network)Non-Database applicationsDatabase ConfigurationInit.oraGather statistics select value, name from v$sysstat order by name;Analyze Tables & Indexes (DBMS_STATS package)DB Links and ReplicationKnow where you are, want to be and how to get thereCurrent performanceWhat is acceptable performanceHow long it will take

MinimizingLogical ReadsDisk ReadsTemp SpaceMemorySortsParsingNetwork TrafficApplication Round TripsResponse Time (Perceived & Real)

Database InternalsSystem Global Area (SGA)Database Block Buffers Block Size usually 8k - 16kHolds portions of Tables and IndexesMinimizes Expensive Disk ReadsV$DB_CACHE_ADVICEBuffer Cache Hit RatioLibrary Cache Dictionary and recent SQLTwo SQL statements are identical only if they match byte for byteProgrammers and Dynamic Query Rewrite (Cursor Sharing) can force queries to be identical via bind variablesIdentical Queries accumulate statistics in V$SQLAREA providing better visibility to statement use

Oracle FeaturesDynamic Query Rewrite (Cursor Sharing)Parallel QueryOptimizer ModesHintsAlter Session statementTable & Index PartitioningStatspack

Dynamic Query Rewrite(Cursor Sharing)Init.oraQUERY_REWRITE_ENABLED=TRUEQUERY_REWRITE_INTEGRITY=TRUSTEDTurns literal strings and values into bind variablesBypasses the programmerDifferent queries become identicalWHERE STATUS = 1 AND NAME = SMITHWHERE STATUS = 3 AND NAME = JONESWHERE STATUS = :b1 AND NAME = :b2Statistics are accumulated under the single query

Parallel QueryAllows multiple CPUs to work parts of the same query (divide and conquer)My experience:Best when 6+ CPUsCan be over done, causing different concurrently executing queries to compete for resourcesConfigure PQ Servers (#CPUs 1)*2

OptimizerRule / Choose / CostGenerates alternative execution plansPlans can be seen and understoodSET AUTOTRACE ONSET AUTOTRACE TRACEONLY

Primary Schema ObjectsTablesFull Table ScanRandom Data Block ReadROWID Direct AccessIndexIndex Organized TableIndexesB*Tree / Bitmap / Index-Only Table / Function BasedIdeal when highly selectiveIndex Only Processing avoids Data Block ReadEntirely NULL values are not stored in the indexALTER INDEX REBUILDTables & Indexes require Data block reads if not in memory already

IndexesSelectivity = (# of Distinct Values / # of Rows) * 100Selectivity = 0% when all the values have the same valueSelectivity = 100% when all the values have different values (PKs)Indexes loose effectiveness when Selectivity falls below 80%Indexes gain effectiveness when Selectivity approaches 100%

Index use can be invalidated (Index Invalidation) when the column reference in the WHERE is involved with a functionSELECT ... WHERE ID = 5 /* Uses an index */SELECT ... WHERE ID + 0 = 5 /* Does not use an index */SELECT ... WHERE NAME BETWEEN SMITH AND THOMAS /* Uses an index */SELECT ... WHERE UPPER(NAME) BETWEEN SMITH AND THOMAS /* Does not use an index */

Secondary Schema ObjectsViewsStored SELECT statementSome Hints do not propagate inside a complex viewTriggersPL/SQL codeRegistered against a specific tableFired upon specific DML events

Tuning ProcessFind SQLSQL Analyze & Top SQLTop SessionsSQL*Plus & SQL Worksheet V$SQLAREALook ForHigh Disk Reads vs. Low Rows ReturnedHigh ExecutionsDISTINCTIdentify ProblemSelectivityAccess PathJoin Method and OrderConsider Possible AlternativesHintsIndexesDocument, Measure & Test AlternativesShow your workDeploy SolutionGet credit for your work

Finding SQL to tuneV$SQLAREAset linesize 1000set pagesize 1000column disk_reads format 999,999,999,999column executions format 999,999,999,999column rows_processed format 999,999,999,999column sql_text format a800 word_wrappedselect executions, disk_reads, rows_processed, sorts, sql_text from v$sqlarea where disk_reads > 3000 order by disk_reads DESC;

Look for high resource consumers

DISK_READSEXECUTIONSSORTS

Limited to 1000 bytes of SQL Statement including white space. Be sure you have the whole statement!The entire statement exists in v$sqltext but it is broken into pieces

Helper PL/SQL FunctionsCREATE OR REPLACE FUNCTION SQUISH (P in varchar2) return varchar2 asI number;V varchar2(32000);Begin/* Change special chars to blanks */V := trim(rtrim(translate(P,chr(9)||chr(10)||chr(13),' '))); /* One blank between quotes */Loop I := instr (V,' '); /* Two blanks between quotes */ Exit when I = 0;/* V := replace two blanks with one */ V := replace (V, , );End loop;Return V;End Squish;

CREATE OR REPLACE FUNCTION CAT_SQL_TEXT (P in varchar2) return varchar2 as V varchar2(32000) := ''; Beginfor C in (select sql_text from v$sqltext where address = P order by piece) Loop V := V || C.sql_text;End loop;Return V;End cat_sql_text;

Grant select on v$sqltext to [function owner];

Finding SQL To Tune with Helper FunctionsV$SQLTEXTset linesize 1000set pagesize 1000column disk_reads format 999,999,999,999column executions format 999,999,999,999column rows_processed format 999,999,999,999column sql_text format a800 word_wrapped

select address, hash_value, executions, disk_reads, rows_processed, sorts, squish(cat_sql_text(address)) sql_text from v$sqlarea where disk_reads > 3000 order by disk_reads DESC;

Finding SQL to tuneDISTINCT

EXECUTIONS DISK_READS ROWS_PROCESSED SORTS SQL_TEXT------------- ------------- ---------------- ------- ---------------------------------------------------- 81 3,499 5,103 81 SELECT DISTINCT TABLE_B.NAME FROM TABLE_A, TABLE_B WHERE ( TABLE_A.ID = TABLE_B.ID )

The Intent of this query is to show the names in Table_B that have any Matching rows in Table_A

OriginalDocument the query as originally writtenSELECT DISTINCT TABLE_B.NAMEFROM TABLE_A, TABLE_BWHERE ( TABLE_A.ID = TABLE_B.ID );

Execution Plan---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=CHOOSE (Cost=921 Card=306 Bytes=5814) 1 0 SORT (UNIQUE) (Cost=921 Card=306 Bytes=5814) 2 1 HASH JOIN (Cost=33 Card=73605 Bytes=1398495) 3 2 TABLE ACCESS (FULL) OF TABLE_B' (Cost=1 Card=307 Bytes=4298) 4 2 INDEX (FAST FULL SCAN) OF 'FK_TABLE_A_ID' (NON-UNIQUE) (Cost=14 Card=73605 Bytes=368025)

Original StatisticsStatistics---------------------------------------------------------- 43 recursive calls 8 db block gets 247 consistent gets 0 physical reads 0 redo size 1368 bytes sent via SQL*Net to client 591 bytes received via SQL*Net from client 6 SQL*Net roundtrips to/from client 5 sorts (memory) 0 sorts (disk) 63 rows processed

AlternativeDocument each version of the query as rewrittenSELECT TABLE_B.NAME FROM TABLE_BWHERE EXISTS ( SELECT 1 FROM TABLE_A WHERE TABLE_A.ID = TABLE_B.ID );

Execution Plan---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=16 Bytes=224) 1 0 FILTER 2 1 TABLE ACCESS (FULL) OF TABLE_B' (Cost=1 Card=16 Bytes=224) 3 1 INDEX (RANGE SCAN) OF 'FK_TABLE_A_ID' (NON-UNIQUE) (Cost=4 Card=1169 Bytes=5845)

Alternatives StatisticsStatistics---------------------------------------------------------- 0 recursive calls 4 db block gets 638 consistent gets 0 physical reads 0 redo size 1409 bytes sent via SQL*Net to client 591 bytes received via SQL*Net from client 6 SQL*Net roundtrips to/from client 2 sorts (memory) 0 sorts (disk) 63 rows processed

DocumentationOriginal versionReformed versionRewrite by hand in a form comfortable to youMake sure it is equivalent to the originalMatch querys joins to data modelEach alternativeA record of your workRecommended solution

TestingFull volume production testingTest or Personal copy of DatabaseDBMS_STATSImport Statistics from Production

What Can Be Done?Analyze Tables & IndexesHistogramsSee Oracles Execution PlanSET AUTOTRACE [ON, TRACEONLY]PLAN tableQuery Costs (Oradollars)Hints You know more than the Optimizer!Optimizer GoalFIRST_ROWSALL_ROWSAccess PathFULLINDEXQuery TransformationORDEREDSTARJoin OperationUSE_NLUSE_MERGEUSE_HASHDRIVING_SITELEADING

Execution PlansA roadmap for executing the queryTop to bottomInside to outsideExecution TechniquesFull Table Scan Not necessarily a bad thingUnique Index ScanIndex Range ScanConcatenationSortAggregationOrder ByGroup ByNested LoopsMerge JoinHash JoinIN-List Iterator

HintsSELECT /*+ hint goes here */ ...Features:Syntax errors are not reported and not usedSpecify the table alias not the table namePut preceding and trailing blankYou can use multiple hintsHints dont have to be used by the optimizer

Other Techniques to Optimize PerformanceALTER SESSION SET SORT_AREA_SIZE=1000000SET ROLLBACK_SEGMENT=RBS_BIGMinimized Chained RowsHow full are the data blocks? Random Deletes reduce efficiency of table scansUse Plan Stability to insure optimizer does not change the plan over time

Distributed QueriesQueries across multiple instancesVia a DB_LinkCreate a view of the remote tablesMinimize round-trips to the remote DBPull data to the local DB via a snapshotWire the local & remote servers together

Effects of tuning

Chart1

2440031

2755646

3002076

3128057

3255563

2732933

4977331

4607288

4439704

3050420

3216624

3359287

1461685

3972817

4634971

2480144

2078049

2247093

2387418

1217823

1100976

1460628

473575

1888641

2022442

2158865

1594354

3351965

1093388

2400641

729951

858740

1300059

939603

418737

654228

491860

628695

767432

400953

726420

988249

667529

470863

780128

940692

1005888

842897

2454981

1120904

1155951

1318518

1473361

896222

416322

931174

535284

361329

511517

664176

509920

853205

1550388

1318985

372583

547183

733474

656377

350939

473671

857583

551996

730219

906466

737113

1279820

498207

1031463

349448

618190

803809

229606

839138

755849

741377

666703

855536

1040570

2373227

58412

5798005

922272

552873

797905

985632

849963

623027

896700

698837

466540

650848

836111

542713

1008870

786579

1680805

1457768

1644372

1830549

889365

797671

1506566

1780922

Time

Reads

Physical Reads

Disk Reads

6/12/02 20:002,440,031

6/13/02 20:002,755,646

6/14/02 20:003,002,076

6/15/02 20:003,128,057

6/16/02 20:003,255,563

6/17/02 20:002,732,933

6/18/02 20:004,977,331

6/19/02 20:004,607,288

6/20/02 20:004,439,704

6/21/02 20:003,050,420

6/22/02 20:003,216,624

6/23/02 20:003,359,287

6/24/02 20:001,461,685

6/25/02 20:003,972,817

6/26/02 20:004,634,971

6/27/02 20:002,480,144

6/28/02 20:002,078,049

6/29/02 20:002,247,093

6/30/02 20:002,387,418

7/1/02 20:001,217,823

7/2/02 20:001,100,976

7/3/02 20:001,460,628

7/4/02 20:00473,575

7/5/02 20:001,888,641

7/6/02 20:002,022,442

7/7/02 20:002,158,865

7/8/02 20:001,594,354

7/9/02 20:003,351,965

7/10/02 20:001,093,388

7/11/02 20:002,400,641

7/13/02 20:00729,951

7/14/02 20:00858,740

7/15/02 20:001,300,059

7/16/02 20:00939,603Murali ran several queries that used 7M disk reads These have been subtracted out

7/17/02 20:00418,737

7/18/02 20:00654,228

7/19/02 20:00491,860

7/20/02 20:00628,695

7/21/02 20:00767,432

7/22/02 20:00400,953

7/23/02 20:00726,420

7/24/02 20:00988,249

7/25/02 20:00667,529

7/26/02 20:00470,863

7/27/02 20:00780,128

7/28/02 20:00940,692

7/29/02 20:001,005,888

7/30/02 20:00842,897

7/31/02 20:002,454,981Roger was running statistical analysis without debugging in Desi first. No adjustment made to #

8/1/02 20:001,120,904

8/2/02 20:001,155,951

8/3/02 20:001,318,518

8/4/02 20:001,473,361

8/5/02 20:00896,222

8/6/02 20:00416,322

8/7/02 20:00931,174

8/8/02 20:00535,284

8/9/02 20:00361,329

8/10/02 20:00511,517

8/11/02 20:00664,176

8/12/02 20:00509,920

8/13/02 20:00853,205

8/14/02 20:001,550,388

8/15/02 20:001,318,985

8/16/02 20:00372,583

8/17/02 20:00547,183

8/18/02 20:00733,474

8/19/02 20:00656,377

8/20/02 20:00350,939

8/21/02 20:00473,671

8/22/02 20:00857,583

8/23/02 20:00551,996

8/24/02 20:00730,219

8/25/02 20:00906,466

8/26/02 20:00737,113

8/27/02 20:001,279,820

8/28/02 20:00498,207

8/29/02 20:001,031,463

8/30/02 20:00349,448

8/31/02 20:00618,190

9/1/02 20:00803,809

9/2/02 20:00229,606

9/3/02 20:00839,138

9/4/02 20:00755,849

9/5/02 20:00741,377

9/6/02 20:00666,703

9/7/02 20:00855,536

9/8/02 20:001,040,570

9/9/02 20:002,373,227

9/10/02 20:0058,412

9/11/02 20:005,798,005

9/12/02 20:00922,272

9/13/02 20:00552,873

9/14/02 20:00797,905

9/15/02 20:00985,632

9/16/02 20:00849,963

9/17/02 20:00623,027

9/18/02 20:00896,700

9/19/02 20:00698,837

9/20/02 20:00466,540

9/21/02 20:00650,848

9/22/02 20:00836,111

9/23/02 20:00542,713

9/24/02 20:001,008,870

9/25/02 20:00786,579

9/26/02 20:001,680,805

9/27/02 20:001,457,768

9/28/02 20:001,644,372

9/29/02 20:001,830,549

9/30/02 20:00889,365

10/1/02 20:00797,671

10/2/02 20:001,506,566

10/3/02 20:001,780,922

10/4/02 20:00

10/5/02 20:00

Disk Reads

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

Time

Reads

Physical Reads

Network

17/31/02 20:002,091,902256,325,1502,197,206,494

18/1/02 20:002,239,260282,333,5712,063,379,523

18/2/02 20:002,207,351270,428,4042,090,123,662

18/3/02 20:003,815,472447,995,3213,817,521,664

18/4/02 20:005,408,174623,036,4735,536,644,656

18/5/02 20:002,212,273268,739,6112,132,318,108

18/6/02 20:002,127,845260,046,4912,046,233,540

18/7/02 20:002,209,658274,765,8572,148,478,896

18/8/02 20:002,170,982266,005,5772,144,696,112

18/9/02 20:002,168,244271,212,4922,130,432,980

18/10/02 20:003,771,721447,292,8033,861,970,642

18/11/02 20:005,381,314624,133,4775,597,558,784

18/12/02 20:002,456,341314,562,8182,162,479,117

18/13/02 20:002,214,900276,328,0762,134,099,931

18/14/02 20:002,381,743292,092,2212,370,914,263

18/15/02 20:002,204,978266,998,9672,494,680,428

18/16/02 20:002,211,136269,789,2262,264,446,668

18/17/02 20:003,963,588469,882,9324,216,100,542

18/18/02 20:005,650,599659,571,0506,143,883,169

18/19/02 20:002,240,032274,784,5692,324,785,124

18/20/02 20:002,159,883261,385,8712,233,512,731

18/21/02 20:002,162,317264,297,4262,214,789,552

18/22/02 20:002,226,966276,852,4092,294,296,996

18/23/02 20:002,247,475274,817,4072,314,773,070

18/24/02 20:003,985,713467,509,3654,264,394,853

18/25/02 20:005,725,333658,426,0906,199,726,285

18/26/02 20:002,205,118266,720,0922,237,353,398

18/27/02 20:002,243,834274,945,3942,343,590,402

18/28/02 20:002,207,405271,316,1422,254,007,953

18/29/02 20:002,444,890318,179,2972,332,966,736

18/30/02 20:002,460,768301,712,3752,411,194,958

18/31/02 20:004,327,800513,699,0334,394,270,919

19/1/02 20:006,088,029706,755,0536,400,283,288

19/2/02 20:001,745,520191,989,3552,006,168,562

19/3/02 20:002,440,004301,463,0272,407,361,517

19/4/02 20:002,347,529288,013,1572,358,188,928

19/5/02 20:002,359,229282,060,7952,367,129,485

Network

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

Network Bytes Sent Server to Client

Sheet3

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

Network Bytes Sent Client to Server

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

Network Round Trips (SQL*Net)

ApplicationsStart with optimized queries so when code is cloned it has a better chance of working wellReview 3rd party produced code such as PowerBuilderWrite code for the proper platform (application segmentation)PL/SQL is not cool but it is fast and effectiveUse Bind Variables whenever possibleAvoid Dynamic SQL if possible

ReferencesOracles TechnetDatabase Performance Tuning Guide & Referencehttp://otn.oracle.com/docs/products/oracle9i/doc_library/release2/server.920/a96533/toc.htm

www.iccohio.com

[email protected]

Summary

SQL Tuning is not necessarily hard, it just takes some practiceand the willingness to try

Documents

SQL Tuning Ohio Oracle User’s Group October 2002 © Copyright, Kris T. Mason, 2002