The Best Way to Tune Your Parallel Statements: Real-Time ...€¦ · Real-Time SQL Monitoring •...

TheBestWaytoTuneYourParallelStatements:Real-TimeSQLMonitoring

YasinBaskanProductManager,ParallelExecuPonDataWarehouseDevelopmentSep20,2016yasin.baskan@oracle.com@yasinbaskan

SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirecPon.ItisintendedforinformaPonpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfuncPonality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andPmingofanyfeaturesorfuncPonalitydescribedforOracle’sproductsremainsatthesolediscrePonofOracle.

• Availablenow– ExadataExpressCloudService

• Comingsoon– DatabaseCloudServices– ExadataCloudMachine

AnnouncingOracleDatabase12cRelease2onOracleCloud

OracleispresenPngfeaturesforOracleDatabase12cRelease2onOracleCloud.WewillannounceavailabilityoftheOn-PremreleasesomePmeaZerOpenWorld.

Agenda

IntroducPontoReal-TimeSQLMonitoring

StepstoanalyzeaSQLMonitorreport

CommoncausesofsubopPmalperformanceinparallelstatements

IntroducPontoReal-TimeSQLMonitoring

Real-TimeSQLMonitoring•  IntroducedinOracleDatabase11gtomakeSQLtuningeasier•  ShowsdetailedinformaPonabouttheexecuPonofasingleSQLstatement– BothatrunPmeandaZerthestatementends

• MakesitveryeasytoidenPfycausesofperformanceproblemsbyshowingwherePmeisbeingspentintheexecuPonplan– NoneedtoqueryV$viewsorenableanytracing

•  EspeciallyusefulforparallelstatementswheretheexecuPonplanmaybemuchlonger•  ForguidanceonhowtogetSQLMonitorreportssee:–  hcp://www.oracle.com/technetwork/database/manageability/sqlmonitor-084401.html

Real-TimeSQLMonitoring

StepstoAnalyzeaSQLMonitorReport

HowtoanalyzeaSQLMonitorreport•  LookatwherethePmeisbeingspent– Time&waitstaPsPcs– FindtheplanstepdoingmostoftheacPvity•  “AcPvity%”column

– CheckiftheesPmatedcardinalitymatchestheactualcardinality•  “EsPmatedrows”and“actualrows”columns

• GatheringappropriateopPmizerstatsfixesmostoftheproblems– Extendedstats,histograms,etc…

CommonCausesofSubopPmalPerformanceinParallelStatements

Top3thingstobeawareofinparallelstatements• WorkdistribuPonacrossparallelprocesses• DatadistribuPonbetweenparallelprocesses• Degreeofparallelism(DOP)

WorkDistribuPonAcrossParallelProcesses

WhyisworkdistribuPonimportant?• MulPpleprocessesworkingforasingleparallelSQLstatement• UnevenworkdistribuPoncausessubopPmalperformanceandlowscalability– e.g.IncreasingtheDOPdoesnotmakethequeryrunfaster

WorkdistribuPonacrossprocesses•  LookattheParalleltabtoseetheworkdistribuPonacrossprocesses

UnevenworkdistribuPon

WorkdistribuPonacrossprocesses•  Lookatwhataspecificprocessisdoing•  TrytounderstandwhatcausesunevenworkdistribuPon

CommoncausesofunevenworkdistribuPon•  SubopPmaldatadistribuPonmethodchosenbytheopPmizer• Dataskewinthejoincolumns

CommonDataDistribuPonProblems

Producer/consumermodelanddatadistribuPon

ProducerConsumerDistribute

QueryCoordinator

SCANJOIN

SELECT*FROMCUSTOMERSORDERBYcust_last_name;

CommondistribuPonmethods• HASH– AppliesahashfuncPononthejoinkeysandsendstherowtoasingleconsumer

• BROADCAST– Sendsallrowstoeveryconsumer– Suitableforsmallerdatasets

HashdistribuPon

PX1cust_id

c.cust_id

cust_id

PX1andPX2scantheleZsidetable

PX1andPX2applyahashfuncPononthekeycolumnsanddistributetherowsbasedontheresulttoPX3andPX4

TableC

selectcount(*)fromc,swherec.cust_id=s.cust_id;

HashdistribuPon

PX1cust_id

c.cust_id s.cust_id

1000 1000

4000 4000

c.cust_id s.cust_id

3000 3000

2000 2000

cust_id

PX1andPX2scantherightsidetable

PX1andPX2applyahashfuncPononthekeycolumnsanddistributetherowsbasedontheresulttoPX3andPX4

TableS

BroadcastdistribuPon

PX1cust_id

cust_id

PX1andPX2scantheleZsidetable c.cust_id

PX4c.cust_id

PX1andPX2broadcastallrowstoPX3andPX4

TableC

cust_id

PX3andPX4scantherightsidetablecust_id s.cust_id

3000 3000

1000 1000

PX4cust_id s.cust_id

2000 2000

4000 4000

TableS

NodistribuPonfortherightside

CommondistribuPonproblems• OpPmizermaypickasubopPmaldistribuPonmethodbecauseofstale/missingstats– HashdistribuPonwithlowcardinality•  Causessomeprocessesdomorework

– BroadcastdistribuPonwithhighcardinality•  CauseshighnumberofrowstobesentacrossRACnodesandprocesses• MaycausehighPGAandtemporarytablespaceusage

HashdistribuPonwithlowcardinalityWorkdistribu2on

HashdistribuPonwithlowcardinalityWaitac2vityfordistribu2on

EsPmatedrowsvs.ActualRows HighwaitPmefordistribuPonHashdistribuPonasaresultofwrongcardinalityesPmates

HashdistribuPonwithlowcardinality• OpPmizermaypickhashdistribuPonforsmalltablesbecauseofstalestats• Causessomeprocessestodomoreworkthanothersdegradingscalability• MakesureopPmizerstatsareup-to-datetogettheopPmaldistribuPonmethod• OracleDatabase12cprovidesadapPvedistribuPonmethodstopicktheopPmaldistribuPonmethodatrunPme

HashdistribuPonwithlowcardinalityA9ergatheringop2mizerstatsinOracleDatabase11g

HashdistribuPonwithlowcardinality

Adap2vedistribu2onwithstalestatsinOracleDatabase12c

AdapPvedistribuPon

AdapPvedistribuPoninOracleDatabase12c

•  DistribuPonmethoddecisionbasedonactualnumberofrowsratherthanopPmizerstats

•  NewadapPvedistribuPonmethodhybridhash–  StaPsPccollectorsusedontheleZsideofthejointocounttheactualrowsatrunPme–  IfnumberofrowsislessthantheopPmizerthresholdusebroadcast,elseusehash

Hybridhashdistribu2on

AdapPvedistribuPoninOracleDatabase12c

Howtofindtherun2medistribu2onmethod

6=>Broadcast16=>Hash

CommondistribuPonproblems• OpPmizermaypickasubopPmaldistribuPonmethodbecauseofstale/missingstats– BroadcastdistribuPonwithhighcardinality•  CauseshighnumberofrowstobesentacrossRACnodesandprocesses• Maycausehightemporarytablespaceusage

BroadcastdistribuPonwithhighcardinality

EsPmatedrowsvs.actualrows Hightempusage HighacPvityfortempIO

BroadcastdistribuPonwithhighcardinality• OpPmizerpicksbroadcastdistribuPonbecauseofstalestats• Causesalotofrowstobebroadcastedtoallconsumerprocesses• MaycausehighPGAandtempusageinhashjoinsaseachconsumerprocessbuildsahashtableforallrows• MakesureopPmizerstatsareup-to-datetogettheopPmaldistribuPonmethod

BroadcastdistribuPonwithhighcardinality

A9ergatheringop2mizerstats

HashdistribuPonatrunPme Notempusage

EffectofDataSkewonWorkDistribuPon

WorkdistribuPonwithdataskew• HashdistribuPonisthemostcommondistribuPonmethodforlargetables– DoesnottakedataskewintoaccountunPlOracleDatabase12c

• Dataskewcancausesomeconsumerprocessestogetmorerows

PX1cust_id

c.cust_id

cust_id

TableSPX3getsmorerowsthanPX4

WorkdistribuPonwithdataskew

AdapPvedistribuPon(HashdistribuPonatrunPmeinthiscase)

A9ergatheringhistogramsonthejoincolumn

HashskewdistribuPon

Skewhandlingforjoins

• Value-baseddistribuPonofskeweddataforhashdistribuPon– BroadcastrowscontainingpopularvaluesontheleZsideofthejoin– Randomizerowscontainingpopularvaluesontherightsideofthejoin

• Popularvaluesareobtainedusingsampling• DecisionbasedonskewinformaPonfromhistograms• Considersskewontherightsideofthejoin(largertable)

IntroducedinOracleDatabase12cRelease1

Skewhandlingforjoins

• Histogramneededonthejoincolumn• Popularvalues– Valuesthatareatleast10xtheaveragefrequency– Valuesthathavemorethan30%frequency

•  Ifnumberofskewedvalues>DOP,noskewhandling

IntroducedinOracleDatabase12cRelease1

DegreeofParallelismandDowngrades

• Checkifyouarege}ngtheDOPyourequested– Ifnot,findoutwhy

• DOPisthefactorimpacPngperformanceforwelltunedSQLstatements– TunetheSQLbeforeplayingwiththeDOP

Degreeofparallelism(DOP)

DOPdowngrades• OthercolumnforPXcoordinatorshowsthedowngradereason

DOPdowngradereasonsselectqksxareasons,indx

fromx$qksxa_reason

whereqksxareasonslike'%DOPdowngrade%’;

QKSXAREASONSINDX

----------------------------------------------------------------------

DOPdowngradeduetoadaptiveDOP351

DOPdowngradeduetoresourcemanagermaxDOP352

DOPdowngradeduetoinsufficientnumberofprocesses353

DOPdowngradebecauseslavesfailedtojoin354

DOPdowngradereasons

• AdapPveparallelism– DowngradestheDOPbasedonsystemload

•  Enabledbyinit.oraparameterPARALLEL_ADAPTIVE_MULTI_USER• DisableadapPveparallelismbyse}ngtheparametertoFALSE.– DisabledanddeprecatedinOracleDatabase12cRelease2

• UseParallelStatementQueuingtopreventsystemsaturaPon

DOPdowngradeduetoadap2veDOP

DOPdowngradereasons• DOPdowngradeduetoresourcemanagermaxDOP– DatabaseResourceManager(DBRM)plancappedtheDOP•  PlandirecPvePARALLEL_DEGREE_LIMIT_P1•  IfyouneedhigherDOPsincreasethelimitifthereisampleunusedprocessingbandwidth•  OrdecreaseconcurrencyandallowhigherDOPs

• DOPdowngradeduetoinsufficientnumberofprocesses– #ofin-usePXserversreachedPARALLEL_MAX_SERVERS•  IncreasePARALLEL_MAX_SERVERSifthereisampleunusedprocessingbandwidth•  UseParallelStatementQueuingtolimitthenumberofin-useprocesses

DOPdowngradereasons• DOPdowngradebecauseslavesfailedtojoin– SomeprocessescouldnotjointheexecuPonofthestatement– UsuallyaresultofplanmismatchbetweenPXserversondifferentinstances– RaiseanSR

Summary

LeverageSQLMonitorforSQLtuning•  EasilyidenPfycausesofsubopPmalperformance– CheckwherethePmeisbeingspent– MakesureesPmatedcardinalitymatchesactualcardinality– GatherappropriateopPmizerstatstofixcardinalitymismatch

•  Forparallelstatements– CheckifworkanddatadistribuPonisevenforopPmalperformanceandscalability– FixunintendedDOPdowngradesdegradingperformance

ForfurtherinformaPon•  MonitoringParallelExecuPonusingReal-TimeSQLMonitoringinOracleDatabase12c–  hcps://blogs.oracle.com/datawarehousing/entry/monitoring_parallel_execuPon_using_real

•  CommonDistribuPonMethodsinParallelExecuPon–  hcps://blogs.oracle.com/datawarehousing/entry/common_distribuPon_methods_in_parallel

•  AdapPveDistribuPonMethodsinOracleDatabase12c–  hcps://blogs.oracle.com/datawarehousing/entry/adapPve_distribuPon_methods

•  FindingtheDistribuPonMethodinAdapPveParallelJoins–  hcps://blogs.oracle.com/datawarehousing/entry/finding_the_distribuPon_method_in

•  FindingtheReasonforDOPDowngrades–  hcps://blogs.oracle.com/datawarehousing/entry/finding_the_reason_for_dop

SafeHarborStatementTheprecedingisintendedtooutlineourgeneralproductdirecPon.ItisintendedforinformaPonpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfuncPonality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andPmingofanyfeaturesorfuncPonalitydescribedforOracle’sproductsremainsatthesolediscrePonofOracle.

The Best Way to Tune Your Parallel Statements: Real-Time ...€¦ · Real-Time SQL Monitoring •...

Documents

1 SQL in the Real World Chapter 10. 2 Interactive vs. Non-Interactive SQL Interactive SQL: SQL statements input from terminal; DBMS outputs to screen

REAL SQL QUERIES

Real-Time Business Intelligence with SQL Server 2005 Analysis Services

Real-World Performance Training - Oracle · Real-World Performance Training SQL Reference ... Sub-query ( correlated ) SQL select empno, ename, sal from emp e1 where sal > (select

Online Wiring Informaon

Informaon Brochure For Students

Meetup Real Time Aggregations Spark Streaming + Spark Sql

1 Real SQL Programming Embedded SQL Java Database Connectivity Stored Procedures

Real Data For Real Change - SQL Power Brochure

Real-Time SQL Monitoring

Proposed Rezoning and Plan Amendment • Informaon Session

Real SQL Programming

Real-Time Integration Between MongoDB and SQL Databases

CAREERS IN INFORMATION TECHNOLOGY · in the ﬁeld of informaon technology, you might want to turn your hobby into a career. Today, informaon is paramount and informaon technology

1 Real SQL Programming Persistent Stored Modules (PSM) PL/SQL Embedded SQL

Expanding the SQL Horizons: PL/SQL User-Defined Functions ...nyoug.org/Presentations/2014/Rosenblum_SQL_Horizons.pdf · PL/SQL User-Defined Functions in the Real World Michael Rosenblum

Interesting SQL Solutions to Real Life Problems

SQL Server StreamIinsight - data processing in real time

Real-Life SQL Server 2012: AlwaysOn Lessons Learned · Real-Life SQL Server 2012: AlwaysOn Lessons Learned Brent Ozar: MCM, MVP, ... AlwaysOn AGs 5-minute intro ... Failover unit

Protec’ngInformaonAssets* · 2015-06-22 · MIS$5206$Protecng$Informa/on$Assets$ GregSenko Protec’ngInformaonAssets* Week4 * Risk Evaluation *