13
Watching SQL Execute on Oracle - Part I - James Koopmann Do you have SQL running within your database? Of course you do. This article is the first in a series to introduce you to a method of finding information about the SQL your users are executing in your databases. Your ability as a D! to detect who is accessing the database and the SQL they are executing is "aramount in your ability to "ro"erly manage and give feedbac# on the ty"e of wor# your database is re$uested to do. This article will begin our series with an introduction to determining who is logged into your database and what SQL they are executing. This article is not concerning itself with the tuning of SQL but rather a "rimer so that you can get familiar with or re%ac$uainted with the underlying tables within Oracle that give information on who and what is being done around the SQL. V$SESSION The &'S(SS)O* is often the +um"ing off "lace to determine who is logged into the database and a high level overview of what they are doing. )n Table 1 ) have given a subset of the columns this view contains. ,hile there are other columns in this view that relate to o"erating system information and the a""lication being executed- ) am only "resenting those columns that give us the basis for our +um"ing off "oint to determine that this is in fact an active us and also the columns we will need later to +oin to the actual SQL being executed. Table 1. Limited V$SESSION information Column Description SADDR Identifies a unique Oracle session address SID Identifies a unique Oracle session USERNAME The Oracle user (same as from dba_users) STATUS Tells us the status of the session. We will be concerned with ACT sessions those that are e!ecutin" #$% PROCESS This is the o&eratin" s'stem &rocess id for the connection. Onl' a reference so that 'ou can "o loo on the O*# side. TYPE The t'&e of session connected to the database SQL_ADDRESS +sed with SQL_HASH_VALUE to identif' the #$% statement that is curren bein" e!ecuted. SQL_HASH_VALUE +sed with SQL_ADDRESS to identif' the #$% statement that is currentl'

Watching SQL Execute on Oracle

Embed Size (px)

DESCRIPTION

Watching SQL Execute on Oracle

Citation preview

Watching SQL Execute on Oracle - Part I - James Koopmann

Watching SQL Execute on Oracle - Part I - James Koopmann

Do you have SQL running within your database? Of course you do. This article is the first in a series to introduce you to a method of finding information about the SQL your users are executing in your databases.

Your ability as a DBA to detect who is accessing the database and the SQL they are executing is paramount in your ability to properly manage and give feedback on the type of work your database is requested to do. This article will begin our series with an introduction to determining who is logged into your database and what SQL they are executing. This article is not concerning itself with the tuning of SQL but rather a primer so that you can get familiar with or re-acquainted with the underlying tables within Oracle that give information on who and what is being done around the SQL.

V$SESSION

The V$SESSION is often the jumping off place to determine who is logged into the database and a high level overview of what they are doing. In Table 1 I have given a subset of the columns this view contains. While there are other columns in this view that relate to operating system information and the application being executed, I am only presenting those columns that give us the basis for our jumping off point to determine that this is in fact an active user and also the columns we will need later to join to the actual SQL being executed.

Table 1. Limited V$SESSION informationColumnDescription

SADDRIdentifies a unique Oracle session address

SIDIdentifies a unique Oracle session

USERNAMEThe Oracle user (same as from dba_users)

STATUSTells us the status of the session. We will be concerned with ACTIVE sessions, those that are executing SQL

PROCESSThis is the operating system process id for the connection. Only given here as a reference so that you can go look on the O/S side.

TYPEThe type of session connected to the database

SQL_ADDRESSUsed with SQL_HASH_VALUE to identify the SQL statement that is currently being executed.

SQL_HASH_VALUEUsed with SQL_ADDRESS to identify the SQL statement that is currently being executed. This SQL_HASH_VALUE is unique, or should be unique, to the same SQL statement no matter when it is executed. Thus 'select * from dual' will always produce the same SQL_HASH_VALUE.

In order to extract the information, you can issue a SQL statement such as the following in Listing 1. This is a very simplistic statement but gives us all the information we need to determine if the user is actually executing SQL at the time. If a user has SQL that is executing, the status column will be ACTIVE and the SQL_ADDR & SQL_HASH_VALUE will be populated. I have given the output of two different executions of this particular SQL against the V$SESSION view. The first shows where a user was noticed logged into our database but is basically inactive and there is no SQL address or hash value. The second execution of this SQL shows a user who is actively executing SQL, denoted by the values in the address and hash value columns.

Listing 1. Extracting simplistic columns to show sessions that are executing SQL

select sid,

to_char(logon_time,'MMDDYYYY:HH24:MI') logon_time,

username,

type,

status,

process,

sql_address,

sql_hash_value

from v$session

where username is not null

Inactive session with no SQL executingSID LOGON_TIME USERNAME TYPE STATUS PROCESS SQL_ADDR SQL_HASH_VALUE

---- -------------- --------- ---- -------- ------------ -------- --------------

150 06252004:06:23 JKOOPMANN USER INACTIVE 3528:3036 00 0

Active session with SQL executingSID LOGON_TIME USERNAME TYPE STATUS PROCESS SQL_ADDR SQL_HASH_VALUE

---- -------------- --------- ---- -------- ------------ -------- --------------

150 06252004:06:23 JKOOPMANN USER ACTIVE 3528:3036 6879D780 2803425422

V$SQLAREA

When Oracle executes a query, it places it in memory. This allows Oracle to reuse the same SQL if needed by the executing session at a latter date or by another user that may need the same SQL statement. Remember that one of the steps in Oracle is the assigning of a unique SQL_HASH_VALUE and SQL_ADDRESS to each SQL statement. By Oracle doing this, it provides us a method to determine who is executing what SQL based on the join columns from the V$SESSION of SQL_ADDRESS & SQL_HASH_VALUE to the V$SQLAREA view and columns ADDRESS and HASH_VALUE. Table 2 gives another short list of columns that are of concern when we start looking at the SQL being executed by the users. There are other "statistical" columns that are provided again in the view but I have purposely only provided the two statistical columns of CPU_TIME and ELAPSED_TIME because I am getting more and more convinced that it really does not matter if we do one zillion reads in a SQL statement, what matters is that we can produce a result set under our SLAs.

Table 2. Limited V$SQLAREA columns

ColumnDescription

SQL_TEXTThis is the first 1000 characters of the SQL being executed by the user. If more than 1000 characters, you should use V$SQL_TEXT which is described latter in the article.

OPTIMIZER_MODEThe optimizer mode being utilized by the query

ADDRESSThis is the address to the parent of this cursor/sql

HASH_VALUEThis is the hash value to the parent statement in the library cache

CPU_TIMEThe accumulated microseconds of CPU time used by the SQL

ELAPSED_TIMEThe accumulated microseconds elapsed time used by the SQL

In the V$SQLAREA view there is not any direct indication of who is executing the particular SQL at any given time. To get this information we must join to the V$SESSION view on HASH_VALUE and ADDRESS. Listing 2 gives the SQL to do this join, shows the active SQL executing in your database and a sample output of a "dumb" query.

Listing 2 Extracting the active SQL a user is executing

select sesion.sid,

sesion.username,

optimizer_mode,

hash_value,

address,

cpu_time,

elapsed_time,

sql_text

from v$sqlarea sqlarea, v$session sesion

where sesion.sql_hash_value = sqlarea.hash_value

and sesion.sql_address = sqlarea.address

and sesion.username is not null

Active session and the SQL it is executingSID USERNAME OPTIMIZER_MODE HASH_VALUE ADDRESS CPU_TIME ELAPSED_TIME

---- --------- --------------- ---------- -------- ---------- ------------

150 JKOOPMANN ALL_ROWS 2803425422 6879D780 11923758 12106196

SQL_TEXT

--------------------------------------------------

select a.table_name from dba_tables a,dba_tables b

V$SESS_IO

Often you may have an active session and actually show a valid SQL statement through the V$SESSION and V$SQLAREA views that seems to be taking very long. Users may be complaining that their query is "stuck" or not responsive. You as a DBA can validate that the SQL they are executing is actually doing something in the database and not "stuck" be simply querying the V$SESS_IO view to determine if the query is in fact "stuck" or is actually doing work within the database. Granted, this does not mean there isn't a tuning opportunity but you can at least show the SQL is working. Table 3 shows the V$SESS_IO view and the columns associated with it. As you can see, there is an Oracle session identifier (SID) that you can link back to the V$SESSION view for the active session. If the GETS, READS, or CHANGES columns continue to increase for a session, you can be assured that the SQL statement is not "stuck."

Table 3. I/O values for a V$SESSION connection This view lists I/O statistics for each user session.

ColumnDescription

SIDIdentifies a unique Oracle session

BLOCK_GETSNumber of block gets done

CONSISTENT_GETSNumber of consistent gets done

PHYSICAL_READSNumber of physical reads done

BLOCK_CHANGESNumber of blocks that where changed

CONSISTENT_CHANGESNumber of consistent block changes done

A very simple join, depicted in Listing 3, to the V$SESSION view will give you the results to determine the i/o being done by active sessions. As an example, I have given two executions of this SQL to show the increase in i/o for my active session.

Listing 3 I/O being done by an active SQL statement

select sess_io.sid,

sess_io.block_gets,

sess_io.consistent_gets,

sess_io.physical_reads,

sess_io.block_changes,

sess_io.consistent_changes

from v$sess_io sess_io, v$session sesion

where sesion.sid = sess_io.sid

and sesion.username is not null

First poll for i/oSID BLOCK_GETS CONSISTENT_GETS PHYSICAL_READS BLOCK_CHANGES CONSISTENT_CHANGES

---- ---------- --------------- -------------- ------------- ------------------

150 4 470149 446 4 0

SQL> /

Second poll for i/o SID BLOCK_GETS CONSISTENT_GETS PHYSICAL_READS BLOCK_CHANGES CONSISTENT_CHANGES

---- ---------- --------------- -------------- ------------- ------------------

150 4 1002523 448 4 0

V$SQLTEXT

If by chance the query shown earlier in the V$SQLAREA view did not show your full SQL text because it was larger than 1000 characters, this V$SQLTEXT view should be queried to extract the full SQL. It is a piece by piece of 64 characters by line, that needs to be ordered by the column PIECE. Table 5 shows the columns that are of concern to us and Listing 5 gives the SQL to extract the SQL based on active sessions in the V$SESSION view.

Table 5 V$SQLTEXT columns of concern for show the SQL text of an executing query

ColumnDescription

ADDRESSUsed with SQL_HASH_VALUE to identify the SQL statement that is currently being executed.

HASH_VALUEUsed with SQL_ADDRESS to identify the SQL statement that is currently being executed. This SQL_HASH_VALUE is unique, or should be unique, to the same SQL statement no matter when it is executed. Thus 'select * from dual' will always produce the same SQL_HASH_VALUE.

PIECEA sequential number used to piece individual parts of the SQL statement together

SQL_TEXTThe individual piece of SQL text

Listing 5 SQL to show the full SQL executing for active sessions

select sesion.sid,

sql_text

from v$sqltext sqltext, v$session sesion

where sesion.sql_hash_value = sqltext.hash_value

and sesion.sql_address = sqltext.address

and sesion.username is not null

order by sqltext.piece

This article should have gotten you familiar with the basics around determining who and what is being executed within your database. It is only the surface to the vast amount of information available within these views and if you have not ventured into them lately you might want to list the contents out and see what you have been missing.

Do you have SQL running within your database? Of course you do. This article is the second in a series to introduce you to how you can extract more information about the SQL executing in your databases.

In the first of this series we looked at how to determine if sessions were running SQL from the V$SESSION & V$SESS_IO views and then how to extract the particular SQL from the V$SQLAREA & V$SQL_TEXT views. This article will take us a bit further and look deeper into SQL statements that have bind variables. We will then turn our attention to how to extract the execution plan of running (NOT static) SQL statements and how to get some better statistics about those SQL statements after they have run. Again, as with the last article, this article is not concerning itself with the tuning of SQL but rather a primer on where to get the information about your SQL statements. I have only scratched the surface of the views that are in this article and I encourage you to at least describe the views through SQL*Plus to get a glimpse of the information they contain.

V$SQLAREA

This view was covered in the first part of this series and I would encourage you to go back and take a look at it. In the last article, the SQL_TEXT was just straight SQL. In this article we are going to concern ourselves with SQL that has a bind variable in it. If you look at the SQL presented in Listing 1, it is the same in part I of this series but the output shows the SQL with a bind variable. I have also added the columns V$SESSION.SERIAL#, V$SESSION.SQL_ID, and V$SESSION.SQL_CHILD_NUMBER to the query. These additional columns and their descriptions have been given in Table 1 and their use will be shown latter in this article.

Listing 1 Extracting the active SQL a user is executing

select sesion.sid,

sesion.serial#,

sesion.username,

sesion.sql_id,

sesion.sql_child_number,

optimizer_mode,

hash_value,

address,

sql_text

from v$sqlarea sqlarea, v$session sesion

where sesion.sql_hash_value = sqlarea.hash_value

and sesion.sql_address = sqlarea.address

and sesion.username is not null

Active session and the SQL it is executingSID SERIAL# USERNAME SQL_ID SQL_CHILD_NUMBER OPTIMIZER_MODE HASH_VALUE ADDRESS

--- ------- --------- ------------- ---------------- -------------- ---------- --------

149 8 JKOOPMANN 5qk509xugpmpv 1 FIRST_ROWS 1962593979 69A7558C

SQL_TEXT

--------------------------------------------------

SELECT count(*) FROM sys.dba_tables WHERE owner = :1

Table 1. Additional V$SESSION column information

ColumnDescription

SERIAL#Along with SID uniquely identifies a session and the objects it uses.

SQL_IDAn identifier to the SQL currently executing.

SQL_CHILD_NUMBERA child number of a SQL statement that is currently being executed

V$SQL_BIND_CAPTURE

When looking at the SQL statement through V$SQLAREA we only see the bind variable. If we wanted to see the contents of the variable in past versions of Oracle we would have had to issue a trace. This is no longer the case and we can use the V$SQL_BIND_CAPTURE view to look at the value hidden from normal eyes. Be aware that the ability to see the bind variable contents is contigent upon you setting the STATISTICS_LEVEL to ALL. Table 2 shows a subset of the columns available to us in the V$SQL_BIND_CAPTURE view and their descriptions. Listing 2 shows the SQL you can use to join the V$SESSION and V$SQL_BIND_CAPTURE views together to look at the contents of bind variables for currently executing SQL.

Table 2 V$SQL_BIND_CAPTURE column information

ColumnDescription

ADDRESSThis is the address to the parent of this cursor/sql

HASH_VALUEThis is the hash value to the parent statement in the library cache

NAMEThe name of the bind variable.

VALUE_STRINGValue of the bind variable

Listing 2 Extracting the bind variable contents for SQL that is executing

select sesion.sid,

sesion.username,

sesion.sql_id,

sesion.sql_child_number,

sql_bind_capture.name,

sql_bind_capture.value_string

from v$sql_bind_capture sql_bind_capture, v$session sesion

where sesion.sql_hash_value = sql_bind_capture.hash_value

and sesion.sql_address = sql_bind_capture.address

and sesion.username is not null

Active session and the SQL it is executing SID USERNAME SQL_ID SQL_CHILD_NUMBER NAME VALUE_STRING

---- ------------ ------------- ---------------- ---- ------------

149 JKOOPMANN 5qk509xugpmpv 1 :1 ORDEROWNER

V$SQL_OPTIMIZER_ENV

During the course and life of a database, applications and sessions typically will begin to alter their environments to optain the best performance. It can sometimes be very hard to track down anyone with this information, let alone extract the information from either in-house or third party code. For this reason you can use the V$SQL_OPTIMIZER_ENV view to pull from memory the actual optimizer environment the individual SQL is executing. Table 3 shows the subset of columns that are used from this view in order to see the values for the optimizer's environment. Listing 3 shows the SQL to do this join for active SQL and a subset of the output it generates.

Table 3 V$SQL_OPTIMIZER_ENV column information

ColumnDescription

ADDRESSThis is the address to the parent of this cursor/sql

HASH_VALUEThis is the hash value to the parent statement in the library cache

NAMEParameter name

ISDEFAULTShow if the parameter is set to its default value.

VALUEParameter value

Listing 3 Extracting the optimizer environment settings for SQL that is executing

select sesion.sid,

sesion.username,

name,

isdefault,

value

from v$sql_optimizer_env sql_optimizer_env, v$session sesion

where sesion.sql_hash_value = sql_optimizer_env.hash_value

and sesion.sql_address = sql_optimizer_env.address

and sesion.username is not null

Optimizer environment settings for current active SQLSID USERNAME NAME ISD VALUE

---- ------------ ---------------------------------------- --- ----------

149 JKOOPMANN parallel_execution_enabled YES true

149 JKOOPMANN optimizer_features_enable YES 10.1.0

149 JKOOPMANN cpu_count YES 2

149 JKOOPMANN active_instance_count YES 1

149 JKOOPMANN parallel_threads_per_cpu YES 2

149 JKOOPMANN hash_area_size YES 131072

149 JKOOPMANN bitmap_merge_area_size YES 1048576

149 JKOOPMANN sort_area_size YES 65536

149 JKOOPMANN sort_area_retained_size YES 0

Extract the Execution Plan

If you are tired of extracting the raw SQL_TEXT and formatting it and then running it through the old method of producing explain plans, this section is for you. There is an Oracle object called DBMS_XPLAN that allows you to supply the SQL_ID and SQL_CHILD_NUMBER and have it produce a nicely formatted explain output. Listing 4 gives an example SQL query that uses the SQL_ID and SQL_CHILD_NUMBER we obtained earlier from V$SESSION and gives the output of the query.

Listing 4 Extracting the optimizer environment settings for SQL that is executing

SELECT * FROM table(DBMS_XPLAN.DISPLAY_CURSOR(('5qk509xugpmpv'),1));

Explain outputPLAN_TABLE_OUTPUT

----------------------------------------------------------------------------------------------------

SQL_ID 5qk509xugpmpv, child number 1

-------------------------------------

SELECT count(*) FROM sys.dba_tables WHERE owner = :1

Plan hash value: 580517646

----------------------------------------------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)|Time|

----------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | | | 242 (100)| |

| 1 | SORT AGGREGATE | | 1 | 149 | | |

| 2 | NESTED LOOPS | | 1 | 149 | 242 (1)| 03 |

| 3 | MERGE JOIN CARTESIAN | | 1 | 136 | 218 (1)| 03 |

| 4 | NESTED LOOPS OUTER | | 1 | 81 | 194 (1)| 03 |

| 5 | NESTED LOOPS OUTER | | 1 | 78 | 193 (1)| 03 |

| 6 | NESTED LOOPS OUTER | | 1 | 70 | 192 (1)| 03 |

| 7 | NESTED LOOPS OUTER | | 1 | 59 | 191 (1)| 03 |

| 8 | NESTED LOOPS | | 1 | 54 | 191 (1)| 03 |

|* 9 | HASH JOIN | | 8 | 368 | 183 (1)| 03 |

| 10 | NESTED LOOPS | | 7 | 119 | 4 (0)| 01 |

| 11 | TABLE ACCESS BY INDEX ROWID| USER$ | 1 | 14 | 1 (0)| 01 |

|* 12 | INDEX UNIQUE SCAN | I_USER1 | 1 | | 0 (0)| |

| 13 | TABLE ACCESS FULL | TS$ | 7 | 21 | 3 (0)| 01 |

|* 14 | TABLE ACCESS FULL | TAB$ | 16 | 464 | 178 (0)| 03 |

|* 15 | TABLE ACCESS BY INDEX ROWID | OBJ$ | 1 | 8 | 1 (0)| 01 |

|* 16 | INDEX UNIQUE SCAN | I_OBJ1 | 1 | | 0 (0)| |

|* 17 | INDEX UNIQUE SCAN | I_OBJ1 | 1 | 5 | 0 (0)| |

| 18 | TABLE ACCESS CLUSTER | SEG$ | 1 | 11 | 1 (0)| 01 |

|* 19 | INDEX UNIQUE SCAN | I_FILE#_BLOCK# | 1 | | 0 (0)| |

| 20 | TABLE ACCESS BY INDEX ROWID | OBJ$ | 1 | 8 | 1 (0)| 01 |

|* 21 | INDEX UNIQUE SCAN | I_OBJ1 | 1 | | 0 (0)| |

| 22 | TABLE ACCESS CLUSTER | USER$ | 1 | 3 | 1 (0)| 01 |

|* 23 | INDEX UNIQUE SCAN | I_USER# | 1 | | 0 (0)| |

| 24 | BUFFER SORT | | 1 | 55 | 217 (1)| 03 |

|* 25 | FIXED TABLE FULL | X$KSPPI | 1 | 55 | 24 (0)| 01 |

|* 26 | FIXED TABLE FIXED INDEX | X$KSPPCV (ind:2) | 1 | 13 | 24 (0)| 01 |

----------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):

---------------------------------------------------

9 - access("T"."TS#"="TS"."TS#")

12 - access("U"."NAME"=:1)

14 - filter(BITAND("T"."PROPERTY",1)=0)

15 - filter("O"."OWNER#"="U"."USER#")

16 - access("O"."OBJ#"="T"."OBJ#")

17 - access("T"."BOBJ#"="CO"."OBJ#")

19 - access("T"."TS#"="S"."TS#" AND "T"."FILE#"="S"."FILE#" AND "T"."BLOCK#"="S"."BLOCK#")

21 - access("T"."DATAOBJ#"="CX"."OBJ#")

23 - access("CX"."OWNER#"="CU"."USER#")

25 - filter("KSPPI"."KSPPINM"='_dml_monitoring_enabled')

26 - filter("KSPPI"."INDX"="KSPPCV"."INDX")

V$SQL_PLAN_STATISTICSTypically when looking at execution statistics for a SQL statement, DBAs will go to the V$SQLAREA view. The only problem with going to the V$SQLAREA view is that it is an accumulation of statistics for a particular SQL statement. If you want single run statistics for an SQL statement then you can go to the V$SQL_PLAN_STATISTICS view as there are columns in this view that report on the last time the statement was executed. The exciting thing about this view is that it tells you the actual statistics for each step in the execution plan. Table 5 shows a subset of the columns for this view. More than these columns are of importance and you should venture to take a look at them. Listing 5 gives an example of how to extract the statistics for the active executing SQL by joining to the V$SESSION view.

Table 5 Subset of columns for V$SQL_PLAN_STATISTICS

ColumnDescription

ADDRESSThis is the address to the parent of this cursor/sql

HASH_VALUEThis is the hash value to the parent statement in the library cache

PLAN_HASH_VALUEPlan hash value

OPERATION_IDThe number for each step in the execution plan

OUTPUT_ROWSThe number of rows returned

LAST_CR_BUFFER_GETSNumber of consistent gets for the last run of the plan for a given step.

LAST_DISK_READSNumber of physical disk reads for the last run of the plan for a given step

Listing 5 Extracting the statistics for a single execution of a SQL statement

select sesion.sid,

sesion.username,

sql_plan_statistics.operation_id "Id",

sql_plan_statistics.last_output_rows "Rows",

sql_plan_statistics.last_cr_buffer_gets "Consistent Gets",

sql_plan_statistics.last_disk_reads "Disk Reads"

from v$sql_plan_statistics sql_plan_statistics, v$session sesion

where sesion.sql_hash_value = sql_plan_statistics.hash_value

and sesion.sql_address = sql_plan_statistics.address

and sesion.username is not null

Explain output

SID USERNAME Id Rows Consistent Gets Disk Reads

--- -------------- ---------- ---------- --------------- ----------

149 JKOOPMANN 1 1 8594 0

149 JKOOPMANN 2 643 8594 0

149 JKOOPMANN 3 643 8594 0

149 JKOOPMANN 4 643 8594 0

149 JKOOPMANN 5 643 7540 0

149 JKOOPMANN 6 643 6369 0

149 JKOOPMANN 7 643 4614 0

149 JKOOPMANN 8 643 3969 0

149 JKOOPMANN 9 1572 823 0

149 JKOOPMANN 10 7 11 0

149 JKOOPMANN 11 1 2 0

149 JKOOPMANN 12 1 1 0

149 JKOOPMANN 13 7 9 0

149 JKOOPMANN 14 1572 812 0

149 JKOOPMANN 15 643 3146 0

149 JKOOPMANN 16 1572 1574 0

149 JKOOPMANN 17 50 645 0

149 JKOOPMANN 18 555 1755 0

149 JKOOPMANN 19 555 645 0

149 JKOOPMANN 20 526 1171 0

149 JKOOPMANN 21 526 645 0

149 JKOOPMANN 22 526 1054 0

149 JKOOPMANN 23 526 2 0

149 JKOOPMANN 24 643 0 0

149 JKOOPMANN 25 1 0 0

149 JKOOPMANN 26 643 0 0

Determining what SQL is executing within the Oracle engine and watching its access path and the true statistical values accumulated for each run are invaluable. Give these scripts a try, add to them, and you too can begin to finally determine what SQL is executing within your environment.