33
Copyright © 2013 Apex Evangelists Troubleshooting APEX Applications Nienke Gijsen en Roel Hartman

Troubleshooting APEX Performance Issues

Embed Size (px)

DESCRIPTION

Real life story about an APEX Application that suffered severe performance issues.

Citation preview

Page 1: Troubleshooting APEX Performance Issues

Copyright © 2013 Apex Evangelists

Troubleshooting APEX ApplicationsNienke Gijsen en Roel Hartman

Page 2: Troubleshooting APEX Performance Issues

About Roel

• Director of APEX Evangelists NL

• 20+ years of Oracle Experience

• “RoelH” in the APEX Forum

• Presenter at KSCOPE, OOW, Collaborate, DOAG, UKOUG, ...

• Oracle ACE Director

• Blog @ http://roelhartman.blogspot.com

• Twitter : @RoelH

Page 3: Troubleshooting APEX Performance Issues

Enterprise Manager

Page 4: Troubleshooting APEX Performance Issues

Alert.log

• Twee tot drie keer per week

• Apex process

• ORA-04030: out of process memory when trying to allocate 82456 bytes (pga heap,control file i/o buffer)

• ORA-04030: out of process memory when trying to allocate 16328 bytes (koh-kghu sessi,pmucalm coll)

Page 5: Troubleshooting APEX Performance Issues

Trace FilePRIVATE HEAP SUMMARY DUMP

15 GB total:

15 GB commented, 183 KB permanent

81 KB free (0 KB in empty extents),

15 GB, 1 heap: "session heap "

declare

rc__ number;

simple_list__ owa_util.vc_arr;

complex_list__ owa_util.vc_arr;

begin

owa.init_cgi_env(:n__,:nm__,:v__);

htp.HTBUF_LEN := -1;

null;

null;

simple_list__(1) := 'sys.%';

simple_list__(2) := 'dbms\_%';

simple_list__(3) := 'utl\_%';

simple_list__(4) := 'owa\_%';

simple_list__(5) := 'owa.%';

simple_list__(6) := 'htp.%';

simple_list__(7) := 'htf.%';

simple_list__(8) := 'wpg_docload.%';

if ((wwv_flow_epg_include_modules.authorize('wwv_flow_file_mgr.get_file') = false) or (owa_match.match_pattern(p_string =>

'wwv_flow_file_mgr.get_file'

Page 6: Troubleshooting APEX Performance Issues

MetalinkPLSQL Procedure Causing ORA-04030: (pga heap,control file i/o buffer) And ORA-04030: (koh-kghu sessi,pmuccst: adt/record) or ORA-04030: (koh-kghucall ,pmucalm coll) Errors [ID 1325100.1]

Solution

• Change the page count at the OS level:

• $ more /proc/sys/vm/max_map_count$ sysctl -w vm.max_map_count=262144 (256K)

Page 7: Troubleshooting APEX Performance Issues

Apex Developers

• Error log

• Tijdstippen onderzoek

• Wat doet de get_file?

• Wat kan er 16GB groot zijn?

Page 8: Troubleshooting APEX Performance Issues

En  nu?

DBA

Page 9: Troubleshooting APEX Performance Issues

Troubleshooting APEX Applications

Debugging Tracing Logging

Page 10: Troubleshooting APEX Performance Issues

Debugging APEX Applications

Page 11: Troubleshooting APEX Performance Issues

Debugging APEX Applications

• Develop Mode

• Runtime Mode

• Websheets needs a Development environment !

• @apxdvins.sql / @apxdevrm.sql

Page 12: Troubleshooting APEX Performance Issues

Debugging Enable - Develop Mode

• Application Level Setting

Page 13: Troubleshooting APEX Performance Issues

Debugging Enable - Runtime Modebegin

wwv_flow_api.set_security_group_id(p_security_group_id=>2616513429239104);! !

wwv_flow_api.set_enable_app_debugging(103,1);

end;Workspace ID

Application ID 1 = ON0 = OFF

Page 14: Troubleshooting APEX Performance Issues

Debugging - Develop Mode

Page 15: Troubleshooting APEX Performance Issues

Debugging - Runtime Mode

LEVEL1 .. LEVEL9

Page 16: Troubleshooting APEX Performance Issues

Debugging - Instrumentation

apex_debug

Page 17: Troubleshooting APEX Performance Issues

Remote Debugging

• Using SQL Developer

Page 18: Troubleshooting APEX Performance Issues

Tracing APEX Applications

Page 19: Troubleshooting APEX Performance Issues

Tracing Enable - Develop Mode

• Workspace Level Setting (default “Yes”)

Page 20: Troubleshooting APEX Performance Issues

Tracing Enable - Runtime Modebegin apex_instance_admin.set_parameter( p_parameter => 'TRACING_ENABLED' , p_value => 'Y' );

end;

SYS, SYSTEM, APEX_040200 or APEX_ADMINISTRATOR_ROLE

Page 21: Troubleshooting APEX Performance Issues

Tracing - Develop & Runtime Mode

DEBUGGING must be enabled as well!

Page 22: Troubleshooting APEX Performance Issues

Logging APEX Applications

Page 23: Troubleshooting APEX Performance Issues

Logging Enable - Develop Mode

• Workspace Level Setting (default “U”)

• Application Level Setting (default “Yes”)

Page 24: Troubleshooting APEX Performance Issues

Logging Enable - Runtime Mode

• Workspace Level Setting (default “U”)begin

apex_instance_admin.set_parameter( p_parameter => 'APPLICATION_ACTIVITY_LOGGING' , p_value => 'U' ); -- or [A]lways, [N]everend;

• Application Level Setting (default “Yes”)begin

wwv_flow_api.set_security_group_id(p_security_group_id=>2616513429239104);!wwv_flow_api.set_logging(103,‘YES’); -- Or ‘NO’ to switch it off

end;

Page 25: Troubleshooting APEX Performance Issues

Logging - Where Is It Going?SELECT apex_view_name, comments FROM apex_dictionary where column_id = 0 AND apex_view_name LIKE '%LOG%'

Page 26: Troubleshooting APEX Performance Issues

Logging - How Long Is It In There?

• Workspace Level Setting

• Develop Mode

• Runtime Mode

begin apex_instance_admin.set_log_switch_interval ( p_log_name in 'ACTIVITY' -- or ‘ACCESS’, ‘CLICKTHRU’, ‘DEBUG’

, p_log_switch_after_days => 180 ); -- between 1 and 180, default 14 end;

• Rotating Logs

Page 27: Troubleshooting APEX Performance Issues

Logging Examining - Develop Mode

Page 28: Troubleshooting APEX Performance Issues

Logging Examining - Runtime Modeselect application_id, page_id, to_char(view_date, 'DD-MM-YYYY HH24:MI') datetime, log_context, elapsed_time, rows_queriedfrom apex_workspace_activity_logorder by elapsed_time desc

Page 29: Troubleshooting APEX Performance Issues

The Results

• Fluctuating Performance (avg 0.1 - max 1000 sec)

• Calls to WWV_FLOW_FILE_MGR.GET_FILE

• Used to retrieve images from the database

Page 30: Troubleshooting APEX Performance Issues

The Cause•SELECT ‘<img src=”#WORKSPACE_IMAGES#no_picture.jpg”>’ ...FROM <awesome join>WHERE <more or less restrictions>

• Calls WWV_FLOW_FILE_MGR.GET_FILE tons of timeto retrieve ... nothing ....

Page 31: Troubleshooting APEX Performance Issues

The Solution

• Moved all images (and CSS / JavaScript) to web server

• Replaced ‘<img src=”#WORKSPACE_IMAGES#no_picture.jpg”>’ with ‘<img src=”/my_img/no_picture.jpg”>’

• Now using web server caching

• Database isn’t suffering from millions of calls anymore

Page 32: Troubleshooting APEX Performance Issues

Copyright © 2013 Apex Evangelistshttp://apex-evangelists.com

Q & A