Technical Panel Developing High-Performance Applications on Oracle Solaris

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.1


Technical Panel:Developing High-Performance Applications on Oracle Solaris

Don KretschSr. Director, Oracle

CON7196


Program Agenda

Panel Presentations– Presenters will share some of the development practices and tools

their teams use to deliver reliable, scalable, high performance enterprise-ready products on Solaris

Audience Q&A


Panel Moderator

Don KretschSr. DirectorSolaris Developer Tools


Panel Speakers

Todd LittleChief Architect, Oracle Tuxedo

Binoy SukumaranSr. Director, Oracle Database

Stefan SchneiderTechnologist, Oracle Solaris


DON KRETSCH

SENIOR DIRECTOR

ORACLE SOLARIS STUDIO

ORACLEPRODUCT

LOGO


Evolution of Deployment Environments

Multiple applications running on virtualized

servers

Single applications running on dedicated

servers

Multiple applications running on engineered

systems

OrderEntry

Database

WebTier Finance

Mail

Order Entry

WebTier

Database

HR

HR

Mail

Finance


Developer Challenges

Development systems that

differ from deployment

environments Developers need

platform-aware tools to

meet these challenges

Single code base that must

perform well across multiple

deployment environments

Ability to develop and

analyze applications that are growing in

size and complexity

Application correctness and performance in an engineered

systems environment

DevelopmentEnvironment

Application Size ApplicationEfficiency

ApplicationPorting


Oracle Solaris Studio

© 2011 Oracle Corporation – Proprietary and Confidential 4

Performance Analyzer provides unparalleled insight into your app, allowing you to identify bottlenecks and improve performance by orders of magnitude

Code Analyzer ensures app reliability by detecting app vulnerabilities, including memory leaks and memory access violations

Thread Analyzer simplifies complex parallel programming errors by detecting hard to pinpoint race and deadlock conditions

C, C++ Compilers use advanced code generation technology to optimize apps for highest performance on SPARC & x86

Fortran Compiler optimizes compute intensive app performance

Debugger ensures app stability with event handling & multi-thread support

Performance Library maximizes compute-intensive app performance using advanced numeric solver libraries

Integrated Development Environment increases developer efficiency

New

Compiler Suite Analysis Suite


Oracle Solaris Studio used to develop many Oracle products

– SPARC SuperCluster– Exadata on Solaris– Oracle Solaris– Oracle Database– Java VM– Tuxedo Transaction Manager– Peoplesoft, JD Edwards, Hyperion, …

Optimized for Oracle hardware systems Development platform of choice for top

Oracle Customers and ISV Partners

Oracle Product Teams Develop with Studio

Studio Optimizes the Stack


Solaris 11 on SPARC T4

Solaris 11 on Intel x86

Compiler Choice Impacts Performance

Estimated results from SPECcpu 2006 benchmarks


Profile fully optimized and parallelized production code

Support for 100’s of processes and 1000’s of threads

Shows execution data in easy to understand timeline charts and relates to source code

Compare multiple experiments

Minimal run-time impact (< 5%)

Identify Application Performance Bottlenecks

Studio Performance Analyzer


Remotely build, debug, run, analyze Solaris applications from Unix, Windows or Mac OS desktops

User gestures are identical in both local and remote environments

– Remote commands are invoked over secure connection (SSH)

– Remote File System browser accesses server files through SFTP or any file sharing mechanism available in OS

Built-in terminal sharing connection to remote host

Access Remote Servers for Development

Studio Remote Access IDE

C++

SFTP

SSH


TODD LITTLE

CHIEF ARCHITECT

ORACLE FUSION MIDDLEWARE

ORACLEPRODUCT

LOGO


Tuxedo – Application Server

SOA based – everything is a service 30+ years of development Supporting C, C++, COBOL, Python, Ruby, PHP, Java


Solaris Studio Tuxedo SupportNew plug-in

Integrated development environment for Tuxedo applications

– Creating

– Building

– Running

– Debugging

– Static and dynamic analysis

Code completion and highlighting


Tuxedo and Solaris – Leading TPC-C

Goal – New TPC-C world record iPlanet front-end as Tuxedo client Needed 768 threads, bottleneck at 150 threads Severe performance issues – mostly mutex related Used DTrace to identify mutexes and performance issues Eliminated unnecessary mutexes Achieved 30,249,688 tpmC (500,000 tps), 3x best IBM results


Tuxedo Message Queue

New product – merger of Oracle MessageQ and Tuxedo /Q Persistent enqueue performance unacceptable High percentage of kernel time in queue servers DTrace identified shared memory locks and system calls Removed unnecessary locks and system calls (settimer)

Slow message enqueue() performance


Best Practices

Consider performance in process – arch, design, code, review, test– Scaling, availability, parallel execution, contention

Establish repeatable automated benchmarks Things to avoid:

– Porting – build and sanity test regularly on all platforms

– OS kernel - use instead Shared memory, user level mutex, libibverbs

– Context switches – stream and/or poll

Focus one or two late loads primarily on performance


BINOY SUKUMARANSENIOR DIRECTORORACLE DATABASEPLATFORM-SPECIFIC DEVELOPMENT

ORACLEPRODUCT

LOGO


Oracle Database on Solaris

Database – a very sophisticated software product– Development involves millions of lines of code

– Several product components built automated nightly

– Several thousand functional tests run nightly OS and system platform support essential

– Offers the most comprehensive set of Database features out there Relies on low level kernel support

– Key emphasis on performance

Leverages key development tools and OS support on Solaris

Supported on SPARC and x86



Oracle Solaris Studio 12 compiler– Builds the product bundle and day to day development on Solaris– Generates the most optimized machine code

Using aggressive optimization techniques– CPU optimized routines for common library calls like mem* functions

Profile Guided Optimization– Provides added level of performance optimization at object level

– Instrument the Oracle binary and profile based on “common” workload

– Predicts runtime behavior of the oracle binary

– Expected probabilities of branches & execution of program blocks



Leverages a variety of key Solaris technologies– Solaris Zones & Oracle VM for SPARC

Provides isolation in our virtualized development server farm

– For building the product nightly

– Runs thousand’s of nightly Regression tests

– To integrate hundreds of developer transactions/code updates

– AES hardware assisted encryption on T4 and x86



DTrace – Provides excellent diagnosability during development

– Understand and fix program behavior – I/O patterns etc.

– Helps with transient problems that are hard to debug – like process deaths

– Understand and fix performance issues High user time, high system time/calls

– Greatly reduces problem resolution time !



Performance Analyzer– Collect/analyze performance data

– Collector generates profile data Call stacks Hardware counters Memory allocation

– Analyzer reports on the collected metrics Indentify areas of performance issues Resource usage patterns

– Key to ensuring database performance

Supported on SPARC and x86


Next-gen Database stability was improved by using a Studio memory analysis tool to detect stale pointers

New method is 60x faster than current method - test cycle shrunk from hours to minutes

– Faster bug identification and fixing

Improved Reliability of Oracle Next-Gen Database

Studio Stale Pointer DetectionDatabase Test Run-Time (in secs)


STEFAN SCHNEIDERCHIEF TECHNOLOGISTORACLE SOLARIS

ORACLEPRODUCT

LOGO


Solaris Performance and Independent Software VendorsMotivation

Application PerformanceCustomer Acceptance

Supportability of total stack Support costs Development time and costs Product life cycle


Example: Voice over IP ApplicationSPARC T4 with Tuxedo, Spring 2012

Problem: slow call routing– Root cause: database bulk load on same system slowed down call routing

Misconfigured network switch causing spurious interrupts Used Solaris FX class priority 60 for 80+ threads (at the same time)

Diagnostics: DTrace– 10x more network interrupts compared to V890

– Many threads were stuck on CPUs

Result: from 60% system load at 400 CPS on V890 to single digit system load on T4-2 (load generators exhausted...)

400 CPS(V890) 30 CPS (T4-2) 1500 CPS (T4-2)


Example: Telco Application + Red StackSPARC T4 with Tuxedo, Spring 2012

Issue 1: Serialized logging– Root cause: ostringstream() in C++ STL

– Solution: Change request 7144618->Fix in STL and compiler->Run time library got updated

– Lesson learned: Standardized libs benefit as well from software support

Issue 2: Naive memory allocation within application– Solution: use libumem (optimized for concurrent memory allocation)

– Lesson learned: Fast growing applications face paradigm shifts

Tools used: DTrace, mpstat, prstat, plockstat

380 TPS 5000 TPS


Solaris Studio Features that Improved Product Performance

Oracle DB team used compiler’s profiled feedback option resulting in 5-10% better performance (working with Siebel code)

PeopleSoft product code increased level of compiler optimization to “-O4” resulting in ~10% better performance

Compiling Oracle Business Integration EE code with Interprocedural Optimization turned on (“-xipo”) led to ~7% better performance

Example: Oracle ProductsImproved Performance on Solaris

Solaris OS Features that Improved Product Performance

ATG Web Commerce,PeopleSoft, Siebel, JDE benchmarks used: Processor sets to benefit performance and

scalability by 5-10% Solaris zones for scalability and consolidation Priocntl() for some compute intensive process

to speed up performance Large page sizes (256GB) Different memory allocators (libumem or

libmtmalloc) instead of default malloc() in libc Separate threads to handle interrupts


Panel Q & A


Summary

Consider performance during development - its difficult to retrofit Don’t “port” – instead build and sanity test regularly on all platforms Write code to take advantage of each platform’s unique capabilities,

e.g., on Solaris explore using:– Processor sets, large page sizes, different memory allocators, DTrace,

multiple threads for scalability, zones for isolation

Establish repeatable automated benchmarks to measure your progress

Development tool selection matters, using one toolset for all platforms is guaranteed to be sub-optimal


Oracle Solaris 11.1 & Oracle Solaris Studio 12.3Best Place to Run Your Enterprise Applications

World record performance leader

Strongest compatibility guarantee

Non-stop innovation investment

25+ years enterprise experience

Oracle Solaris Oracle Solaris Studio

#1 UNIX for Enterprise

#1 Development Tool suite for

Oracle Systems


More Information Download, technical articles and more: oracle.com/goto/solarisstudio

OpenWorld Sessions

Mon, Oct 1, 10:45 - 11:45 AM: Maximizing Your SPARC T4 Oracle Solaris Application Performance, CON 6382 (Marriott Marquis - Golden Gate)

Mon, Oct 1, 3:15 - 4:15 PM: Technical Panel: Developing High Performance Applications on Oracle Solaris, CON 7196 (Marriott Marquis - Golden Gate)

Hands-on Lab

Wed, Oct 3, 1:15 - 2:15 PM: Develop C/C++ Applications for the Cloud with Oracle Tuxedo and Oracle Solaris Studio, HOL 10276 (Marriott Marquis - Salon 5/6)

JavaOne Sessions

Mon, Oct 1, 8:30 – 9:30 AM: Mixed-Language Development: Leveraging Native Code from Java, CON 6714 (Hilton San Francisco -Continental Ballroom 6)

Tues, Oct 2, 1:00 – 2:00 PM: Take Performance Tuning of Your Enterprise Java Applications to the Next Level , CON 10213 (Hilton San Francisco -Continental Ballroom 6)


The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract.It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Technology

Technical Panel Developing High-Performance Applications on Oracle Solaris