26
8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 1/26  ALM113 Large SAP Systems, Performance References and Load Testing William Adams Performance & Scalability SAP AG Nikolai Sauerwald Performance & Scalability SAP AG © 2010 S AP AG. All rightsreserved. / Page 2 Disclaimer This presentation outlines our general product direction and should not be relied on in making a  purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP. SAP has no obligation to pursue any course of business outlined in this  presentation or to develop or release any functionality mentioned in this presentation. This  presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.

ALM113 Large SAP Systems Performance References and Load Testing

Embed Size (px)

Citation preview

Page 1: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 1/26

 ALM113

Large SAP Systems, Performance

References and Load Testing

William Adams

Performance & Scalability

SAP AG

Nikolai Sauerwald

Performance & Scalability

SAP AG

© 2010 S AP AG. All rights reserved. / Page 2

Disclaimer 

This presentation outlines our general product direction and should not be relied on in making a

 purchase decision. This presentation is not subject to your license agreement or any otheragreement with SAP. SAP has no obligation to pursue any course of business outlined in this

 presentation or to develop or release any functionality mentioned in this presentation. This

 presentation and SAP's strategy and possible future developments are subject to change and

may be changed by SAP at any time for any reason without notice. This document is provided

without a warranty of any kind, either express or implied, including but not limited to, the implied

warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP

assumes no responsibility for errors or omissions in this document, except if such damages

were caused by SAP intentionally or grossly negligent.

Page 2: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 2/26

© 2010 SAP AG. All rights reserved. / Page 3

Agenda

1. Introduction

2. KPIs and Scalability of SAP Software Architecture

Frontend

Backend – Application Layer 

Backend – Database Layer 

3. Verify Scalability with Load Testing

4. Performance Metrics and Key Volume Drivers

5. Miscellaneous

© 2010 S AP AG. All rights reserved. / Page 4

"Large" systems can be described with very different characteristics

Likely additional users, higher

throughput

Likely legacy/third party software

Higher level of integration More load on interfaces

Less controlled

Possible Evolvement of Large Systems

"home-made" growth

Increasing user numbers, higher

throughput

Increasing number of applications Rather controlled

Same software, perhaps

template approach

Organic Growth Mergers / Acquisitions

Some Characteristics for Large Systems

Need to consolidate

Users connect over WAN

"Central system"

One database One client

 All processes are centrally

managed

 Application operations

Downtimes, planned or

unplanned

Single Global Instance

Page 3: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 3/26

© 2010 SAP AG. All rights reserved. / Page 5

Excerpt Gartner Study on ERP TCO aboutSingle Global Instance ERP

"…

1.1.Will you deploy single-instance ERP

One of the main drivers of ERP TCO is the

number of instances that are deployed and how

they are managed for distributed enterprise. …

Deploying a single instance typically reduces

support requirements, increases consistency in

processes and data, and enables an integrated

view of the organization's data. …

 A single instance is not always possible or

preferable. The more diverse a business, the

more difficult it will be to identify and achieve the

synergies of a single-instance strategy. …

…"

Source: The Ten 'How' Facto rs Th at Can Affect ERPTCO , Gartner RAS Core Research Note G00172356,

Denise Ganly, 1 February 2010, V1RA9 04082011

© 2010 S AP AG. All rights reserved. / Page 6

Feasibility of Large Systems

Feasibility Custom coding

Can SAP software payroll 3,000,000

employees and pensioners in less than

three hours?―

Can SAP software fulfill our requirements

concerning 30 million business partners?―Can SAP software meet the service level

agreement of < 1 sec server response time

in our corporate portal?―Can SAP software handle database tables

of more than 1.5 TB and what are the

implications?―

The most common frequency of

 performance problems is quarterly or

monthly and custom code the number one

culprit.ERP ―Terabyte Club‖ - Critical Operations

from ASUG/AMR, Dec, 2007

Research at customers indicates that most

 performance pain points (i.e. long response

times, long-running jobs) were caused by

custom coding

[...] 

Furthermore, studies reveal that much of

the custom-coded functionality is either

already part of the standard applications or

is hardly ever used Computerwoche, November 2007

Page 4: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 4/26

© 2010 SAP AG. All rights reserved. / Page 7

Excerpt ASUG/AMR Study on Large ERPSystems

"The charts below show the frequency and top 3

root causes of performance problems for Terabyte

Club companies, The most common frequency of

performance problems is quarterly or monthly andcustom code the number one culprit, There are

many excellent performance tools in the SAP

ecosystem which can greatly assist

troubleshooting and proactive performance

tuning."

"For Terabyte Club companies capacity planning

is an essential best practice to support

performance management, not a luxury, Again,

specialist tools are available to help model and

predict performance throughout project roll outand the entire ERP Lifecycle."

Source: ERP “Terabyte Club” - Critical Operations from

 ASUG/AMR, December 2007

Performance Problems:The Most Likely Root Causes

0,0

0,1

0,2

0,3

0,40,5

0,6

0,7

0,8

0,9

1: Custom

Code

2: Database

Too Big

2: Batch

Processing

FREQUENT PERFORMANCE PROBLEMS

Custom code

Large database

Background processing

© 2010 S AP AG. All rights reserved. / Page 8

Agenda

1. Introduction

2. KPIs and Scalability of SAP Software Architecture Frontend

Backend – Application Layer 

Backend – Database Layer 

3. Verify Scalability with Load Testing

4. Performance Metrics and Key Volume Drivers

5. Miscellaneous

Page 5: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 5/26

© 2010 SAP AG. All rights reserved. / Page 9

Typical Load Distribution on the Different Layers*

Browser 

Browser 

Application

Application

Web server 

Web server 

Typical SAP Software Architecture in Layers

Database

Application

Frontend

Database

Application

Database

Application

Browser 

Web server 

Central System, 2-Tier 3-Tier Multi-Tier  

Frontend Frontend Other  

Frontend: 10-30%

Middleware: 5-10%

 Application: 50%

Database: 10-20%

Frontend: 10-30%

Backend: 70-90%

Frontend: 10-30%

 Application: 50-70%

Database: 10-20%

* The percentages can vary significant ly, depending on the complexity of the business logic, the user interface, the programming language or whether huge amounts ofdata are manipulated at DB level (Business Intelligence).

© 2010 SAP AG. All rights reserved. / Page 1 0

Possible performance metrics:

Response time – user experience / productivity

System throughput  – business requirement,

system resource consumption, IT budget

Good performance:If the performance metrics match the business

requirements or Service Level Agreements

(―expectation handling‖)

Several parts that constitute the end-to-end(E2E) response time:

Server response time

Network time

Frontend time

Performance, Response Time, and Throughput

Application Server 

Application Server(s)

Database

Web Server(s)

   R  e  s  p  o  n  s  e   T   i  m  e  =   Σ

    t   [   i   ]

t[1]

t[2]

t[3]

t[4]

t[5]

t[6]

t[i]

Browser 

Page 6: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 6/26

© 2010 SAP AG. All rights reserved. / Page 11

KPIs for Network Performance in a Wide AreaNetwork (WAN)

Transferred data volume  – bandwidth of a network connection

Number of network roundtrips  – latency time of a network connection

Performance Design Pattern One network roundtrip per user interaction step

Keep the data volume as low as possible (approx. 10 ~ 20 KB per user interaction step)

Major concepts to optimize WAN performance for browser-based UIs (HTTP)

Browser caching, supported in entire SAP NetWeaver infrastructure

Only for static content like icons, JS, CSS, etc.

Dynamic and security-relevant content doesn't appear in browser cache

HTTP Compression, supported in entire SAP NetWeaver infrastructure

 Achieving a factor 4 -10 for data reduction, depending on the content and size

Use SAP NetWeaver Accelorated Application Delivery (AccAD)

Tunneling network connections using highly efficient data compression

Use Windows Terminals Server (WTS)

Only for specific groups of users with extremly bad network qualities

 Addtional hardware and software needed

© 2010 SAP AG. All rights reserved. / Page 1 2

Example Connections Single Distance in Kilometers Theoretical RoundtripTime*

Actual Avg. Measured in aProject

Satellite connection 45,000 miles / 75,000 km 500 ms 600 ms

Europe – US East Coast 4000 miles / 6400 km 70 ms 170 ms

US East Coast – US West Coast 2600 miles / 4200 km 42 ms 85 ms

US West Coast  – Europe 6600 miles / 10,000 km 100 ms 210 ms

 Asia (China) – Europe 5500 miles / 8600 km 85 ms 400 ms

Impact of Latency on E2E Response Times

E2E response time measurements

of user interaction steps, using aWAN simulation

Number of network roundtrips of a

user interaction step drives the

increase of the network time

4 roundtrips: User Logon

2 roundtrips: SO OWL, New SO

1 roundtrip: WoC SO,

Sold2Party, Add Row, Line

Item, Save

Page 7: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 7/26

© 2010 SAP AG. All rights reserved. / Page 13

Recommendations to Improve WANPerformance

1 Consider employing a browsersetting policy to avoid roundtrips

Cache sizes: ~ 100 MB

HTTP: allow compression by selecting HTTP 1.1

(accept encoding) De-select

Do not save encrypted data to disk

Empty temporary Internet folder 

Keep roundtrips and amount oftransferred data at a minimum incustom applications

Use compression techniques

 Avoid synchronous round trips in WAN

Typical avg, bandwidth per user interaction step

(browser): 10-20 KB

Consider potential additional load Central services, such as local printing will need to be sent

over the WANIf you upload data from scanners (e.g. barcode) or RFID

WAN traffic

Balance infrastructure costs withperceived performance

Consider a mixed access approach: e.g. BI users may

work through WTS

Consider SAP NetWeaver Accelerated Application Delivery

Tool-based optimization of network traffic in browser-based

applications

-

-

-

-

Since latency cannot be tuned, avoid "chatty" applications

-

© 2010 SAP AG. All rights reserved. / Page 1 4

Agenda

1. Introduction

2. KPIs and Scalability of SAP Software Architecture Frontend

Backend – Application Layer 

Backend – Database Layer 

3. Verify Scalability with Load Testing

4. Performance Metrics and Key Volume Drivers

5. Miscellaneous

Page 8: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 8/26

© 2010 SAP AG. All rights reserved. / Page 15

Small test

system Small data

source

One user 

Thousands of users Terabytes of data

High throughput Large objects

Globally distributed users

Hardware adaptivity

Many servers

Dimensions of internal and external scalability Effects of scalability

Dimensions and Effects of Scalability

Size of

data

Number ofobjects in

the DB

Number ofprocessed

objects per

time

Concurrent

users

Number

of servers /

cores /

threads

Scalability

Number of

parallel jobs

© 2010 SAP AG. All rights reserved. / Page 1 6 Time

Ensure Linear Scalability in ResourceConsumption

Processing time

Linear behavior can be best checked withvarying object numbers and sizes, and varying

number of concurrent users

Memory consumption

Efficient data design and release no moreneeded memory as early as possible

Using short living transactions / sessions in

 ABAP can easily eleminate memory leaks

Extensive garbage collection activity in Java

impacts the performance and scalability

significantly

Number of processed units / conc. users

CPU utilization(%)

increases linearly

to 100%

10,000

   T   i  m

  e

100

   J  a  v  a   h  e  a  p  u  s  a  g  e

CPU consumption per processed

unit / conc. user remains constant

Response timebehaves according

to queuing theory

Page 9: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 9/26

© 2010 SAP AG. All rights reserved. / Page 17

Most Important Performance Topics on theApplication Layer 

Linear resource consumption CPU time

Linear to the number/size of

processed objects

Memory usage

Unintended long-lived/growth

of internal tables / strings

Pseudo- garbage

 Anonymous data objects that

are pseudo-garbage

Communication load Potentially less communication

overhead in single instance

Potential impacts on

performance

CPU consumption on sending

and on receiving system

Granularity: Fixed overhead per

message

Serialization, compression and

authorization

Choose the right granularity of

message size, especially in WAN

Interfaces

Many processes in parallel Higher demand on system

components (CPU, memory, I/O

system, network, …)

Balance workload

Fixed or dynamic distribution

Give preference to higher

number of small packages than

only a few large packages

 Avoid deadlocks

Follow sorted sequence

Keep locks as short as possible

choose right granularity

Consider impact on disk I/O

Parallel Processing CPU and Memory Resources

Effects on the Application Layer

© 2010 SAP AG. All rights reserved. / Page 1 8

Parallel Processing: Choose the RightGranularity of the Work Items

Often, customers tend to choose rather large

 job sizes For example they do their billing according to

country

If the country is very large, the individual job may

take very long and block central resources

Processing time of single step processing time of

the whole chain

 Advantages of configuring small jobs

More even load distribution decrease of lockingtime on central objects

No preprocessing needed to take the largest jobs

first

In case of non-performance-optimized coding,

negative effects are not as negative as in large jobs

 Advantages of configuring large jobs

Easier defined and established, for example on

business units

So, what is a useful package size?

Too small message overhead

Too big possible problems with workload

balancing, data transfer, memory requirements,

runtime restriction for DIA work processes

Therefore: test for optimal throughput

Throughput

optimalNumber of parallel processes

Page 10: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 10/26

© 2010 SAP AG. All rights reserved. / Page 19

Interface Design – Classify the Interfaces You

Need

 According to time constraints

Batch-type bulk interfaces

Bulk transfer smaller performance footprint

 Advanced scheduling possible, perhaps once perday

Near real-time interfaces

Single transfer larger performance footprint

 Asynchronous, immediately after transaction viamessage protocol, for example

 –  Asynchronous services calls

 – Notification messages to one receiver 

 – Information messages (multiple receivers)

 –  Asynchronous q/t-RFCs

Synchronous interfaces

Single transfer larger performance footprint

Wait for the answer: for example Available-to-Promise check

Two systems up and running

 According to technical aspects

Standard protocols such as Web Services

via SOAP potentially mediated via PI

Easy and standardized integration

Large flexibility

larger performance footprint

SAP-proprietary protocols such as RFC or

simple non standardized file interfaces

Larger initial integration efforts

Less flexible

smaller performance footprint

In the end, it's a price performancequestion

© 2010 SAP AG. All rights reserved. / Page 2 0

Recommendations for the Application Layer 

1 Many scalability issues on the

application layer can be solvedby adding hardware capacity

Number of CPUs/cores/threads

Many slower ones versus few fast ones mayimprove parallel processing

 A faster CPU yields better response times in dialog

mode

Memory Make sure there are no memory leaks in your custom-

coded application

Linear CPU time  Analyze different dimensions of scalability (number,

size), use more than two measurement points

Response time measurements do not prove scalability

Programs designed to run inparallel

Release locks as soon as you can

 Analyze the program for bottlenecks and remove them

as soon as you can

Consider locking time, granularity of the ENQ locks and

importance of the object for the system

 Are there work items competing for the same locking

objects?

Interface If you need to design interfaces, carefully weigh the pros

and cons

-

-

Beware: Adding hardware does not necessarily solve performance issues!

-

-

-

Page 11: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 11/26

© 2010 SAP AG. All rights reserved. / Page 21

Agenda

1. Introduction

2. KPIs and Scalability of SAP Software Architecture Frontend

Backend – Application Layer 

Backend – Database Layer 

3. Verify Scalability with Load Testing

4. Performance Metrics and Key Volume Drivers

5. Miscellaneous

© 2010 SAP AG. All rights reserved. / Page 2 2

Possible Effects of Large Databases (1/2)

System availability Use of resources System performance

Downtime Lossesby Application

Typical HourlyLoss of

UnplannedDowntime (US$)

Financial/Trading $2,400,000

Supply Chain $600,000

ERP $600,000

CRM $480,000

E-Commerce $480,000

E-Business $480,000

Busine ss Applica tion $300,000

Database $300,000

Messaging $60,000

Infrastructure $42,000

Data Records10 GB

Mirrors20 GB

   A  r  c   h   i  v   i  n  g

2 GB(Compre ssion 20%)

System Copies80 GB

0

500

1000

1500

2000

2500

Month 1 Month 2 Month 3 Month 4

MB or msec

MSEG MB MB51 ms ec

Upgrade to higher software

releases, Runtime for backup and

recovery

Hardware costs for disk, CPU,

memory as well as administration

costs

Response times in dialog mode

for all employees,

Optimal coding ensures access

times independent of DB size

Page 12: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 12/26

© 2010 SAP AG. All rights reserved. / Page 23

Background jobs usually have a fairly high DB load

If there is no night (i.e., in single global instances),batch jobs may compete with (and win) user

requests increase in end-to-end response time

Danger of time-outs because of runtime limitation

caused by system parameter settings (i,e, adjust

max_wprun_time)

Possible Effects of Large Databases (2/2)

Disk I/O Parallel dialog and background job usage

Size of single spindle vs. access speed, one access

at anyone time CPU processing capabilities much faster than

read/write operations

Database cache must grow linearly with DB size

often overlooked

Size of file system cache

0

10

20

30

40

50

60

70

80

90100

   0 2 4 6 8   1   0   1   2   1 3 5 7 9   1  1

User load

Batch load

Example: CPU load profile in parallel user/batch mode

© 2010 SAP AG. All rights reserved. / Page 2 4

Most Frequent Mistake

Poor DB performance due to missing or sub-optimal indexes

Ensure proper index design

 All frequently executed database accesses (selects, updates and deletes) must be supported by

indexes

 A highly modified table, i.e., a table with many INSERT or UPDATE or DELETE operations,

should not have too many indexes, The indexes must be maintained by the DBMS, too

The more columns an index has, the higher the chance that an UPDATE will affect one of the

indexed columns

More than 10 msec in the column Minimum Time / Record in Performance Trace (ST05) suggest

improper indexing use explain function

Good index design requires coordination and a potential compromise among alldevelopers that need to access the data

Page 13: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 13/26

© 2010 SAP AG. All rights reserved. / Page 25

Recommendations for Database Performance

1 Ensure proper index design    All frequently executed database accesses (selects,

updates and deletes) must be supported by indexes

Make sure you do not keep datain the database any longer thanneeded

Combination of data archiving, retention management

(Bear in mind legal implications of system shut down )

Parallel Processing High load requirements demand careful implementation

of DB accesses

Efficient database access (result set, search area, use

of caches and proper indexes)

Check the I/O rate   Use transaction ST04 to check the I /O rate

DB file sequential read for read operation

Logfile synch for synchronous write logging most

important operation, If slow (exceeds 15 msec), overallperformance affected

 Another indicator is if you observe an increased runtime

over several days

With high throughput parallelprocessing spend extra time onI/O design

Disk I/O (sufficient number of spindles) virtual I /O in

high-load systems not yet best practice

-

-

-

-

-

© 2010 SAP AG. All rights reserved. / Page 2 6

Agenda

1. Introduction

2. KPIs and Scalability of SAP Software Architecture

Frontend

Backend – Application Layer 

Backend – Database Layer 

3. Verify Scalability with Load Testing

4. Performance Metrics and Key Volume Drivers

5. Miscellaneous

Page 14: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 14/26

© 2010 SAP AG. All rights reserved. / Page 27

Some Key Performance Indicators

CPU  Processing times of business transactions or tasks

Cost factor: Number and processing power of servers

Disk size

Disk I/O

Data that resides on the database

File read and write activity to storage

Cost factors Backup/recovery depends on size of database

Storage capacity

Memory

 Allocated to a user or background process

Garbage collection, acceleration, planningcapabilities, buffers, caches

Cost factor: Physical memory slots

Front-endNetwork

Load

Transferred amount of data

Network time and roundtrips

Cost factor: Leasing bandwidth

© 2010 SAP AG. All rights reserved. / Page 2 8

Ensuring Scalability With Performance Tests  – 

Approaches

Single User Test

Small test system

(QA, development), one user 

Volume Test

Equivalent to multi-user test, stresstest, load test, benchmark

Quality and implications ofaccesses to persistence layer 

Linear resource consumption

Parallel processingmechanisms, load balancing

Memory usage and memory

leaks

 Analyze & measure

scalable behavior 

Performance predictions

for high volume environment

Verify assumptions

and ...

Page 15: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 15/26

© 2010 SAP AG. All rights reserved. / Page 29

Scalability Testing Test Cases

The test cases are:

Representative

Reproducible

Right to the point

Representative in the sizing environment includes

Bread-and-butter transactions because they cause the greatest load, such as sales orderentry

Critical business cases that must run in a tight time frame, such as settlement runs at the endof a period, nightly MRP runs

Reproducible test cases

Conducting the test several times must not change the load behavior 

Can be rapidly understood and followed through

Remain the same over the years/releases

Right to the point

Pragmatic, we don't want to squeeze out the last 10%

© 2010 SAP AG. All rights reserved. / Page 3 0

Testing for Linear Resource Consumption

To test whether the solution is scalable, perform ―linearity tests―.

Scalability tests are basically a repeatable pattern for (example):

Scalability with an increasing number of objects

1 object, five line items

10 objects, five line items

100 objects, five line items

Scalability with an increasing of line items

1 object, 1 line item

1 object, 10 line items

1 object, 100 line items

The results of these controlled tests can be analyzed to determine if there are anyperformance bottlenecks

 Again by understanding the resource consumption of a system that was writtento scale

Tests like these enable more detailed questionnaires, where you do not rely onparticular test scenarios but can ask for variations

Page 16: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 16/26

© 2010 SAP AG. All rights reserved. / Page 31

Scalability Testing EvaluatingMeasurement Results

 Application

DB

Total

0.0000

0.0200

0.0400

0.0600

0.0800

0.1000

0.1200

0.1400

0.1600

0.1800

0.2000

0 10 20 30 40 50 60 70

No. of bidders

   C   P   U

   t   i  m  e

Make sure you define at least three measurement points per scenario – the more, thebetter 

Example: Scalability with the number of bidders – three tests

 Another set of tests would include, for example, 10 bids with 5, 50, and 150 line items, respectively

© 2010 SAP AG. All rights reserved. / Page 3 2

Scalability Testing Steps

Provide CPU processing time, Memory consumed, Disk Access, and Network timefor all tests described using the following:

For AS ABAP and AS JAVA:

Disk Analysis DB02

CPU Analysis ST03N, STAD

User Analysis ST07, STAD

Memory Analysis SM04, STAD

Front-End Network Load STAD, ST03N

CPU and memory status ST06

For AS JAVA:

NWA: Open SQL Monitors

Wily Introscope

SAP JVM Profiler 

Page 17: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 17/26

© 2010 SAP AG. All rights reserved. / Page 33

Scalability Testing System Characteristics

 Aspect Comment

Generalperformanceand stability

Transports to the test system are not allowed as they may change Coding, DDIC objects and customizing settings

Try to be alone If this cannot be helped, make sure the additional users do not change coding, DDIC objects and

customizing data or settings

Optimally, no other processes, systems or DBs should be active on the server  If there are, they should not incur extra load

Prefer central system: Appl. Server and DB on one machine  A system with one application server only may do as well For batch process testing a central server is best

CPU Measurements are extremely platform-dependent Impaired by technologies that simulate several logical CPUs

(hyperthreading, hardware-multithreading, …)

For reliable measurements: turn it off  If you can't, you must make sure the load on the system is kept at a minimum, below 10%

in peak

 Allocate at least two CPUs Three or four would be even better  The less CPUs you have available, the more diff icult it is, because of potential "interfering" users,

batch jobs, etc...

Memory Memory must not become a bottleneck SAP memory (Extended Memory, Roll, Paging) Physical memory (avoid OS paging by all means)

© 2010 SAP AG. All rights reserved. / Page 3 4

Scalability Testing Steps (1/5)

1. We assume that your process fulfills one of the criteria given previously for

performing load tests

2. Determine the expected throughput figures and/or user and system response

times

For example:

Upload of 500,000 sales order line items in two hours

23 million emails in eight hours

Processing of 100,000 workflow items per hour 

Processing of nn HTTP requests per sec from mm parallel users

Usually these figures are part of a requirement or

specification document

3. Determine the complete set of functionalities needed for the load test and the

resulting software components, for example:

 ATP (Available-to-Promise) Check in APO system,

Pricing in IPC,

Workflow or interfaces to other systems

Page 18: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 18/26

© 2010 SAP AG. All rights reserved. / Page 35

4. Establish test scenarios for the processes

Specify in detail and document well

Process steps Input/output for the steps

Customizing environment

5. Determine the data distribution – i.e., the type of master data you need

 Avoid artificial bottlenecks; for example, all orders use the same customer and products

It is not enough to create a vast amount of data by simply copying it

6. Perform the initial capacity planning for the load test system

Based on measurements in the test or development system of the same or very similartest scenario

Scalability Testing Steps (2/5)

© 2010 SAP AG. All rights reserved. / Page 3 6

7. Define procedure for repeatable test executions

Starting conditions must allow reproducible measurements

It might be necessary to ensure this, using a backup and restore strategy; in other

cases, a clean-up program may be sufficient

Even if you test in a small system, you ought to have regular back-ups in order to start a

test from scratch again or to repeat a test phase

8. Create test system and system infrastructure

Ensure that the system landscape is well-configured

Prepare test environment

In some cases it makes sense to start testing with a small system that you gradually

extend

9. Determine the methods of load and user simulation

Decide which tools are used for generating load

Some tools require expert knowledge

Scalability Testing Steps (3/5)

Page 19: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 19/26

© 2010 SAP AG. All rights reserved. / Page 37

10. Create the actual test data environment

For example: Create the business partners which are used in sales order processing

11. Run the single-user test

Measure and analyze the performance

Eliminate performance bugs

Check scalability of application

Linear behavior 

Critical path lengths and locking

Proper indexes

12. Run the multi-user load test Start with a small load and increase load step-by-step towards the final throughput

target

Measure the throughput

Measure the resource consumption using appropriate monitoring tools

Optimize system parameter settings and repeat the test executions if necessary

Scalability Testing Steps (4/5)

© 2010 SAP AG. All rights reserved. / Page 3 8

13. Evaluate the test results

Document the achieved throughput numbers and the corresponding resource

consumption

Document the test system environment (hardware and software releases) carefully

The numbers may be needed for sizing of further systems

Scalability Testing Steps (5/5)

Page 20: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 20/26

© 2010 SAP AG. All rights reserved. / Page 39

Agenda

1. Introduction

2. KPIs and Scalability of SAP Software Architecture

Frontend

Backend – Application Layer 

Backend – Database Layer 

3. Verify Scalability with Load Testing

4. Performance Metrics and Key Volume Drivers

5. Miscellaneous

© 2010 SAP AG. All rights reserved. / Page 4 0

What Are Typical Metrics to Define LargeSystems?

Number of CPUs/cores (ondatabase on app servers)

Number of dialog steps* Size of Database & databasegrowth

Number of servers Number of concurrent users Data volume per time period(e.g. n messages / second)

Total number of systems(incl, legacy)

Traffic on the interfaces Size of memory

Actual total CPU time pertime period

... ...

* User interaction steps frontend-backend where data are transferred

On the one hand very easy and on the other hand not really meaningful

For example: what are users?

How many dialog steps would a background-driven system have?

Page 21: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 21/26

© 2010 SAP AG. All rights reserved. / Page 41

Remote Office

Remote Office

Remote OfficeRemote Office

Remote Office

Data Cente r 

Disclaimer 2

The data on large customers are examples

SAP does not systematically collect performance references

Exception Early Watch Alert: customers send extracted technical data to SAP (voluntary)

Other examples are first-hand experience no general statements can be derived

Out of date as of today

Out of scope

TCO discussion

Business value for moving single instance

Price/performance

 Architecture/landscaping/

configuration/virtualization

© 2010 SAP AG. All rights reserved. / Page 4 2

A Little Analysis: Relationships BetweenActive Users And Various KPIs

Source: Early Watch Alert

Top 50 systems accordingto the number of

interaction/dialog steps

per week

Users high

Users

mediuUs

ers

low Users DB Size in GB DB Year Dialog steps

(week)

CPU Time

Page 22: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 22/26

© 2010 SAP AG. All rights reserved. / Page 43

Examples for "Large" or "Challenging"Systems

Performance is driven by the application demands, not necessarily by technology

Customer A: Consumer Industry

Processing of customer orders

24 million items per day, 1.2 million sales orders

 Assume an evenly distributed work day of 10 hours

2.4 million items per hour 

Doable, confirm with sap.com/benchmark SD 2-

tier 

Locks on a central DB table

 At least one identical product is contained in each

order

120,000 updates (i.e. locks) on the same table per

hour 

33 requests per second that cannot be parallelized

The Catch

Customer B: Consumer Industry

Keeping historical information Sales history from Point-of-Sale kept for 2 years

100,000+ products in 5000+ stores

Quite a large database table One database table with 20 billion entries

The table had ~ 7 columns and the size was ~7 TB

The Catch

© 2010 SAP AG. All rights reserved. / Page 4 4

Examples for "Large" or "Challenging"Systems

Performance is driven by the application demands, not necessarily by technology

Customer C: Automotive Industry

Processing of sales order items

30 legacy systems overall, SAP is currently only

one of many

SAP load profile: 75,000 order line items per day

Time constraints

Processing needed to be done in 15 minutes!

75,000/(15*60) = 80+ order items per second

Including Available-to-Promise check

The Catch

Customer D: Consumer industry

100,000 users in corporate portal

More than 40,000 logins anticipated per peak hour 

1 sec, server response time for log-in

Keep the home page very lean

20 physical servers, 40 J2EE server processes, 2

J2EE server processes per node (~40,000 SAPS)

The Catch

Page 23: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 23/26

© 2010 SAP AG. All rights reserved. / Page 45

Examples for "Large" or "Challenging"Systems

Performance is driven by the application demands, not necessarily by technology

Customer E: Consumer Industry

Managing large databases

30 TB in ERP system

One financials table with 1.2 TB

 Another financials table with 2 billion entries

Handling system downtimes

"Database growth cannot be stopped, it can only

be controlled" archive 24 TB per year 

 Archived data are already compressed, projected

growth would have been 100 TB

The Catch

Customer F: Service Industry

Thousands of downloads from extranet One central global extranet for users to download

software packages

RTT up to 20sec, plus download time for package

Packages Sizes ranged from KB to GB Split Extranet into regions, have all packages

available there

Use third party vendor to create network of reverse

proxies and cache data near where it is being

downloaded

The Catch

© 2010 SAP AG. All rights reserved. / Page 4 6

Business, Architecture & Technology

Performance feasibility evaluations must beginn with the business process

Technology is secondary, as it is a moving target

Business Process Evaluation

 Are the key business processes ready for high-volume/multi-user?

How close to the original template idea can you stick?1

Software Architecture Evaluation

Can you avoid bottlenecks in the standard design? Do you consider performance/sizing already in the design phase

of your custom code?2

Technology Evaluation

Is the hardware capable enough?

Is the hardware within economical limits?3

3

2

1

Page 24: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 24/26

© 2010 SAP AG. All rights reserved. / Page 47

Agenda

1. Introduction

2. KPIs and Scalability of SAP Software Architecture

Frontend

Backend – Application Layer 

Backend – Database Layer 

3. Verify Scalability with Load Testing

4. Performance Metrics and Key Volume Drivers

5. Miscellaneous

© 2010 SAP AG. All rights reserved. / Page 4 8

Ensure Scalability: Improve Performance

 Add hardware

 Additional application servers

(consider impact on DB server)

 Additional CPUs/memory on server(s)

Tune the system

Profile parameters

Increase buffer sizes

Re-schedule load

Distribute load more evenly acrossservers and time

Tune the application

Customizing, configuration

Function and performance

Optimize the coding

Requires time, knowledge, …

0

1

2

3

4

5

Expertise

Veryfast

Testingeffort

Sustainability

Farreach

Highrisk

Add hardwareTune the systemRe-schedule loadOptimize the codingTune the application

Possible Measures Consequences

Page 25: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 25/26

© 2010 SAP AG. All rights reserved. / Page 49

Concluding Remarks

Performance is hardly ever described by technical terms alone

Demands from the application design must be considered

Challenge to turn business requirements into technical requirements

 Application/implementation team and IT-team must work together 

"Largeness" is defined by very different characteristics

make sure to include them in your considerations

In principle, SAP software scales very well

Proven by SAP Standard Application Benchmarks (www.sap.com/benchmark)

Proven by customer load tests

There are a number of potential bottlenecks

Network environment, especially latency limitations

Linear scalability on the application server, memory leaks, batch processing

Database access design and implementation

Disk I/O, especially in mass-processing environments

 

© 2010 SAP AG. All rights reserved. / Page 5 0

Further Information

SAP Service Marketplace

http://service.sap.com/performance Information about load and volume tests: procedure, examples

Recommendations for system tuning (http://service.sap.com/performance --> Media

Library)

Guidelines for efficient programming (http://service.sap.com/performance --> MediaLibrary)

http://service.sap.com/sizing

http://service.sap.com/quicksizing

Page 26: ALM113 Large SAP Systems Performance References and Load Testing

8/9/2019 ALM113 Large SAP Systems Performance References and Load Testing

http://slidepdf.com/reader/full/alm113-large-sap-systems-performance-references-and-load-testing 26/26

Contact

Feedback

Please complete your session evaluation.

Be courteous — deposit your trash,

and do not take the handouts for the following session.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG.The information contained herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation.

IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, System z9, z10, z9, iSeries, pSeries, xSeries, zSeries,eServer, z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390 Parallel Enterprise Server, PowerVM, Power Architecture, POWER6+, POWER6, POWER5+,POWER5, POWER, OpenPower, PowerPC, BatchPipes, BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, Parallel Sysplex,MVS/ESA, AIX, Intelligent Miner, WebSphere, Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM Corporation.

Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.

 Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or registered trademarks of Adobe Systems Incorporated in the United States and/or othercountries.

Oracle is a registered trademark of Oracle Corporation.

UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.

Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc.

HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.

Java is a registered trademark of Sun Microsystems, Inc.

JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape.

SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer and other SAP products and services mentioned herein as well as their respectivelogos are trademarks or registered trademarks of SAP AG in Germany and other countries.

Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcels ius, and other Business Objects products andservices ment ioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. in the United States and in othercountries.

 All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only.National product specifications may vary.

The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without theexpress prior written permission of SAP AG.

This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies,developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/ordevelopment. Please note that this document is subject to change and may be changed by SAP at any time without notice.

SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or otheritems contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of

© 2010 SAP AG. All Rights Reserved