ArcGIS for Server: Performance and Scalability Testing Methodologies

Andrew Sakowicz

Definitions

Performance

• Speed, e.g. response time (seconds)

Scalability

• The ability to increase output and maintain acceptable performance

Capacity

• The maximum level of output the system can produce, e.g.• X cars/sec• X maps/sec

At capacity Over capacity

Bottleneck

• Resource(s) limiting the performance or capacity

Not bottleneck bottleneckThink of :Lanes -as CPU processorToll -as ArcGIS Server instancesCars -as map requests

Step Load and Response Time

Response Time (sec)

time

Step Load (users)

Throughput (request/hr)

Throughput(req/hr)

Response Time (sec)

time

Step Load (users)

Resource utilization: CPU, Memory, Network

Throughput(req/hr)

CPU Utilization (%)

Memory used (Mb)

Network used (Mbps)

Response Time (sec)

time

Step Load (users)

Capacity

Throughput(req/hr)

CPU Utilization (%)

Memory used (Mb)

Network used (Mbps)

Content length (bytes)

Response Time (sec)

Time

User load

Capacity (~ 85% utilization)

Process and Tools

Process and Tools

Relationship between System Tools

User LoadCPU%

Capacity models

Performance Tests

System Tools frameworkSystem Tools are not just tools

Tool

Patterns

Discipline

Infrastructure Capacity Planning

Provide sufficient hardware resources

GIS Systems are bound by:1. CPU - typically2. Memory – when large number of services3. Disk – Image Service, Synchronization4. Network – low bandwidth deployment5. Poorly configured virtualization can result in 30% or higher performance degradation

Most systems are CPU bound

Most well-configured and tuned GIS systems are CPU bound.

InfrastructureArcSOC Process Memory requirements

Wide ranges of memory consumptions

Service Type Low HighMap 50 MB 500 MB

Image 20 MB 1,024 MB

GP 100 MB 2,000 MB

System DesignerSolution Architecture design methodology

• Gathering requirements

• Designing

• Capacity: CPU, Network, Memory

• Reporting

System Designer output

Performance Testing

Tuning methodologyProfile each tier starting from the top

Browser

Web Server

ArcGIS Server

ArcSOC

Total Response Time (t1-t2)

Wait Time

Search & Retrieval Time

Usage Time

ArcSDE/DBMS

t1 t2

FiddlerFiddler measurement approximately 5.2 seconds

ArcGIS Server 10.3.1 Statisticshttp://server.arcgis.com/en/server/latest/administer/windows/about-server-statistics.htm

• Total requests• Average response time• Maximum response time• Timeouts• Maximum running instances• 30 min resolution reports

ArcGIS Server Logshttp://www.arcgis.com/home/item.html?id=90134fb0f1c148a48c65319287dde2f7

System Monitor –ArcGIS Server Statistics

• https://systemmonitoring-emcs.esri.com/#/arcgis/ESLSRV12• User: esridemo• Password: esridemo

Mxdperfstathttp://www.arcgis.com/home/item.html?id=a269d03aa1c840638680e2902dadecac

Oracle TraceCompare elapsed time

Elapsed time slightly changed due to different test runs

Oracle Execution plan

Inefficient spatial index

Testing process

Application

GIS Services

Infrastructure: Hardware and Software

Required skill set Configuration, Tuning, Testing

Testing

Configuration

Tuning

System Test Tool featuresGIS Test Automation

• ArcGIS Services- Mapping- Feature Service- OGC- Geocoding- Image Service - Network Analyst- Geoprocessing- Tile Cache

• Application Testing• Discipline relevant report

Application

GIS Services

Infrastructure: Hardware and Software

Test tools feature comparison

Tool Cost LearningCurve

OS Metrics GIS Data Generation

GIS Test Automation

Load Runner High High Windows/Linux No No

Visual Studio Medium High Windows No No

JMeter Free High Requires additionalplugin

No No

System Test Free Low Windows/Linux Yes Yes

Tech Support by Esri PS as part of consulting support

Performance testingValue

• Identify bottlenecks• Determine system capacity• Demonstrate performance SLA

DemoSystem Test

System Monitoring

Monitoring Enterprise GISChallenges

• Multiple administrators • Multiple disparate monitoring/diagnostic tools• Data collected in a reactive fashion: on demand and for limited time• Correlation of data with different timestamp is difficult• ArcGIS administrators do not have access to all tools, data and reports• Challenging to quickly identify the root cause and take appropriate measures

• Many excellent monitoring tools on the market• Few provide GIS dashboards• System Monitor can be used as reference implementation

Standards for effective GIS monitoring

Enterprise GIS effective monitoring“PIECE” of mind with System Monitor

• Proactive• Integrated

- Dashboards across all tiers

• End-to-End- All tier monitoring

• Continuous- %Coverage provided

• Extendable- Custom queries

Network

Hardware

Web Server

ArcGIS Server

Geodatabase

RDBMS

Monitoring Value

• Proactive validation: Configuration

Resource Utilization

Usage Trends

Performance SLA

Uptime SLA

DemoSystem Monitor

System Tools download

• http://www.arcgis.com• owner:EnterpriseImp• Show ArcGIS Desktop Content

Demohttps://systemmonitoring-emcs.esri.com/Portaluser: esridemopassword: esridemo

Applied use of System Monitor and Test tools

Use Cases

DemoSimulate CPU spike (e.g. Antivirus scan)

DemoSimulate ArcGIS user load

DemoIntermittent slow performance: Networks latency (SM extensions)https://systemmonitoring-emcs.esri.com/#/url/eslsrv12_6080_arcgisoutput_1mb_file

DemoArcGIS Server services stopped

DemoArcGIS Server service usagehttps://systemmonitoring-emcs.esri.com/apps/dashboard_sample/#/ArcGIS

DemoUnbalanced resources:ArcGIS Server clustershttps://systemmonitoring-emcs.esri.com/apps/dashboard_sample/#/ArcGISConfig

DemoArcGIS Server slow serviceshttps://systemmonitoring-emcs.esri.com/apps/dashboard_sample/#/ArcGIS

DemoArcGIS Server service usagehttps://systemmonitoring-emcs.esri.com/apps/dashboard_sample/#/ArcGIStuning

DemoArcGIS Server instance configurationhttps://systemmonitoring-emcs.esri.com/apps/dashboard_sample/#/ArcGISInstances

Please Take Our Survey!

Select the session you attended

Scroll down to find the survey Complete Answersand Select “Submit”

Download the Esri Events app and find your event