Transcript
Page 1: Performance & Automation Testing

INDIUM SOFTWAREAn Independent Software Testing Firm

Indium SoftwarePerformance & Automation

Testing Services

“Business Assurance through Quality Assurance”

Page 2: Performance & Automation Testing

© 2017 Indium Software

Performance Testing Service Overview

2

Focused Performance Testing practice – over past 10 yearsOver 50 Performance test engagements executed.Expertise in HP Mercury, Microfocus, IBM Rational & Open Source tools.Strong focus on Independent, Objective and unbiased analysis.

Performance Test Centre Services:• Load / Stress / Volume Testing• Fail-Over testing• Scalability Testing• Performance Benchmark/Baseline Testing• Performance Test Tool and Process Consulting

Differentiators – Simple, Flexible and Scalable• Process (with toll gate rigor) focus• Transaction based pricing (per script model)• High ROI on QA Investments• Innovative engagement models• Just in time resourcing model – core and flexible

teams.

Validate Architecture | Validate Response times | SLAs | Validate Scalability

Page 3: Performance & Automation Testing

© 2017 Indium Software3

Proficiency on Tools• Commercial - HP Loadrunner, MS VSTS, OATS, MF-Silk

Performer, IBM’s RPT• Open Source – Jmeter, Badboy, OpenSTA, Grinder

Key Deliverables• Overall Performance Roadmap• Test Plan for all major Releases• Assess high risk aspects of design w.r.t Technology• Comprehensive Test Summary with Observations ,

Recommendations and Extrapolations

Focus Areas - • Application Benchmarking & Predictive Usage Pattern• WAN simulation Using Cloud based load testing• Infrastructure Upgrade – Validation• Mobile application based Performance Testing• Maturity based Performance Engineering Models 

Value Additions• Load Balancer / Memory Bottleneck• Processor Bottleneck in DB & App. Server • Disc I/O constraints / Network Latency• Server Configuration / Network Bandwidth • DB Deadlock / SQL response time / Indexing issues• JVM / JDBC Connections Constraints• Timeout / TCP Connection stability

Performance Test Approach | Overview

Page 4: Performance & Automation Testing

© 2017 Indium Software

SMART Performance Test Strategy

Prioritization Matrix for Business Scenarios

# of Concurrent UsersFrequency of transaction(s) usage Customer Facing vs Financial Impact

Define High level scenarios

Business Processes• Sub-Processes

• Transactions or Process Steps

Define Test Data

Volume of Test Data needed (Static vs Volatile)Mining of historical data for quick test data preparation

Performance Test Environment

VPN connectivity – Configure Static IP address (Offshore) within client firewallApplication access – Test ID creation, user privilegesTools – Depend on the feasibility

Readiness to Start

People – Performance Testers with multiple test tool experienceProcess – Performance Testing process, methodology, scripting, execution, Analysis & ReportingTools – Templates, Jump-Start-Kits

4

SMART stands for Script, Model, Analyse, Report and Tune

Page 5: Performance & Automation Testing

© 2017 Indium Software

Performance Test Schema

5

PC 1 Application / Production Server

PC 3 PC 4

Performance Test Lab:

To conduct load test, monitor performance, analyze and report, remotely.

Load Generators: For hosting Controller & Load Generators for load generation via LAN/VPN onto Application Server. These could be at same location or at different geographies.

PC 2

VPN/LAN

Virtual UsersLoad

PCOffshore (Indium)

PC

CUSTOMER Infrastructure

Page 6: Performance & Automation Testing

© 2017 Indium Software

Performance Test Metrics

6

• OS Resource Utilizations CPU Usage Memory Usage Disk IO Utilization

• Network Utilization Bandwidth Latency Traces

• Servers Utilization (Web /App) Heap Usage & GC JVM JDBC Threads Application Logs

• SLA compliance• Certify the build for production deployment

• Trend analysis• SLA compliance

Metrics

•Test Tool Results•Monitors Data•Diagnostics tool data•Data from Support Teams

Analysis Report

•Execution & Monitors data analysis•Log analysis

•Diagnostics report•Bottlenecks and Root causes

Execution Report

•Response times•Server resource usage statistics•Defect tracking and comparison

Test Summary Report (TSR)

•Consolidated report of al the tests conducted•Recommendations

Page 7: Performance & Automation Testing

© 2017 Indium Software7

Recommendations

Usage distribution – Response time

Server utilization

Performance Test metrics

Deep Dive Performance Dashboard

Artefacts (Samples) – Performance Tests

Page 8: Performance & Automation Testing

© 2017 Indium Software

Test Automation Portfolio

8

• Functional Test Automation, Test Data Generation, Testing Tools Administration, Regression automation

• Multi Browser / Multi OS• Multi Device – PC / Mobile / Tablets

• Usage of reusable functions to ease coding • Reusable Automation Frameworks with proven track record in effort reduction • Process , checklists and templates to act as jumpstart kit

• Web Technologies - Java, .NET, LAMP• Enterprise Apps – SAP, Oracle, Peoplesoft, Salesforce.com• BI / DW, Big Data• Mobile, Social Media, Cloud, Analytics

Services

Jump Start

Technologies

Page 9: Performance & Automation Testing

© 2017 Indium Software

Tools• Commercial - HP QTP, IBM Rational

Functional Tester, Silk Test, TestComplete, Ranorex

• Open Source - Selenium RC, SOAPUI, Watir

Key Deliverables• Test Strategy / Plan• Test Automation Framework• Automation Suite• Test Automation Execution SummaryFocus Areas – <<Customer Name>> : • Define the process for the automation

suite management and maintenance Lorem Ipsum

• Implement Test Automation Framework and develop Test Automation Suites for Smoke , Sanity & Regression

• Automation suite should ensure that , the application build is stable across all major / minor releases which could include defect fixes or new feature enhancements and extend good test coverage

Value Additions• Reporting alerts over email / SMS• Cloud ready• Interface with multiple tools• Hybrid automation framework

9

Test Automation Approach | Overview

Page 10: Performance & Automation Testing

© 2017 Indium Software

Indium’s Selenium Automation Framework Enhanced (iSAFE) Framework

10

Framework Layer

Mobile A

pplication Under Test

Automation Suite

Automation

Scripts

Generic

Libraries

Input Data

Driver Script

Common

Libraries

Object Repository

Config and Global Variables Reporting Engine

Scheduler

Start up/Clean Up

Properties

Execution

Page 11: Performance & Automation Testing

© 2017 Indium Software

iSAFE Capabilities

11

iSAFE Capabilities

Test Reporting in multiple formats

with historical graphs

Ability to work with multiple OS /

Browsers / Devices

Ease of Setup and Execution resulting in

minimum learning

Fully Configurable Tests using

Spreadsheets

Support for API and DB Testing

Email/SMS Text

Status/Voice calls to

Stakeholders

Target based & parallel

execution using Selenium

Grid

Exception Handling and Screen Shot

Management