Case Study: Nordstrom Succeeds in E-Commerce with Innovative Performance Testing to Reduce Cycle Times and Improve Quality

Nordstrom Succeeds in E-Commerce with Innovative Performance Testing to Reduce Cycle Times and Improve Quality

Gopal Brugalette

DevOps: API Management and Application Development

Nordstrom

Sr. Applied Architect Performance Engineering

DO3T21S

#CAWorld

2 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

© 2015 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.

The content provided in this CA World 2015 presentation is intended for informational purposes only and does not form any type

of warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA.

For Informational Purposes Only

Terms of this Presentation


Abstract

Using SV and TDM solutions from CA and w/ CA Cloud Test for load test execution, Nordstrom has reduced cycle times/support effort and increased accuracy in the performance testing of its e-commerce platform, store and supporting systems.

In this session, Nordstrom will share its challenges to ensure quality and the approaches it has taken to overcome them, along with insights into lessons learned along the way.

Gopal Brugalette

Nordstrom

Sr. Applied Architect Performance Engineering


Agenda

INTRODUCTIONS

PERFORMANCE AND THE CUSTOMER EXPERIENCE

THE FUTURE

CHALLENGES

USE CASES

RESULTS AND CONSIDERATIONS

1

2

3

4

5

6


Innovative Performance Testing

to Reduce Cycle Times and

Improve Quality

GOPAL BRUGALETTE

Senior Applied Architect

Performance Engineering


1901 Founded

65,000 employees

121 Nordstrom stores

194 Nordstrom Rack stores


SENIOR ARCHITECT, PERFORMANCE ENGINEERING

GOPAL BRUGALETTE

MANY TOUCH POINTS, SAME GREAT EXPERIENCE


PERFORMANCE IS A FEATURE

0 2 4 6 8 10 12

Load Time (s)

Co

nve

rsio

n R

ate

How does Load Time impact

Conversion Rate?


e-Commerce

Availability

Order Mgmt

Inventory

Rewards

Payment

AuthPOS

Product

Customer

E-Commerce

THE NORDSTROM SELLING ENVIRONMENT


CHALLENGES IN END TO ENDPERFORMANCE TESTING

Limited

environment

availability

Approximations

compound

Issues block

all testing

Federated data

Cost, time to build

& maintain

All code

required


SERVICE VIRTUALIZATION AND TEST DATA MANAGEMENT SOLVE THESE CHALLENGES

Services, APIs, Queues enable the approach

Performance distribution models increase accuracy

Data easily modeled on workload


Customer unavailable/issues => 20%-40% (1-2 days a week)

Recreating customers – 2+ days a sprint

Customer

user

API’s

API’s

Monthly Changes - eComm

CloudTest

User

Shop

Secure

USE CASE 1: E-COMMERCE – CUSTOMER


Customer

user

API’s

API’s

Cloud Test User

Shop

Secure

API’s

Model the customer

responses need

Work in Recreating Customers

reduced

SV performance model improves

accuracy

Increased lab availability by

80+ Days a year

APPROACH: CREATE A VIRTUAL CUSTOMER


Yearly changes to Early Access

2-3 Sprints (10-15 weeks) data and configuration preparation

Customer

user

CloudTest

User

Shop

Secure

Rewards

Default

Qualification

Alternate Qualification

USE CASE 2: E-COMMERCE – REWARDS


APPROACH: CREATE A VIRTUAL REWARDS

user

CloudTest User

Shop

Secure

% Based distribution of Qualification Paths

~3 months of prep time

to ~3 days to update model each year

Rewards

Customer


USE CASE 3: E-COMMERCE LOGIN

User DB refresh every sprint

1 day to recreate accounts – critical path

user

CloudTest User

Shop

Secure

Rewards

Customer


APPROACH: TDM FOR USER

TDM scrubs and inserts test data into prod copyNo “manual” creation of user dataUser Data setup completely out of the critical path

user

CloudTest User

Shop

Secure

TDM Prod

User

CustomerRewards


USE CASE 4 – ECOMMERCE- APPLY AND BUY

Vendor unable to support performance testing

Anniversary critical feature would have missed deadline

Proactively created Service Virtualization saved us

Nordstrom

e-Commerce

3rd-Party

Credit


SV AND TDM IN E-COMMERCE

Improved Performance Accuracy

Decreased Cycle Time

CloudTest User E-Commerce

Decreased Effort

Enables Resiliency Testing

user

Customer

3rd Party Credit

Rewards

Payment

Order Mgmt

Availability


IMPLEMENTATION CONSIDERATIONS

Start with biggest pain points

External Services and Queues

Data Models based on Workload Distributions

SV & TDM models will need

maintenance


SV AND TDM ARE CRITICAL FOR THE FUTURE

e-Commerce

Availability

Order Mgmt

Inventory

Rewards

Payment

AuthPOS

Product

Customer

E-Commerce


Q & A


Recommended Sessions

SESSION # TITLE DATE/TIME

DO3X116SLloyds Banking Group Leverages Service Virtualization to

Deliver Quality Technology Services11/19/2015 at 3:00 pm

DO3X114SFreeing the World from Slow: How Service Virtualization

and the Concept of S.P.E.E.D. 11/19/2015 at 3:45 pm

DO3T22S DevOps for a 10-Year-Old (Codebase) 11/19/2015 at 4:45 pm


Must-See Demos

SV Innovations

DevOps

Theater 3

DevOps Sim

DevOps

Theater 3

SV on Demand

DevOps

Theater 3

SV Use Cases

DevOps

Theater 3


Follow Conversations At…

Smart Bar

DevOps

Theater 3 & 4

Tech Talks

DevOps

Theater 3 & 4


For More Information

To learn more, please visit:

http://cainc.to/Nv2VOe

CA World ’15

http://cainc.to/Nv2VOe

http://bit.ly/1wbjjqX