View
72
Download
1
Category
Preview:
DESCRIPTION
See how RTTS' performance/load testing team utilized Microsoft's load testing tool in Visual Studio to test a high availability web site and its web services while reducing the total cost of ownership (TCO) for the customer.
Citation preview
Industry
Retail
Solution & Services Performance/Load Testing
Software Test Management
Test tool evaluation
Tools
Microsoft Visual Studio Ultimate version
CASE STUDY
Leading Retailer Chooses
Microsoft Visual Studio for Load Testing Critical Web Services
Background
RTTS’ client operated numerous heterogeneous customer-
facing (i.e. revenue producing) web applications, as well as
administrative and customer support applications, that relied
on a common set of services, including customer account
retrieval, delivery date availability, and address validation.
Using IBM’s Integration Bus solution (formerly known as
WebSphere Message Broker), RTTS’ client was launching an
enterprise server bus (ESB), so that these individual
applications (and future applications) could more easily scale
in terms of business growth.
Challenges
High availability and responsiveness were critical factors to the customer’s business model, since
these factors directly affected order conversion for the customer-facing applications, as well as mean
time to resolution regarding customer service inquiries. RTTS’s customer required a load testing
solution that possessed a minimal total cost of ownership (TCO) that could scale in proportion to its
business growth and the consumption of the web services provided by the ESB.
Licensing expenditures and load generation infrastructure costs were paramount in being able to
meet budget constraints, while the ability to scale to the transaction rates and concurrency were
imperative to the application and infrastructure teams. The short-term load testing goal was to be
able to support the demands of at least one (1) of the customer’s primary brand’s e-retail web site
using a single instance of the IBM Integration Bus. Peak usage had been observed to be
approximately 7,000 online customers.
Strategy
RTTS put forth a proposal that would assess the available load testing solutions. Proof-of-concepts
were arranged that addressed the gamut of open source load testing solutions and commercial load
testing solutions. The results of the proof-of-concepts were presented to the stakeholders in
conjunction with the projected costs of the solution in order to determine the best return on
investment. RTTS also took ownership of the rollout of the load testing solution by providing
professional services that installed the testing products and engineered the baseline performance
tests.
Solution
Several load testing solutions were evaluated using the criteria set forth by RTTS’ customer. Open
source, commercial, and cloud-based load testing solutions were evaluated via proof-of-concepts
against a subset of the ESB’s suite of web services. Load test solutions that were provided by open
source testing tools were deemed a risk in terms of having product support. Cloud-based testing
solutions turned out not to be a fit a the time due to concerns regarding security, proprietary scripting
languages, and the client’s current corporate culture. However, RTTS’ client did express interest in
having the flexibility to move to a cloud-based solution, especially regarding its load generation
infrastructure. In the end Microsoft Visual Studio Ultimate was selected to be the best fit based upon
its capabilities, familiarity, and cost.
The initial cost of Microsoft Visual Studio Ultimate was significantly less expensive when compared
to some of the more established load testing solutions in the market. Microsoft’s licensing model
permitted unlimited numbers of virtual users, so that testing could easily scale along with the growth
of the customer’s business. Along with the ability to systematically test the ESB using a familiar
Visual Studio IDE, the ability to add capacity to its load generation infrastructure was seamless.
RTTS was able to incrementally add to the customer’s load generation infrastructure by simply
spinning up additional virtual machines and adding them to the current Visual Studio Load Test Rig.
If and when the need arises, these tests could be ported to the Windows Azure cloud platform for
greater flexibility for implementing its load generation infrastructure.
Benefits
RTTS successfully installed and configured the necessary Microsoft Visual Studio components and
was able to jumpstart the load testing initiative by testing a subset of the ESB web services related
to product availability, delivery dates, and shipping options for one (1) of the customer’s family of
brands. Using a Visual Studio Load Test Rig of six (6) agents, load tests reaching 12,000 concurrent
users were executed. Microsoft Visual Studio’s load testing capabilities were able to assist in
discovering bottlenecks related to the threading model being used by the ESB’s database
connection pool, as well as within the delivered HTTP configuration of the ESB’s web interface.
Once the environment was optimized, 90% of all web service requests were returned in under one
(1) second. The IBM Integration Bus was shown to support the processing of 55 web service
requests per second with its CPU utilization averaging about 60%.
At the conclusion o the project RTTS was tasked with leading the testing effort by augmenting the
collection of web services being tested, incorporating additional retail brands and back-office
applications, and subsequently increasing the end-user consumption of the web services.
RTTS is a Microsoft Gold Partner. Since 2006, RTTS has been an education & consulting partner with
Microsoft – implementing solutions and providing educational services around Microsoft's Visual Studio
testing tools. RTTS has worked with the Visual Studio team to develop top-notch solutions around their
excellent toolset. For more information about our solutions and services, go to www.rttsweb.com
copyright Real-Time Technology Solutions, Inc. 2014
Recommended