Session 5:

Federal Secure Cloud Testing as a Service - TaaS Center of Excellence (CoE)Robert L. Linton

Agenda

• HP ALM Solution Review• HP Cloud Potential• Cloud Portal• HP ALM Solutions in a virtual environment

– Testing Center of Excellence ( CoE )• Cloud based• Provisioning HP Solutions• TaaS• Subscription Model• Benefits

• Lessons Learned • Summary• Q & A

HP Application Lifecycle Management 11.5

Requirements Management

DevelopmentManagement

QualityManagement

Requirements ManagementRequirements Management

Requirements Definition

Requirements Definition

Agile Project ManagementAgile Project Management

Defect Management

Defect Management

FunctionalFunctional PerformancePerformance SecuritySecurity

Foundation

TraceabilityTraceability ReportingReportingProcess Agnostic(i.e. Waterfall, Agile)

Process Agnostic(i.e. Waterfall, Agile)

Extensibility (Open APIs)

AGILE integrations

Extensibility (Open APIs)

AGILE integrations

Application Lifecycle Management 11.5

Project Planning & Tracking

Project Planning & Tracking

Enterprise Release ManagementEnterprise Release Management

Secure Development

Secure Development

Application Lifecycle IntelligenceApplication Lifecycle Intelligence

Service VirtualizationService Virtualization

Requirements Management

DevelopmentManagement

QualityManagement

Requirements Management

Requirements Definition

Agile Project Management

Defect Management

Functional Performance Security

Foundation

Traceability ReportingProcess Agnostic(i.e. Waterfall, Agile)

Extensibility (Open APIs)

AGILE integrations

Application Lifecycle Management 11.5

Project Planning & Tracking

Enterprise Release Management

Secure Development

Application Lifecycle Intelligence

Service VirtualizationNew

Updated

Fortify

HP BTO Software CoverageBusiness outcomesBusiness outcomes

Universal CMDBUniversal CMDB

Service Management 

Center

Service Management 

CenterData Center Automation 

Center

Data Center Automation 

CenterNetwork 

Management Center

Network Management 

Center

Operations Center

Operations Center

Business Availability Center

Business Availability Center Client 

Automation Center

Client Automation 

Center

Operations OrchestrationOperations Orchestration

OPERATIONSSTRATEGY APPLICATIONS

SOACenterSOA

Center Application Security CenterApplication 

Security Center

PerformanceCenter

PerformanceCenter

QualityCenterQualityCenter

Application Lifecycle

Project & PortfolioManagement 

Center

Project & PortfolioManagement 

Center

Service Portfolio Management

CIO Office

CTO Office

OPERATIONS

SAP, Oracle, SOA, J2EE, .Net

Business ServiceManagement

CorTechs

IT Service Management

Business Service Automation

CorTechs’ Expertise with HP ALM Solutions• HP / Mercury Partner 17+ years

– HP PPM– HP BAC, SiteScope, Diagnostics– HP Testing Tools

• Automated – QC / ALM– UFT ( QTP and Service Test )– Performance Center

» LoadRunner– Fortify – Security Testing

• Manual » HP Sprinter

• Open Source Testing – Can be added to TaaS– Agile supported

Mainframe

Client/Server(including packaged Apps such as SAP, Siebel, Oracle…)

Composite(Custom Applications,

SOA, Cloud)

…    1980   …   1985    …   1990    …   1995    …   2000   …   2005   …   2010    …   2015

ConstraintsComplexity

Costs

Changes in software development

Traditional Testing Tools and 

Methodologies

RMI ObjectsMessaging

Service

BI Tools

Business Rules

FileSystem

ESB

WebInterface

ContentDatabase

DataWarehouse

FinancialsMainframe

Web/WAPInterface

EJB

VirtualService

RoutingService

SOAP

CRM App

Products

CollaborationApp Portal

HelpEngine

BPMSLegacyInfrastructure

Internal

External CloudPartners

# of Interconnected Components # of Inter‐dependent Teams

# of Heterogeneous Technologies

Increase Rate of Change

Change and Complexity Increasing:Composite Application Environments

• IaaS - Infrastructure as a Service • PaaS - Platform as a Service• EaaS - Email as a Service • DaaS - Desktop as a service• SaaS – Software as a Service• RaaS – Recovery as a Service • TaaS - Testing as a Service

– Test Data management– Functional – Performance– Services – Security– Monitoring

“ X “ as a Service

9

Using traditional methods, government agencies cannot precisely represent real-world operating conditions in their application development and testing environments.

This can introduce risks like: Technical problems Servers Networks

Security vulnerabilities Schedule delays Budget overruns

Challenge for Government Agencies: - development and testing environments

Government IT Projects

Budget Schedules Staffing

What is a Center of Excellence ( CoE )?A logical or physical “service bureau” providing expertise across projects in a

“shared services model” to drive standardization of quality products and processes across the Enterprise

Non-CoE Approach

Team B

Team ETeam C

CoE Approach

Applications

Expertise

InfrastructureTeam D

Team A

Team BTeam A

Team C Team D

Cloud Portal ‐Workflow

HP ALM Solutions implemented in the Cloud

On‐Prem

Test 1‐n

Hybrid Public /Private (EC2, etc…)

VLM 1 (AppLogic) VLM 2 (AutoSuite) VLM 3 (VMware)

Test 1‐n

Test 1‐n

Test 1‐n

Test 1‐n

Test 1‐n

Cloud TaaS configuration isn’t a one-time event.…Are all the consumers of Dev and Test cloud expected to know everything?

Volatile Utilization of Lab ResourcesVariable demand in allocating Dev and Test computing capacity 

Variable consumption over time

Dev Team A, B, C…

Scalability of the Cloud

ELASTICITY of Cloud

Lab Capacity & Cost

Cloud based TaaS –Test when you need to

ProductionInfrastructure

1) Pool Lab Resources Together(Leverage the existing capacity - CoE)

2) Implement virtual lab provisioning2a) Catalog the VM Images

3) Bring additional teams into the cloud to use centralized resources

Catalog Central Repository

Functional Testing

Performance Testing

Application Under Test

Service Virtualization

Testing Tool CLINsxxxx1AA‐xxx0 Quality Center Environmentxxxx1AA‐xxx1 Quality Center User Licensesxxxx1AA‐xxx2 Functional Testingxxxx1AA‐xxx3 Test Data Management (1‐500GB Database)xxxx1AA‐xxx4 Test Data Management (500GB+ Database)xxxx1AA‐xxx5 Performance Center ‐ Controllerxxxx1AA‐xxx6 PC ‐ Standard Protocols (each) ‐ 1,000 Virtual User Testxxxx1AA‐xxx7 PC ‐ Standard Protocols (each) ‐ 5,000 Virtual User Testxxxx1AA‐xxx8 PC ‐ Standard Protocols (each) ‐ 10,000 Virtual User Testxxxx1AA‐xxx9 PC ‐ Standard Protocols (each) ‐ 25,000 Virtual User Testxxxx1AA‐xx10 PC ‐ Enhanced Protocols (each) ‐ 1,000 Virtual User Testxxxx1AA‐xx11 PC ‐ Enhanced Protocols (each) ‐ 5,000 Virtual User Testxxxx1AA‐xx12 PC ‐ Enhanced Protocols (each) ‐ 10,000 Virtual User Testxxxx1AA‐xx13 PC ‐ Enhanced Protocols (each) ‐ 25,000 Virtual User Testxxxx1AA‐yy00 On‐Demand Support ‐ Grade 1xxxx1AA‐yy01 On‐Demand Support ‐ Grade 2xxxx1AA‐yy02 On‐Demand Support ‐ Grade 3xxxx1AA‐yy03 On‐Demand Support ‐ Grade 4xxxx1AA‐yy04 On‐Demand Support ‐ Grade 5xxxx1AA‐yy05 On‐Demand Support ‐ Grade 6

Real World Government Entity’s dilemma• Performance Testing

– Owned the software – still in the BOX• Authorized a STUDY to calculate - plan and costs• Came back with 6 months to one year• Costs would be close to $ 100K before it started• RFPs – Award• Assign tasks to Gov. and contractors

– Then they could start Performance Testing • Possibly, if the environment is ready…

“ Testing as a Service “ - TaaSWhat one Government Agency had to decide -

CPU/MIPSCosts

100.0 Units

10,000 Regressions

Business Case for Massively Parallel Testing

10 ServersIn Cloud

1000 ServersIn Cloud

1 hour

10,000 Regressions

Results DataComplete

CPU/MIPSCosts

100.0 Units

Test Suite 1: Limit to 10 Available Test Servers

Test Suite 2: Provision Enough Servers to Finish Job in 1 hour

100 hours

Development and Test as a Service

Service Description

•Fully functioning Dev‐Test environment for use across  multiple platforms:• Open source, .NET,  Agile and JAVA

•Development, test and prep/prod environments are aligned and problems are reduced because of differences between the development and production environments including security common controls

•Application lifecycle management; testing and resourcing available (PM’s, developers, engineers, etc.)• Rapid boarding of user community 

Service Benefit• Fast scale‐up or scale‐down of resources with usage‐based billings, old servers can be decommissioned  when project is finished

•High machine utilization, fewer physical servers less capital expense•Time to market is reduced, services can be provisioned in 24 hours• 99.9% uptime Service Level Agreement in the contract• Operational and maintenance costs included in the service

20

Compare - Traditional Costs vs. TaaS

Typical‐ separate Costs and Effort

• TOTAL COST to Test • Tool Admin $• Hardware $• OS , DB  License $• Lab Space $• O&M $• C&A $• Outages $

TaaS‐ all included in ONE price

• TOTAL COST for ‐ TaaS• Tool Admin ‐ included• Hardware ‐ included• OS , DB ‐ included• Test Lab Space ‐ included• O&M ‐ included• C&A ‐ included• SLAs‐ included

• Chargeback and Metering– Chargebacks help customers control their cost

• Meter the use of resources consumed during test cycle• Assign rates to metered resources • Issue usage and charge reports to customers • Pay as you go Model

– Metering • Virtual Machines

– Workstations– Servers– Databases– Storage

– Services• Training• Consulting• Mentoring• Turn Key Delivery

Test as a Service Key Features

• End-to-end Test Orchestration • Automated test lab provisioning

– Application under test – accessible– Test Tools provisioned– Test Scenarios ( test plans, test data, test scripts, test results, defects )

• Central Repository for secure test library – Reuse– ALM / QC

• Functional Testing– QTP– Service Test

• Performance Testing – Performance Center– LoadRunner

• Security Testing– Fortify

Test as a Service Key Features

Cloud TaaS Benefits = more ROI

• Immediate Benefits:– “Ready to Use” Implementation for HP ALM Solutions

• HP ALM• HP Performance Center• HP UFT • HP Fortify

– Test More, Test Earlier, Utilize gov resources effectively– Elasticity and Scalability when you need it !

• Reoccurring Benefits:– Eliminate ongoing maintenance, administration, upgrades

• Same Test Resources get more testing done sooner

– Reduce cost – Hardware, Software, Resources, Setup Time

– Constant Improvement of Software deliverables• Higher Quality Business Applications • Better Continuous Development• Increased Stability of Applications in production

Value proposition of CLOUD – TaaS

Speed‐up release cycles• Allow parallel development and earlier functional testing. • Reduce wait time for unavailable systems.

Reduce budgets• Required to setup, manage, and run complex test environments• Lowering the cost of additional Hardware

Decrease third‐party expenses• Incurred by using 3rd party production systems

Lower the risk of project failure• Reduce risk by engaging quality teams early. • Decrease risk with broader test coverage

Improve total user experience• Achieve desired levels of quality and performance levels

QUALITY

COST

There has never been as much Change….Quality is Being Re-defined

Cloud

Agile Composite

Responsiveness

Quality

Durability

ElasticityScalability

Reduced Cycle Time

Reusability

Modularity

Modernization requires discipline

“The market is shifting as organizations seek greater business value and agility.   Agility doesn’t mean run as fast as possible, it requires great discipline”

Gartner Research, “Don’t let Short‐Term Agile Create Long‐Term Pain”

Modernization does not grant IT a “hall pass” for 

quality

Now is the time to master -QA / Test Automation

Limited Resources

•Tight Budgets•Reduced Staff• Business Analysts/Developers as Part‐Time Testers

More Headless Testing

•Cloud‐Based Apps•Mobile Apps• Embedded Systems•XML/Web‐Services…

Agile Test Execution

• Shorter Release Cycles

•Continuous Integration

• Build regression  tests – with Unit Test

Complex verification 

•Myriad of technologies

•Mobile testing•Multiple Integration points for comp’ apps

The only way to keep up is to automate“ environments, AUT, tools, virtualized services “

• Make QA more efficient by testing more– Less time on support tasks, more focus writing tests and executing them

• Leverage CLOUD / TaaS solutions to our advantage– Save time and Money $ $ $

• Use Virtualization – to “ Shift Testing “ Left ! – compresses the implementation schedule

• Testing as a Service can be local, public or private Cloud– Increases the Stability and Quality of deployed application– Better utilization of Corp. resources across the Enterprise

• Major Government Entities are executing this CoE model• More Government Agencies are embracing this approach

– “X” as a Service– Center of Excellence– Cloud based Dev and Test

Summary :A Few Words to Review

Questions and Answers

CorTechs, Inc.5900 Fort DriveCentreville, VA 20121(703) 968 – 7021rlinton@cortechs.com

© 2012 CorTechs, Inc.

For More InformationCome see us at our Table # 2 

Deliver to the 

customer 

their      

Vision

Iterative Development

30 30

Data Tier Application Tier

Business Users

System TestersUAT Testers

DevelopersDevelopment

Teams

Jenkins

Team Foundation Server

HP ALM

ʘ

ʘ

ʘ

ʘ

ʘ

ʘ

ʘ

Business Users

System TestersUAT Testers

Eclipse

STS

Visual Studio

NetBeans   

Analysis

6

Analyze Code

8Deploy

Continuous Integration

C.I.

• Reduce configuration, license, testing and training costs• Reduce costs related to defects and production downtime• Increase testing with fewer resources

Reduce cost

• Increase cross‐project visibility to quality metrics and ensure KPIs are aligned with business needs• Increase quality via continuous monitoring of testing status, progress and trends• Ensure go/ no‐go decisions are based on quantifiable business risk

Reduce risk

Optimize operational efficiency and staff utilization while reducing production risk at a lower cost

Benefits of a Center of Excellence

• Increase staff efficiency and utilization across projects• Increase operational efficiency with repeatable standardized processes• Enable globalization, best practice and asset sharing across geographically distributed teams• Establish cross‐project visibility to ensure assets can be effectively re‐used

Increase efficiency