Upload
codefest
View
350
Download
4
Embed Size (px)
Citation preview
Deliver Fast, On Time & With High Quality
Description
Karim Fanadka QA Manager HPE Software
About me: Karim Fanadka
Job: QA Manager at HPE Software, 7+
Product: StormRunner Load
Favorite activates: Traveling, Ski, Blogging
Ski – France 2013
Road Trip – Iceland 2015
We are going to talk about:
AgileModern QA
SaaS
Automation
Environments
Hotfixes Continuous Deployment
SaaS – Software as a Service
Before SaaS – On Premise
SoftwareUsers
SaaS – Software as a Service
Data Base
Servers
File System
Firewall
Users SaaS
journal note
I have been asked to build the first DevTest team that combines:
• Modern technologies and techniques
• Agile Methodologies
• SaaS product testing
• Deliver fast
StormRunner Load - SaaS product
Which product my team is testing :
Performance testing solution that makes it easy to plan, run, and scale web and mobile testing over the cloud.
10
1 Create and run a load test using your favorite scripting tool
2 Load Generators spin up in AWS EC2 or MS Azure
3 Results and app insights are available in real time
The product my team is testing: StormRunner Load
New
Soon
Performance testing solution that makes it easy to plan, run, and scale web and mobile testing over the cloud.
StormRunner Load as a Service
Data Base
Servers
File System
Firewall
Users SaaS Cloud
Agile – Do It Fast
Before Agile - Delivery Timeline
1 or 2 Major releases to production in 1 year
Agile StormRunner Load Delivery Timeline
March April AugustJune Nov Dec
Whats the CodeName of the next release?If you know the answer, I will invite you to a
AlleyCat BubbleBobble Elevator Action Frogger Golden Axe Hang-On
FebruaryDec
CommanderKeen Digger
6 Major releases to production in 1 yearYear: 2015
Agile – The Challenges
• Test new features
• Test Regression
• Automation
• Environments
• Continues Integration / Continues Delivery
• Hotfixes
Do it in 1.5 months
• Data Migration (When update SaaS with the newer version)
• Performance Testing
• Security Testing
• Globalization
• Documentation
• And more …
journal note
As the first DevTest engineer at HPE I have been asked to build the first DevTest Team to handle:
• SaaS product testing
• Deliver fast
Modern QA Team (DevTest)
What is DevTest?Using top end QA/DEV practices combined with DevOps techniques
QA & DEV Working very close, on the same IDE, code base, and in united process
Automation is a MUST
Manual is for new features / exception flows that are difficult to automate
NO MORE “Deliveries for QA” YES for “Push to Production”
Training“we are testing, and breaking code constantly but we help rebuild it better…….”
Continues Testing
Push new code
Release branch (1.5 months)
Test new codeShort Regression
Push to production
Automation
Developer Engineer Test Engineer DevTest Team
Builds in 1 Release (1.5 month): 1,400• Full: 236• Commit: 1,164
Continuous Testing
Push new code
Automation – Full package
Builds in 1 Release (1.5 month): 1,400
Automation – Commit package
Feature branch
Release branch
Full: 236
Commit: 1,164
Developer Engineer
Modern QA
DevTest
DevOpsSaaS
Dev
Product
Customer
Automation
journal note
I was aware that automation must find real defects and not just false alarm, DevTest team has a new challenge, and it’s to build:
• Reliable frameworks
• Fast results (not more than 7 min)
Frameworks:
Frameworks Tests
Client Testing Framework
API Testing Framework
Cloud Framework
Results
Example for Automation challenge17 Regions in the Cloud 6 different images for each region
15 different metrics
102 Tests
(~1500 metrics)
Automation – Cloud Framework - Report
Grunt
Protractor
TestResults
WebDriver
Angular
SeleniumAPI’s
UIControls(Storm)
Test
DBLayer ProtractorWrapper
JasmineStyle JS
AutomationLibrary
Executer
Client automation framework
Grunt
MochaTestFrameworkTests
DBWrapper
APIWrapper
AutomationLibrary
StromRunnerAPIs
Repo
rts
TestsResults
Backend automation framework
API discovery tool
Cost cloud tool
030
80 100 120150 170
200
0 0
75120
147
220
390
600
0 0 0 0
100
200
270 270
0
100
200
300
400
500
600
700
1.1 - AlleyCat 1.2 - Bubble 1.3 - CK 1.4 - Digger 1.5 - Elevator 1.6 - Frogger 1.7 - Golden Axe
1.8 - Hang-On
Automation Active Tests - Trend
Client (UI) Backend Cloud
85% Coverage
100% Coverage
60% Coverage
Next Generation: Application Lifecycle Management
Test Environments
Test Environments Local
Lab – Simulate SaaS Env
• Each tester have his own local machine for fast testing
• Lab Env = 7 Servers• We have 6 Lab Env:
• QA1• QA2• Security• Performance• Integration
Staging
• SaaS responsible for staging:• Maintenance • Deployment• Sanity
Test Environments Farm
QA Security Performance IntegrationLocal (Linux)
Staging
Continuous Deployment
Continuous Deployment
Push new code
Automation – Full package
Builds in 1 Release (1.5 month): 1,400
Automation – Commit package
Feature branch
Release branch
Full: 236
Commit: 1,164
Developer Engineer
Continuous Deployment
Test Engineer
Developer Engineer
Hey Tester, How do you manage to handle this amount of builds and deployments on daily basis
I’m a Modern QA
Continuous DeliveryLab
Nexus
Tel Aviv
PIT
Jenkins
QA2
Security
PCoE
QA2 (Portal)
PIT
PIT
PIT
Hotfixes in 1 day
Hotfixes Lifecycle
07:00 AMProduction bug
08:00 AMResearch by QA and Dev
10:00 AMFixing bug
16:00 AMTesting (Staging)
14:00 AMTesting (Local)
18:00 Push to production
Articles • 10 best practices for QA teams to deliver quality software, fast
• Blurring the lines: How the development tester role changes on agile teams
• 3 steps to building test databases for the real world
Tip
“Changing our mindset takes precedence over technology.
When we work together, we will succeed.”
Thank you
@kim0z85
Question?
https://www.facebook.com/karim.f.as
https://il.linkedin.com/in/[email protected]
Karim Fanadka QA Manager HPE Software