Upload
devops-israel
View
2.109
Download
0
Embed Size (px)
DESCRIPTION
HP Software has an established product line for on-premise and a successful hosted solution, but in the last year we deployed new products in a true SaaS fashion. The SaaS transformation was both a challenge and an opportunity – to overcome traditional hurdles and adopt DevOps and Continuous Delivery practices. We will share our experience so far: moving from CI to CD, leveraging functional and performance tests, automating deployment to multiple environments and more. Presented at DevOps Con Israel 2013 in the Continuous Delivery track.
Citation preview
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
DevOps Summit AGM Continuous Delivery Adam Spektor/ January 28, 2013
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Structure
Continuous Delivery
Jenkins OO Nexu
s
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3
8 Principles of Continuous Delivery
Principles
• The process for releasing/deploying software
MUST be repeatable and reliable
• Automate everything!
• If something's difficult or painful, do it more often
• Keep everything in source control
• Done means “released”
• Build quality in!
• Everybody has responsibility for the release
process
Martin
Fowler
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Continuous integration
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5
CI infrastructure structure
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6
Continuous Integration
Dev L
AB
Yehud
Jenkin
s
Farm
SaaS
Data
Cente
r
Je
nkin
s
Farm
1
2
How ?
What ?
• UT
• Rest system tests
• UI selenium tests
• Upgrade tests
• Performance tests (single user experience)
• Static code analysis/Coverage – Sonar (FindBugs, PMD,
JaCoCo)
• Maven site
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 7
Continuous Integration
Performance test : WHY ?
•Enable performance testing as part of CI
•Utilize Integration & UI testing as Performance tests
•Track performance trending and KPI in addition to functional testing
•Mark build as unstable in case of performance degradation above
threshold
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8
Continuous Integration
Performance test : test perspective
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9
Continuous Integration
Performance test : transaction perspective
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10
Jenkins
Master
1 - Developer checkin
5 - Split tests to chunks based previous execution time
1
2
2 - Jenkins trigger
4 - Build & copy into the Vagrant VM’s shared directory
3 - Execute chef OS modifications & configurations
0 - Prerequisites
1. Pool with vms prepared (include initial Sahara
state) each vm use Jenkins warm plug-in (auto-
discover slave)
2. Each pool have assigned to predefined OS item
Virtual box
Jenkins slaves pool
Jenkins
Physical
Slave 3
4
5
6
6 - Clean vm with Sahara & swarm
installation
1. OS changes like code changes should be tested in CI
system
2. OS changes should be trackable
3. Independent development – no IT involved
4. Quick and updated development environment
Continuous Integration – near future
Developer Lab
Why
?
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Nexus artifact management
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 12
Nexus infrastructure
• LB – active/passive Linux heart beat
clustering model
• Guarantee high availability to read users
– Nexus proxy duplication
– Smart proxy – guarantee that
requested artifacts are available
• Isolation level – only Jenkins can upload
artifacts to Nexus (only after tests
verification)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Continuous deployment
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 14
Continuous Delivery flow
Circular deployment Staging Productio
n
Automatic/By request
By request :
Approved by QA
By request :
Approved by QA &
SaaS
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15
Continuous deployment
CI Master mydtbld0004
1
CI Farm
1 - Time Scheduled/Manual Build
Nexus
master
2
2 - Upload to QA
repository DevLab
IL
SaaS Lab
(Single)
3
Nexus Proxy
3 - Start deploy QA/DEV
QA
5
4
4 - Sync proxy & Upload to RAS
5 - Deploy
Automatic nightly deployment to QA/DEV
environment
OO Server Dev
3 3
3
OO Lab
Server
5
RAS
4
4
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18
Continuous deployment 1 -Request deploy version
2 - Download to RAS
SaaS Lab
(Single)
3 - Deploy
Manually deployment to Security/PCoE
Nexus Proxy
1
PCoE
Security
engineer
OO Lab
Server
RAS
3
PCoE
engineer
Security
1
2
2 3
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 19
Continuous deployment 1 - Request promote to staging
2 - Build
DevLab
IL
SaaS Lab
(Single)
Nexus Proxy
OO Lab
Server
3 - Upload to Staging repository
4
4 - Sync proxy
5 - Request to Deploy
Promote & Deploy to Staging
SaaS Production
(Multi)
1
Staging
QA
operator
SaaS operator
RAS
CI Master
CI Farm
Nexus
master
3
2
RAS 4
6 - Download to RAS & Deploy
5
5
6 6
4
OO Production
Server
2
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20
Continuous deployment 1 - Request promote to production
Nexus
master
2 - Nexus promote to production
DevLab
IL
SaaS Lab
(Single)
2
Nexus Proxy
OO Production
Server
2
Promote to Production
SaaS Production
(Multi)
1
SaaS
Operator
RAS
2
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21
Continuous deployment 1 - Request to deploy
2 - Upload to appropriate
geography RAS
Sidney Austin
3 -Deploy
Deploy to Production
London
SaaS operator
OO Production
server
RAS
Nexus Proxy
RAS RAS
1
2
2
SaaS operator SaaS operator
1 1
2 2
3 3 3 3
Internal
External
External External
2
2
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 22
Deployment Demo
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you