Embed Size (px)
Citation preview
DEPLOYMENT PIPELINE FOR DATABASES Eduardo Piairo
@EdPiairo
#WinOpsAzure SQL Database, SQL Server
ABOUT MEDeployment pipeline for databases
@EdPiairo, #WinOps
@EdPiairo
https://pt.linkedin.com/in/jesuspiairo
http://www.eduardopiairo.com/
Eduardo Piairo
Friend of Redgate
Operations Engineer
DevOps Porto Founder
INDEX
1. Deployment pipeline
2. Database challenges
3. Deployment pipeline stages
4. Deployment pipeline scenarios
5. Final thoughts
Deployment pipeline for databases
@EdPiairo, #WinOps
PIPELINE DEFINITION
In software engineering, a pipeline consists of a chain of processing elements (processes, threads, coroutines, functions, etc.), arranged so that the output of each element is the input of the next.
by Wikipedia
@ https://en.wikipedia.org/wiki/Pipeline_(software)
Deployment pipeline for databases
@EdPiairo, #WinOps
DEPLOYMENT PIPELINE DEFINITION
A deployment pipeline should enable collaboration between the various groups involved in delivering software and provide everyone visibility about the flow of changes in the system, together with a thorough audit trail.
by Martin Fowler
@ https://martinfowler.com/bliki/DeploymentPipeline.html
Deployment pipeline for databases
@EdPiairo, #WinOps
DEPLOYMENT PIPELINE STAGESDeployment pipeline for databases
Stage 1 – Change description (Source Control)
Stage 2 – Change validation (Continuous Integration)
Stage 3 – Change implementation (Continuous Delivery)
Source
ControlContinuous
Integration
Continuous
Delivery
@EdPiairo, #WinOps
DATABASE & DEPLOYMENT PIPELINEDeployment pipeline for databases
Source
ControlContinuous
Integration
Continuous
Delivery
Database
+Application
@EdPiairo, #WinOps
WHAT’S SO SPECIAL ABOUT DATABASES?Deployment pipeline for databases
@EdPiairo, #WinOps
DATABASE CHALLENGESDeployment pipeline for databases
• Database and application changes (most of the time) are handled differently
• Different deployment pipeline
• Synchronization needed
• Database is not included (manual work)
@EdPiairo, #WinOps
DATABASE CHALLENGESDeployment pipeline for databases
• Manual work …
• Lack of traceability of database changes (changes history)
• It’s the most expensive and risky work type
• Prevent CI and CD utilization in their full extent
• Promote the fear of changes
@EdPiairo, #WinOps
DATABASE CHALLENGESDeployment pipeline for databases
Databases become a bottleneck in an agile delivery process
@EdPiairo, #WinOps
AUTOMATIONDeployment pipeline for databases
@EdPiairo, #WinOps
THE VALUE OF AUTOMATIONDeployment pipeline for databases
• Enable control over database development
• Increase speed of response to change
• Greater reliability of the release process
• Remove/reduce human intervention in the release processFrom
(Re)Learn and forget it
to
Improve and forget it
@EdPiairo, #WinOps
THE VALUE OF AUTOMATIONDeployment pipeline for databases
Fearless database changes
@EdPiairo, #WinOps
SOURCE CONTROLDeployment pipeline for databases
@EdPiairo, #WinOps
SQL Script
Fundamental resource
SOURCE CONTROLDeployment pipeline for databases
• First step in your database deployment pipeline
• Traceability through change history
• SQL as documentation
• Shared code-base and shared process
• Enforceable standards to reduce conflicts
@EdPiairo, #WinOps
• State based solutions
• How the database should be
• Migrations based solutions
• How the database should change
MIGRATIONS VS STATEDeployment pipeline for databases
SQL Source Control
@EdPiairo, #WinOps
Migration Migration
State
Delta
SCRIPTING GUIDELINESDeployment pipeline for databases
• One script, one operation type, one object (small batches)
• Merge conflicts management
• Patterns identification
• File system scripts history search
@EdPiairo, #WinOps
CONTINOUS INTEGRATIONDeployment pipeline for databases
• Integrate and validate changes
• Tests: Unit, Integration
• Small batches, less risk
• Considerations:
• What should be tested? And When?
• Can the application help on this?
@EdPiairo, #WinOps
CONTINUOUS DELIVERYDeployment pipeline for databases
• Delivering change(s) in the target environment
• Considerations
• Downtime
• Time to recover
• Small batches – low risk
• Affected applications
@EdPiairo, #WinOps
FAILED DEPLOYMENTDeployment pipeline for databases
• Rollback/roll forward scripts
• Very fast
• Please, do not make mistakes
• It’s like working backwards
• Backups (before deploy)
• Safer but slower
• Keep databases small
• Avoid share databases between applications
@EdPiairo, #WinOps
SCENARIO #1Deployment pipeline for databases
Source Control Continuous Integration Continuous Delivery
APP
DB
@EdPiairo, #WinOps
• Independent deployment pipelines
• High need of synchronization between DBA and development team
SCENARIO #2Deployment pipeline for databases
Source Control Continuous Integration Continuous Delivery
APP
DB
@EdPiairo, #WinOps
• Different code repositories
• Independent CI process
• The contact point will be the deployment moment
SCENARIO #3Deployment pipeline for databases
Source Control Continuous Integration Continuous Delivery
APP
DB
@EdPiairo, #WinOps
• Different code repositories
• Connected/dependent CI process
SCENARIO #4Deployment pipeline for databases
Source Control Continuous Integration Continuous Delivery
APPDB
@EdPiairo, #WinOps
• Share the some code repository, CI process and CD process
• No need of synchronization
• Promotes learning within teams (database code reviews)
DEMODeployment pipeline for databases
@EdPiairo, #WinOps
MY DEPLOYMENT PIPELINE DEFINITION
Pipeline: Cultural and technical tool for managing changes in the software development process (should contemplate databases, applications and infrastructure).
by me ;)
Show me your pipeline and I will tell you the way you work!
Deployment pipeline for databases
@EdPiairo, #WinOps
DEVOPS AND DATABASESDeployment pipeline for databases
• DevOps way
• Increase flow visibility
• Increase feedback
• Increase knowledge base
@EdPiairo, #WinOps
READING MATERIALDeployment pipeline for databases
@EdPiairo, #WinOps
Q&A
Deployment pipeline for databases
@EdPiairo, #WinOps
@EdPiairo
https://pt.linkedin.com/in/jesuspiairo
http://www.eduardopiairo.com/
