Embed Size (px)
Citation preview
Delivering changes for
applications and databases
“The beginning of a beautiful friendship”
Eduardo Piairo
Miguel Alho
Delivering changes for applications and databases
@EdPiairo
https://pt.linkedin.com/in/jesuspiairo
http://www.eduardopiairo.com/
Eduardo Piairo, DBA & DevOps @ Celfinet
Delivering changes for applications and databases
@MytyMyky
https://pt.linkedin.com/in/miguelalho-982331
http://www.miguelalho.pt/
Miguel Alho, Software Engineer @ Faro Europe
@DevOpsPorto
Is Coming
Delivering changes for applications and databases
Source Control
Continuous Integration
Continuous Delivery
Database
+Application
What’s so special about the database?
What’s so special about the database?
DLM – Database Lifecycle Management
Data management and data migration
Data monitoring
Data recovery
http://assets.red-gate.com/products/dlm/DLM_tools_and_processes_diagram.pdf
Typical problems we want to fix
• Databases are out of pace with application development
• Lack of traceability of database changes (changes history)
• Manual databases processes prevent the CI and CD utilization
• Lack of testability
• Database setup time for new environments
• Bugs in production
Typical problems we want to fix
Databases becoming a
bottleneck in an agile
delivery process
Typical problems we want to fix
Databases becoming an
easy target to blame
Why database automation?
• Enable control over database development
• Increase speed of response to change
• Keep a versioned “history” of database states and changes
• Reliability of the release process
• Increase release frequency though repeatability of processes
• Reduce time spent fixing bugs - automated tests
• Remove/reduce human intervention in the release process
Why database automation?
Fearless database changes
1st step: Source Control
• First step in your database deployment pipeline
• Traceability through change history
• SQL as documentation
• Shared code-base and shared process
• Enforceable standards to reduce conflicts
1st Step: Source Control
Fundamental resource:
SQL Script
Migrations-based approach vs State-based approach
• State based solutions
• Script represents the current database state
• Your source of truth is how the database should be
• Migrations based solutions
• Script represents a migration
• Migration represents how to transition to the next database version
• Your source of truth is how the database should change
Flyway
• Flyway – “Database Migrations made Easy”
http://flywaydb.org/
• Open source database migration tool
• Flyway command-line enabled
Simplicity: easy to setup, no need to install
Zero dependencies (java + jdbc)
Scripts are written in SQL
Flyway – How does it work?
• Flyway commands:
• migrate, clean, info, validate, baseline, repair
• Metadata table:
• Used to track the state of the database
• If the database is empty, flyway won’t find it and will create it
• Flyway scans the migrations directory and check migrations against the metadata table
• Migrations are sorted based on their version number and applied in order
Version Control System
Two components:
Database scripts repository
Flyway – database migrations
Rule 1: Script version (timestamp)
Rule 2: Operation type
Rule 3: Object type
Rule 4: Object name Example: V20160728.1100__Create_TB_MyTable.sql
Demo
How about applications?
It takes two to tango
Database and Application change alignment
● Database schemas should evolve along with application evolution
○ deploy app => deploy changes / migrations
● (Integration) Test app and db changes together
○ should pass tests while developing,
○ should pass tests while promoting to new environments
● Ideally, one application per database
● Pipeline deployment only!
Integration test at the database layer
● Use TDD techniques
● Limit database integration tests
to data layer
○ faster tests!
○ validate sql and mappings
● Always test with the latest
schema
DatabaseApplication
DatabaseApplication
Demo
https://github.com/MiguelAlho/Purchases-DbMigration-sample
What we just saw
http://assets.red-gate.com/products/dlm/DLM_tools_and_processes_diagram.pdf
Database and Application change alignment
Fearless database changes
Challenges
• Challenge #1: managing multiple branches
• Manage merge conflicts
• The default branch is the main/master
branch and all the merges for this branch
must be done with caution
Challenges
• Challenge #2: Script expiration date
• Should the migration script have an
expiration date?
• Sometimes, in the next day, the migration
script no longer makes sense
Challenges
• Challenge #3: migration scripts stack
• How can I manage the dependencies
between different databases?
Challenges
• Challenge #4: developer sandbox
• How to supply a disposable development
environment where the database is
included?
What’s Next?
Links
• Demo application “Purchases-DbMigration-sample”
• https://github.com/MiguelAlho/Purchases-DbMigration-sample
• Flyway
• https://flywaydb.org/
• DbUp
• http://dbup.github.io/
• Octopus Deploy
• https://octopus.com/
• http://docs.octopusdeploy.com/display/OD/Deploying+ASP.NET+Core+Web+Applications
• Visual Studio
• https://www.visualstudio.com/
Links
• Redgate
• http://www.red-gate.com/
• http://www.red-gate.com/products/sql-development/sql-source-control/
• http://www.red-gate.com/products/sql-development/sql-compare/
• http://www.red-gate.com/products/sql-development/sql-data-compare/
• http://www.red-gate.com/products/dlm/dlm-automation/
• http://www.red-gate.com/products/dlm/dlm-dashboard
• http://www.red-gate.com/solutions/dlm/
• Simple Talk - DLM Patterns and Practices
• https://www.simple-talk.com/collections/database-lifecycle-management-patterns-practices-library/
Reading material
Delivering changes for applications and databases
@EdPiairo
https://pt.linkedin.com/in/jesuspiairo
http://www.eduardopiairo.com/
@MytyMyky
https://pt.linkedin.com/in/miguelalho-982331
http://www.miguelalho.pt/
