Upload
alex-yates
View
103
Download
0
Embed Size (px)
Citation preview
@_AlexYates_
#SQLRelay
State-Based or Migrations-Based Database Development?
A Review of the Pros and Cons…Alex Yates
@_AlexYates_
#SQLRelay
@_AlexYates_ | #SQLRelay
@_AlexYates_
#SQLRelay
AgileScrumLeanIterativeContinuous Delivery
Developers
@_AlexYates_
#SQLRelay
MonitoringDeployment
IntegrityPerformance
DBAs
@_AlexYates_
#SQLRelay
Woah! Deployment fail?
@_AlexYates_
#SQLRelay
#worksOnMyMachine
Woah! Deployment fail?
@_AlexYates_
#SQLRelay
A cursor?! You just CAN’T do that?!
@_AlexYates_
#SQLRelay
#worksOnMyMachine
A cursor?! You just CAN’T do that?!
@_AlexYates_
#SQLRelay
Hey, you just dropped my hot-fix!
@_AlexYates_
#SQLRelay
#worksOnMyMachine
Hey, you just dropped my hot-fix!
@_AlexYates_
#SQLRelay
Farm Credit Services of America (FCSA)
• 100 person IT team, 14 sub-teams• Database version control inconsistent
• Deployment process manual• Delivery was slow and
unreliable
@_AlexYates_
#SQLRelay
Farm Credit Services of America (FCSA)
• Deployments easier to review
• Delivery faster and more reliable
• Standardised source control and delivery process
• Automated manual deployment tasks
@_AlexYates_
#SQLRelay
The automated deployment pipeline
@_AlexYates_
#SQLRelay
Databases are hard
• Schema changes vs existing data• Reference data vs production data• Teamwork and testing• Database drift (change outside
process, e.g. production hot-fixes)
@_AlexYates_
#SQLRelay
There’s more than one way to skin a cat
@_AlexYates_
#SQLRelay
There’s more than one way to skin automatea cat
@_AlexYates_
#SQLRelay
V1 V2
@_AlexYates_
#SQLRelay
V1 V2
Migrations-based solutions
@_AlexYates_
#SQLRelay
V1 V2
State-based solutions
@_AlexYates_
#SQLRelay
@_AlexYates_
#SQLRelay
“There's nothing more reliable than keeping track of exactly the scripts you intend to run, and running them, without trying to compare state and guess.”
@_AlexYates_
#SQLRelay
“There's nothing more reliable than keeping track of exactly the scripts you intend to run, and running them, without trying to compare state and guess.”
Paul Stovell, Octopus Deploy
http://docs.octopusdeploy.com/display/OD/SQL+Server+databases
@_AlexYates_
#SQLRelay
“As soon as you have multiple changes on a single aspect of an object, ordering and the ability to detect which change needs to be made gets very complicated.”
@_AlexYates_
#SQLRelay
“As soon as you have multiple changes on a single aspect of an object, ordering and the ability to detect which change needs to be made gets very complicated.”
Gert Drapers, built DataDude
https://blogs.msdn.microsoft.com/gertd/2009/06/05/declarative-database-development/
@_AlexYates_
#SQLRelay
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
@_AlexYates_
#SQLRelay
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
@_AlexYates_
#SQLRelay
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
@_AlexYates_
#SQLRelay
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
@_AlexYates_
#SQLRelay
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
@_AlexYates_
#SQLRelay
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
@_AlexYates_
#SQLRelay
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
@_AlexYates_
#SQLRelay
11
2 Create view: kittenTrainersNrLdn
Selects only: kittenTrainers.FullName
Edit table: kittenTrainersAdd column: AcceptsTigerCubs BIT
Both work!Same changes, versioned and deployed two ways
@_AlexYates_
#SQLRelay
Demo
@_AlexYates_
#SQLRelay
11
2 Create view: kittenTrainersNrLdn
Selects only: kittenTrainers.FullName
Edit table: kittenTrainersAdd column: AcceptsTigerCubs BIT
Both work!Same changes, versioned and deployed two ways
@_AlexYates_
#SQLRelay
11Edit table: kittenTrainersSplit column trainerFullName
into trainerFirstName and trainerLastName
But both suck.A tale of two work items…
2
Edit view: kittenTrainersNrLdnAdd column: AcceptsTigerCubs
@_AlexYates_
#SQLRelay
@_AlexYates_
#SQLRelay
0124_edit_view.sqlALTER VIEW kittenTrainersNrLdnAS SELECT fullName, acceptsTigerCubs FROM kittenTrainers
@_AlexYates_
#SQLRelay
0124_edit_view.sql 0125_edit_table.sqlALTER VIEW kittenTrainersNrLdnAS SELECT fullName, acceptsTigerCubs FROM kittenTrainers
ALTER TABLE kittenTrainers
*FUN WITH STRINGS* END
ALTER VIEW kittenTrainersNrLdnAS SELECT firstName, lastName FROM kittenTrainers
@_AlexYates_
#SQLRelay
ALTER TABLE kittenTrainers
*FUN WITH STRINGS* END
ALTER VIEW kittenTrainersNrLdnAS SELECT firstName, lastName FROM kittenTrainers
0124_edit_view.sql 0125_edit_table.sql
!
ALTER VIEW kittenTrainersNrLdnAS SELECT fullName, acceptsTigerCubs FROM kittenTrainers
@_AlexYates_
#SQLRelay
Conflicts easily missedChanges overwritten, hard to spot
Order mattersLast script wins
The “winding path” problem
How do you fix a bug that has been deployed to some environments but not others?
The problem with migrations
@_AlexYates_
#SQLRelay
V123kittenTrainersNrLdn.sqlCREATE VIEW kittenTrai…AS SELECT fullName FROM kittenTrainers
@_AlexYates_
#SQLRelay
V124kittenTrainersNrLdn.sqlCREATE VIEW kittenTrai…AS SELECT fullName, acceptsTigerCubs FROM kittenTrainers
V123kittenTrainersNrLdn.sqlCREATE VIEW kittenTrai…AS SELECT fullName FROM kittenTrainers
@_AlexYates_
#SQLRelay
V125kittenTrainersNrLdn.sqlCREATE VIEW kittenTrai…AS SELECT firstName, lastName FROM kittenTrainers
V124kittenTrainersNrLdn.sqlCREATE VIEW kittenTrai…AS SELECT fullName, acceptsTigerCubs FROM kittenTrainers
V123kittenTrainersNrLdn.sqlCREATE VIEW kittenTrai…AS SELECT fullName FROM kittenTrainers
@_AlexYates_
#SQLRelay
V125kittenTrainersNrLdn.sqlCREATE VIEW kittenTrai…AS SELECT firstName, lastName FROM kittenTrainers
V124kittenTrainersNrLdn.sqlCREATE VIEW kittenTrai…AS SELECT fullName, acceptsTigerCubs FROM kittenTrainers
V123kittenTrainersNrLdn.sqlCREATE VIEW kittenTrai…AS SELECT fullName FROM kittenTrainers
!
@_AlexYates_
#SQLRelay
@_AlexYates_
#SQLRelay
REVISION 123 REVISION 124kittenTrainers.sqlCREATE TABLE kittenTrainers(fullName,
acceptsTigerCubs)kittenTrainersNrLdn.sqlCREATE VIEW kittenTrainersN…AS SELECT fullName
FROM kittenTrainers
kittenTrainers.sqlCREATE TABLE kittenTrainers(firstName, lastName, acceptsTigetCubs)
kittenTrainersNrLdn.sqlCREATE VIEW kittenTrainersN…AS SELECT firstName, lastName, acceptsTigerCubs FROM kittenTrainers
@_AlexYates_
#SQLRelay
Diff script:
DROP COLUMN fullName
ALTER TABLE kittenTrainers ADD FirstName NVARCHAR(50), LastName NVARCHAR(50)
ALTER VIEW kittenTrainersNrLdn AS …
@_AlexYates_
#SQLRelay
Diff script:
DROP COLUMN fullName
ALTER TABLE kittenTrainers ADD FirstName NVARCHAR(50), LastName NVARCHAR(50)
ALTER VIEW kittenTrainersNrLdn AS …
!
@_AlexYates_
#SQLRelay
Script:
DROP COLUMN fullName
ALTER TABLE kittenTrai… ADD FirstName, LastName
ALTER VIEW kittenTrain… AS …
Need to understand your tool
It should be obvious to you that your tool won’t work
What is the Plan B / override?
Because one day you’ll need it
Test for data loss Automatically (naturally)
The problem with state
@_AlexYates_
#SQLRelay
So what is better?
State MigrationsVS
@_AlexYates_
#SQLRelay
State Easier (less control) Better for
sprocs/functions Better for
large/distributed teams Better for frequent
changes Better for dependency
nightmares Drift: rolled back
Migrations More control
(harder/needs discipline) Better for data
migrations Better for small teams Better for infrequent
changes Better for simple data
stores Drift: ignored
VS
@_AlexYates_
#SQLRelay
State Easier (less control) Better for
sprocs/functions Better for
large/distributed teams Better for frequent
changes Better for dependency
nightmares Drift: rolled back Better for development
Migrations More control
(harder/needs discipline) Better for data
migrations Better for small teams Better for infrequent
changes Better for simple data
stores Drift: ignored Better for automation
VS
@_AlexYates_
#SQLRelay
Pick appropriate tooling
State
DevArt SchemaCompare
DB Maestro
Migrations
DbUp
Flyway
Hybrid
@_AlexYates_
#SQLRelay
Let’s talk about:
https://msdn.microsoft.com/en-us/library/mt204009.aspx
@_AlexYates_
#SQLRelay
How does SSDT work?
Most scripts do not require manual scripting
Manual scripting causes problems
@_AlexYates_
#SQLRelay
How does SQL Source Control work?
Most scripts do not require manual scripting
Manual scripting causes problems
Perhaps just these ones?So only write these ones
@_AlexYates_
#SQLRelay
How does SSDT work?https://msdn.microsoft.com/en-US/library/hh272704(v=vs.103).aspx
https://msdn.microsoft.com/en-us/library/jj889461(v=vs.103).aspx
<< RefactorLog
Pre/Post-Deploy scripts >>
@_AlexYates_
#SQLRelay
Let’s talk about:
http://www.red-gate.com/sql-source-control/
@_AlexYates_
#SQLRelay
How does SQL Source Control work?
(This diagram again)We still only need a few upgrade scripts
@_AlexYates_
#SQLRelay
How does SQL Source Control work?
@_AlexYates_
#SQLRelay
How does SQL Source Control work?
https://documentation.red-gate.com/display/SOC5/How+migration+scripts+work c
@_AlexYates_
#SQLRelay
Let’s talk about:
http://www.red-gate.com/readyroll/
@_AlexYates_
#SQLRelay
State Easier (less control) Better for
sprocs/functions Better for
large/distributed teams Better for frequent
changes Better for dependency
nightmares Drift: rolled back
Migrations More control
(harder/needs discipline) Better for data
migrations Better for small teams Better for infrequent
changes Better for simple data
stores Drift: ignored
Better for sprocs/functions Better data migrations
How does ReadyRoll work?
@_AlexYates_
#SQLRelay
How does ReadyRoll work?Programmable objects
(Views, stored procedures etc)V125myproc.sqlEXEC sp_rename 'table_foo', 'table_bar‘
Migrations(Tables, reference data etc)
@_AlexYates_
#SQLRelay
How does ReadyRoll work?Programmable objects
(Views, stored procedures etc)V125myproc.sqlEXEC sp_rename 'table_foo', 'table_bar‘
Migrations(Tables, reference data etc)
@_AlexYates_
#SQLRelay
How does ReadyRoll work?Programmable objects
(Views, stored procedures etc)V125myproc.sqlEXEC sp_rename 'table_foo', 'table_bar‘
Migrations(Tables, reference data etc)
https://documentation.red-gate.com/display/RR1/Programmable+Objects
@_AlexYates_
#SQLRelay
But haven’t we been doing hybrid for years?
@_AlexYates_
#SQLRelay
ProdTestDev
@_AlexYates_
#SQLRelay
ProdTestDev
@_AlexYates_
#SQLRelay
ProdTestDev S0
@_AlexYates_
#SQLRelay
ProdTestDev S0
S0
@_AlexYates_
#SQLRelay
ProdTestDev S0
S0
State Migrations
@_AlexYates_
#SQLRelay
ProdTest
DevS0
“Late migrations”
S1
ProdTest
Dev S0
“Early migrations”
@_AlexYates_
#SQLRelay
Early vs Late
• Test deploy script early
• Easier to automate
• Developers own deployments (typically)
• Optimised deployment scripts (no winding path)
• Easier to intervene
• DBAs own deployments (typically)
@_AlexYates_
#SQLRelay
Drift
@_AlexYates_
#SQLRelay
@_AlexYates_
#SQLRelay
“When making changes directly on production, you are making a decision that the delay due to poor cycle time is more expensive than the risk of making a mistake.”
@_AlexYates_
#SQLRelay
“When making changes directly on production, you are making a decision that the delay due to poor cycle time is more expensive than the risk of making a mistake.”
Alex Yates, Built this slide
@_AlexYates_
#SQLRelay
The bad stuff:
• Accidental roll-backs (state)
• Failed deployments (migrations)
• Environment inconsistency (migrations)
@_AlexYates_
#SQLRelay
DriftStrategies to help
Improve cycle time Strict security policies Monitor drift (play with DDL
triggers) Redgate DLM Dashboard
(free)https://www.simple-talk.com/sql/database-administration/database-deployment-the-bits-database-version-drift/
@_AlexYates_
#SQLRelay
DriftStrategies to help
Improve cycle time Strict security policies Monitor drift (play with DDL
triggers) Redgate DLM Dashboard
(free)http://www.red-gate.com/products/dlm/dlm-dashboard/
@_AlexYates_
#SQLRelay
Demo
@_AlexYates_
#SQLRelay
Farm Credit Services of America (FCSA)
• Standardised DLM processes• FCSA delivering much more efficiently
• The FCSA model being rolled out at FCMA (sister org)
• The team all have excellent CV’s
@_AlexYates_
#SQLRelay
@_AlexYates_
#SQLRelay
Siloed sparrows suck at DLM
@_AlexYates_
#SQLRelay
Heated hippos are closed minded
@_AlexYates_
#SQLRelay
State Easier (less control) Better for
sprocs/functions Better for
large/distributed teams Better for frequent
changes Better for dependency
nightmares Drift: rolled back Better for development
Migrations More control
(harder/needs discipline) Better for data
migrations Better for small teams Better for infrequent
changes Better for simple data
stores Drift: ignored Better for automation
VS
Clever people consider options
@_AlexYates_
#SQLRelay
State
DevArt SchemaCompare
DB Maestro
Migrations
DbUp
Flyway
Hybrid
Hybrid tools are awesomeTools are pretty good
@_AlexYates_
#SQLRelay
ProdTest
DevS0
“Late migrations”
S1
ProdTest
Dev S0
“Early migrations”
It’s a balance, not a choice
@_AlexYates_
#SQLRelay
Hunt his smug face - make pain on it
@_AlexYates_
#SQLRelay
This stuff makes a big difference
http://www.codeaperture.io/2016/09/13/how-redgate-helped-define-our-process/
@_AlexYates_
#SQLRelay
Questions?
@_AlexYates_
#SQLRelay
PREMIER SPONSOR
GOLD SPONSORS
SILVER SPONSORS
BRONZE SPONSORS
SUPPORTERS
@_AlexYates_
#SQLRelay
Please give us your feedback:
sqlrelay.co.uk/feedbackThank you
@_AlexYates_
#SQLRelay
Image sourcesAuthor Source Information
Chiltepinster Wikimedia Commons Mocking Bird Argument.jpg – Wikimedia Commons. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license. Source on Wikimedia Commons: “Own work”
Bit Boy Flickr The elephant in the room – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
Nils Rinaldi Flickr Hippo fight 2/3 – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
My own collection Taken by/property of Alex Yates Kitten, “There’s more than one way to skin a cat!”
Memegenerator.net Memegenerator.net I don’t always edit database. Content designed to be shared and delivered with credit to memegenerator.net.
Ctrl.Alt.Design ctrla.lt Social Media share icons