ALM and practical guidance for VSTS Database ProjectsJens K. SuessmeyerDeveloper hearted, relational mindedVisual Studio ALM Core Rangerhttp://blogs.msn.com/b/JensS
Session objectives
• Get to know the database project advantages over common database development approaches
• Get to know the relevance of Application lifecycle management for a database
• Map the stages of an ALM process to database projects• Get to know the importance of a formalized process to
develop, maintain, test and deploy databases• Get to know what is in the Visual Studio ALM Ranger
Guidance for Database projects for you
2
about:me
• Jens K. Süßmeyer, Consultant from Microsoft Services Germany
• Main focus areas• .Net Development• Database development• Consulting
• SQL Server 2008 MCM• Visual Studio ALM Core Ranger
3
Question
Did you ever had inconsistencies in
your database development that you think could have been
avoidable ?
General understanding of lifecycle Management
5What is ALM ? (http://go.microsoft.com/?linkid=9743693)
Database Lifecycle
DatabaseProject BuildDeployment
Test
Development
Service Creation (Requirements definition / Design / Implementation)
Service management (Deployment / Operation / Optimizing)
The evolution of development tools
7Tools for team development (http://go.microsoft.com/?linkid=9743692)
Version Control Challenge
CREATE TABLE dbo.Auction( id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)
CREATE TABLE dbo.Auction( id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)
class AuctionApplication( int id; void MethodA();)
class AuctionApplication( int id; void MethodA();)
DatabaseDatabase
class AuctionApplication( int id; void MethodA(); void MethodB();)
class AuctionApplication( int id; void MethodA(); void MethodB();)
class AuctionApplication( int id; string cacheTitle; void MethodA(); void MethodB();)
class AuctionApplication( int id; string cacheTitle; void MethodA(); void MethodB();)
Revision History
AppApp
V 1 V 2 V 3
ALTER TABLE dbo.Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)
ALTER TABLE dbo.Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)
ALTER TABLE dbo.Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)
ALTER TABLE dbo.Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)
Hand crafted versioning
-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
Hand crafted versioning
-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
V1 => V2 -- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
V2 => V3
V2 => V3-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
V3 => V4
-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
V1 => V3-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
V2 => V4
-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(
id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN
ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END
V1 => V4
Hand crafted versioning
• Instead of having to guess the order by names, scripts would need to be omnipotent and not version agnostics in first place.
• Produces a high effort in maintaining scripts and persisting the knowledge of the order
The enemy of a copy & paste deployment -Data
• Unlike deployments of applications / assemblies in common, the database deployment will need to preserve data
• Incremental deployment needs to provide a way to migrate data without loosing information or violate referential integrity leaving the database in an inconsistent state
• Data scripts will need to honor the dependencies, constraints, dependent objects
Deployment – Visual Studio Database Projects
CREATE TABLE dbo.Auction( id INT NOT NULL PRIMARY KEY, name VARCHAR(25) NOT NULL UNIQUE, start DATETIME NULL, len INT NULL)
CREATE TABLE dbo.Auction( id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)
CREATE TABLE dbo.Auction( id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)
CREATE TABLE dbo.Auction( id INT NOT NULL PRIMARY KEY, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)
CREATE TABLE dbo.Auction( id INT NOT NULL PRIMARY KEY, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)
IncrementalDeploymentIncrementalDeployment
LogicalDatabaseLogical
Database
Revision HistoryV 1 V 2 V 3
CREATE TABLE dbo.Auction( id INT NOT NULL PRIMARY KEY, name VARCHAR(25) NOT NULL UNIQUE, start DATETIME NULL, len INT NULL)
CREATE TABLE dbo.Auction( id INT NOT NULL PRIMARY KEY, name VARCHAR(25) NOT NULL UNIQUE, start DATETIME NULL, len INT NULL)
NewDeployment
NewDeployment ALTER TABLE dbo.Auction
WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)
The advantages of Database projects
• The from the database uncoupled schema model contains all the necessary information to create / update a database and represents the state “as-should-be”.
• The schema motion scripts are created on deployment time, NOT development time, letting the developer focus on his development, NOT deployment goals. Referential constraints etc. Are automatically honored.
• Maintaining a conceptual models instead of scripts let you do easily use all the features of the ALM platform, like merging Branching
Database development with various tools
15
Benutzer
Testing Database
TesterProductionalDatabase
SQL
Administrator
Design Database
Source Control System
SQLSQL SQLSQL
Developer
Visual Studio ALM Rangers• Mission
• Accelerate adoption of Visual Studio with out-of-band solutions for missing features or guidance
• Joint Venture• Co-funded and co-lead by Services and PG• Bijan Javidi/PG and Jeff Jurvis/Services
• Community readiness • Gather field experience• Empower the community
• Collaboration• ~130 Extended Rangers• ~46 External Rangers• 10 Core Rangers
• Find us here …• MSDN • Blog http://tinyurl.com/RangersBlog• ALM Rangers IP on Campus http://tinyurl.com/WW-VS-ALM-Rangers
• Want to join?• Email us on [email protected]
Visual Studio Database Projects guidance Rangers guidance vs. MSDN
• MSDN explains features not scenarios
• This guidance is scenario based
• Shows ideas and experiences based on real world engagements
The guidance materialization
• We provide best practices and guidance in each sub topics in a
• Guidance document
• Corresponding HOLs
Application lifecycle sub topics
Development
DatabaseProject BuildDeployment
Test
Development
Service Creation (Requirements definition / Design / Implementation)
Service management (Deployment / Operation / Optimizing)
Development
• Why 1 + 1 is sometimes 0…
Development
• Internal database validation decreases the chance of getting unsuccessful builds or inconsistent projects artifacts by modeling and validating the schema on the fly
• Developers can be provided with templates to adhere to certain guidelines and with static code analysis rules to ensure coding standards
• Developers can work without having any dependency on a database prior to deployment.
Development / Database guidance
• The database guidance document shows how to
• manage projects • streamline development• when to use / not to use certain features like different project
types (Server , Composite / Partial)• how to apply known concepts of existing guidance documents
like Branching & Merging to the database world • be aware of limitations during development and project
management and how to get around them
DEMOWorking with database projects
Build
DatabaseProject BuildDeployment
Test
Development
Service Creation (Requirements definition / Design / Implementation)
Service management (Deployment / Operation / Optimizing)
Build
• Why a successful build is sometimes not useful at all …
Build
• Like any other projects Database projects can be included in your build definition within TFS or any external build processes.
• The guidance will in addition help you to:• Continuous integration with VSDB• Building and Deploying outside Team Build• Inclusion of automated tests / test lists to be included within
the build• Provide you with the option to deploy databases
automatically with the build• Build and Deployment packaging using WIX
DEMOManaging a (W)indows (I)nstaller (X)ML Project for Database Projects
Test
DatabaseProject BuildDeployment
Test
Development
Service Creation (Requirements definition / Design / Implementation)
Service management (Deployment / Operation / Optimizing)
Test
• Why testing should be important…
Test
• The cost of lousy tests…
Test
• Creation of tests was put into a designer to enable database developers and tester to create their Unit tests without having to learn any coding language
• Unit Tests can run on sample data as well as random or real data to ensure a stable quality
• Unit Tests can be included in the automated build to ensure continuous checks of your database artifacts automate processes.
Test / Creating useful data
•Simple data generators
• Strings (char, varchar, nvarchar…)• Numbers (smallint, int, bigint, float…)• Binary (varbinary, image…)• Date and Time• GUID and Bit
•Complex generators
• Foreign Key• Regular Expression / Unique Expression Generator• Data Bound / Sequential Data Bound• FileBound Generator• XMLGenerator
You can cover the full complexity of data types here to make your code robust !
Test / Creating useful data
• Extend it to your needand extend the frameworkby simply writing your data generator. Publishing workswith copy & paste to theVisual Studio machine. Include interface that you might already use by accessing internals data sources, etc.
Test / Database guidance
• The database guidance document shows how to
• create simple tests and integrate them in the project• create a multi-step process to build / deploy / load databases
with random or live data and test against that• Integrate your tests within your automated build definitions
DEMOIntegrated creation of Unit tests
Test
• Extend it to your needand extend the frameworkby simply writing your owncondition. Publishing workswith copy & paste to theVisual Studio machine
Deployment
DatabaseProject BuildDeployment
Test
Development
Service Creation (Requirements definition / Design / Implementation)
Service management (Deployment / Operation / Optimizing)
Deployment
• Without knowing your deployment environment you might be surprised of the results…
Deployment
• The transistion to deployment should much more than just throwing the build artifacts over the fence. There is much more valueable information buried in here:
• Finding bugs and deployment issues early• Feasability of upgrade scenarios / discover major shostoppers• Durations of database upgrades• Needed resources for database upgrades (like log space, usage
of database parameters, etc.)• Building confidence about the delivered quality
Deployment
In a perfect world, most of the deployment checks can be
done on the „assumed“ database enviroment as the
source control is the source of truth.
Deployment
Database administrator doubts
• “The database upgrade will drop tables and created index and will make my database slow.”
• “I doubt that the VSDBCMD process really does all the magic behind the scenes and honors all changes I have done in the database”
• “I will not run any command line tool against the database before I haven´t seen the actual .sql code executed beforehand.”
Deployment
Database developer doubts
• “We did all the upgrade testing against the database, but who ensures that the assumed schema on the real database still exists?”
• “The database administrators created several objects in addition which we cannot guarantee to be preserved over the deployment.”
• “The duration of the upgrade process is unpredictable as the database administrator might have created several indexes which need to be preserved.”
Deployment
Finding model drifts (extract from Guidance document)
• Make sure that you tested the application database with the right version you once have been given WITHOUT having access to the database (Trusted DBA version)
• Make sure that you tested the application database with the right version you once have been given having access to the database (Trusted developer version)
• Review the deployment script prior executing them in the database (Suspicious DBA version)
DEMOFinding model drifts
Summary
• Visual Studio Database Projects can ease development , build, test and deployment process by providing the right tools.
• VSTS Database Projects cover the operative part of ALM for Database projects.
• Many formalized processes with little effort will help you to ensure quality across the database projects
• The Database Guidance document can help you to find out how you can get around limitations, implement best practices and handle special situations.
46
Stay up to date with MSDN Belux
• Register for our newsletters and stay up to date:http://www.msdn-newsletters.be• Technical updates• Event announcements and registration• Top downloads
• Follow our bloghttp://blogs.msdn.com/belux
• Join us on Facebookhttp://www.facebook.com/msdnbehttp://www.facebook.com/msdnbelux
• LinkedIn: http://linkd.in/msdnbelux/ • Twitter: @msdnbelux
Download MSDN/TechNet Desktop Gadget
http://bit.ly/msdntngadget
TechDays 2011 On-Demand
• Watch this session on-demand via Channel9http://channel9.msdn.com/belux
• Download to your favorite MP3 or video player• Get access to slides and recommended resources by the speakers
Additional Ressources
• My blog: http://blogs.msdn.com/JensS• Barclay Hill (PM of Visual Studio Data Tools):
http://blogs.msdn.com/BaHill • Visual Studio Database Projects Guidance:
http://vsdatabaseguide.codeplex.com/ • Visual Studio Rangers: http://tinyurl.com/ALMRangers
Questions ??
? ?
THANK YOU