Best Practices for Database Continuous Delivery

DBmaestro and Perforce Joint-Webinar

October 21st, 2015

▪ You will be on mute for the duration of the event

▪ We are now talking so please type a message in the Questions box in the Control Panel if you can’t hear us (please check your speakers and GoToWebinar audio settings first)

▪ There will be a Q+A session at the end, you can start submitting you questions on the Q&A bar on your gotowebinar dashboard.

▪ A recording of the full webinar will be put up online

Before We Begin

Presenters

Mark WarrenProduct Marketing Director, Perforce

Mark has over 20 years experience as a user and developer of tools to improve software delivery processes.

Presenters

Yaniv YehudaCo-Founder & CTO at DBmaestro

Spent the last years raising awareness about the challenges around database development and deployment, and how to support database Continuous Delivery.

Outline

▪ The Adoption of Continuous Delivery▪ The Reasons why the Database is Often Left Behind▪ Single Source of Truth for your Database▪ Best Practices for Including the Database in CD▪ The Bigger Picture – Automation and Stopping the

Line▪ Q&A

Industry Perspective

Waterfall• Annual releases • Mostly manual

Agile• Release more

than once a year

• Some automation

Continuous• Weekly/daily

updates• Massive

automation

“The days when a successful organization could release software once every 12 to 18 months are over.”

“Continuous Delivery is Reshaping the Future of ALM”, Kurt Bittner, Forrester, July 2013

Continuous Delivery Is Moving Ahead!

Recently Conducted DBmaestro Survey• Over 350 participants

Why Continuous Delivery?

What is CD? Product Delivery Pipeline

Requirements Develop Build Test Integrate Deploy

BuildRequirements Develop IntegrateTest Deploy

Rqmts Doc Licenses/IP History

SocialCoding

BuildFarms

Database Scripts “Hardware”(Virtual)

Code OpenSource

Binaries Release Binaries

Code OpenSource Binaries Release BinariesDatabase

Product Delivery Pipeline

BuildRequirements Develop IntegrateTest Deploy

Rqmts Doc Licenses/IP History

SocialCoding

BuildFarms

Scripts “Hardware”(Virtual)

Code OpenSource

Team Collaboration (design, dev, release, devops…)

Binaries Release BinariesDatabase

Accelerate the Pipeline (code, DB, scripts, binaries, etc.)

Product Delivery Pipeline

Less Efficient Product Delivery

Poor visibility between teams introduce friction and design errors

Poor component reuse results in higher production cost

More delays, less efficient product delivery

Increased risk of quality issues

DevOps

A Single Source of Truth

Continuous Delivery Best Practice

Keep everything in version control… this includes requirement documents, test scripts, automated test cases, network configuration scripts, deployment scripts, database creation…

- Continuous Delivery by Jez Humble and David Farley

““

Modern Version Management

Hybrid Workflows• Distributed & Centralized Version control,

code reviews, simple file sharing• Happy developers & contributors

Every File• Efficiently handles large, often binary, data

DevOps Stay Happy & Productive• A mainline source for all builds even with

distributed development

All IP Safe & Secure • Granular permissions, theft risk monitoring

Perforce Helix

CONTRIBUTORS

CONSUMERS

Perforce Manages IP for Market Leaders

13,000 20,000 users

9,500 users500+ terabytes

5,000+ userscoders & designers

10,000,000 Perforce xact/day

Everything! 11,000+ users

Chips Games & Animation Cloud/SW Electronics Systems Automotive

And > 15,000 other companies across industries

Facts– Fortune 1000 (IDC)

▪Average total cost of unplanned application downtime per year is $1.25b - $2.5b

Hourly cost of an infrastructure failure is $100K/hour to $500K – $1 M

Average number of deployments/month is expected to double in two years

DevOps-led projects will accelerate the delivery of capabilities to the customer by 15%–20%

Continuous Delivery is required!

But…what about the

database?

Continuous Delivery is big and getting bigger

Dzone recent study…

* Dzone guide to continuous delivery, 2015 edition

What about the database?

▪ Based on the survey questions, only 13% from the ones reported doing CD for DB, are actually performing basic CD practices!!!

The rest are plugging the automated process withvarious manual steps…

▪ Why?▪ What is so special about the database?

Not 49%, actually DOWN to 13% !!!

Mostly manual implementation

So? What is the problemwith manual implementation?

Manual steps in an automated process don’t work!!

1. At first, you think “This is easy. We can handle it!”2. Then “I think we’re fighting a losing battle…”3. And then you have to double your speed !!!

Also, manual steps lead to Chaos in Emergency Times

90%Rate this as a risk factor, but

53%Break the process and test urgent hot-fixes in pre-

production

Old adage but true– The database is often neglected and therefore

can become the weakest link– Manual processes

Database/Code Silos exist…– Don’t always communicate effectively– Need to follow same procedures & best

practices

Essential from a compliance and business point of view

Should be the strongest link

Is database the Weakest link in a chain ???

The Database is a constraint

What is the problem?

▪Root Causes for issues:− Challenging manual source control process− Static deployments code overrides− Dynamic deployments tools unaware of version control− No release automation red-flags – don’t know when to stop the

line…

Version Control Process (file based)

Database

Check-Out Script

Modify Script

Get updated Script from DB

Check-In Script

Compile

Scriptin DB

Debug

Scriptin DB

?

??

?

A

A’

Traditional isolated processes

Risk of falling out of sync

90%Rate this as a risk factor, yet

72%Admit database may not be in sync with the source

repository

Scripts & version control▪ Challenges:

− Code-overrides− Working on the wrong revisions − Scripts do not always find their way to the version control solution− Out of process updates go unnoticed− Hard to locate outdated update scripts

▪Playing safe? What we really need: − The upgrade script− The actual code of the object− A roll-back script

Modern VC integrated process

X1.11.1.11.11.21.31.41.51.61.7

Scripts… build once deploy many

Int QA Stage Prod

Database Deploy Script

Environment

Re-Base (due to defects)

DevDev

DevModel

1.1 1.2

1.2 1.3

1.3 1.4

1.4 1.5

1.5 1.6

1.6 1.7

1.11.11.41.7

1.1 1.2

1.2 1.3

1.3 1.4

1.4 1.5

1.5 1.6

1.6 1.7

1.1 1.2

1.2 1.3

1.3 1.4

1.4 1.5

1.5 1.6

1.6 1.7

Out of Process Change

XX

XXX

? 1.1.1

X

a

a

60%Of those manually building scripts have to fix

or tweak them regularly as part of a deployment process

80%Rate this as a risk factor, yet

71%Follow manual processes to create their database

deploy scripts

Build & deploy on demand

1.11.21.31.41.51.61.7

*

Int QA Stage Prod

Database Deploy Script

Environment* Execute the same script being

executed at the Stage environmentRe-Base (due to defects)

DevDev

DevModel

1.1 1.2

1.2 1.3

1.3 1.4

1.4 1.5

1.5 1.6

1.6 1.7

1.1 1.4

1.4 1.7

1.1.1 1.7

1.1 1.1 1.11.41.7

File Based Version Control

Out of Process Change

1.1.11.7 1.1.11.7

Validate

Safe to automate?(?)

Compare & sync tools

Challenges…A compare & sync tool:▪ Is unaware of any changes that occurred before the time it ran▪ Has no knowledge of changes that took place at the target environment

▪ Does not leverage version control for more information▪ Unable to deal with conflicts & merges between different teams

▪ Requires manual inspection ▪ Requires detailed knowledge regarding each change as part of the process

Mistrust AGAIN… So…no automation…

We fear for automating problems into production and a major risk!!!

70%of those using compare & sync tools have to

review and fix the results as they can't always trust them to automatically deploy correctly

￭ Understand the nature of the changes

￭ Raise red flags on conflicts ￭ Support out-of-process changes￭ Utilize baseline aware analysis

Safety Net Deployment Automation

Deploying changes if neededDevelopment Baseline

Previous Label /Production Golden Copy

Production

If we had the index in the baseline => we should take it down from production…

(Deploy Change)

Or protecting target environment…Development Baseline

Previous Label /Production Golden Copy

Production

BUT… If no index in baseline => we should protect the NEW index on production!!!

(Protect Target)

Dealing with conflicts => merging changes

Conflict Resolving – Meta Data/Content

Impact Analysis

Impact Analysis! not Damage Control…

Raise red flags to stop the line… before actual deployment…

if requires human intervention

Automate “everything”– Package the deployment of database changes along with all

your other application components to give a unified picture Move the process upstream

– Easily promote the same package (including database changes!) from one environment to the next, handling environment-specific differences automatically

Create the deployment pipeline

For successful CD:

What does DBmaestro offer?▪ Database Enforced Change Management solution

– Database source control– Enforce best practices– Plugs into Perforce– Database merge & change impact analysis

▪ DevOps & CD Solution for databases– Baseline aware deployment automation, rollback & recovery– Reduce database deployment issues– Plugs into release management & Continuous Delivery

▪ Security– Control roles & responsibilities when introducing DB changes

▪ Audit & Compliance– Know who can do what, where, when & why

Allows you to package, verify, deploy and promote database changes just as you would do

with application code… putting you in a position

to build a full delivery pipeline…

▪ Focusing on changes rather than managing changes and dealing with re-work, boosted overall productivity of 250 developers. We estimate we were able to do 15% more with the same resources.

▪We went from several fix-centric deployments a day, to one feature-centric deployment a week.▪The amount of incidents in production has declined as well. We had 20% less incidents. ▪CIO @ Credit Card company

“Testimonials - Efficiency

“

About DBmaestro

▪The leading provider of DevOps for Database▪Database Development and Continuous Delivery

Thank you!

Q & AMark Warren Yaniv Yehudamwarren@perforce.com yanivy@dbmaestro.com www.perforce.com www.dbmaestro.com