Mastering Complex Application Deployments

1

Mastering the Complex Application Deployment

Eric Minick, Lead Consultant

2

Your Deployments might be hard if….

• You have long deployment• The business complains about long outage

windows• Numerous planning meetings• You spend all weekend at the office• Deployments are seen as risky• We celebrate when we are successful

3

The Goal

Production releases should be a non-event

4

From “Hard” to a “Non-Event”

• Why deployments are so complex• Where basic automation helps• Coordinating deployments• Increasing consistency

5

From “Hard” to a “Non-Event”

• Why deployments are so complex• Where basic automation helps• Coordinating deployments• Increasing consistency

6

Modern systems have many pieces

• Complex Architecture: Tiers, SOA, busses…• Inter-related systems mean changes have wide

impact.

Image from ischool.tv

7

Componentized applications are staying

• Agile encourages smaller teams

• Conway’s Law tells us that smaller teams, cooperating on a big system will produce more components. – And to deploy, we need all their expertise

• 84% of Global 2000 using SOA by end of 2010 (Forrester)

8

Lots of steps and people

• Manual releases require lots of…– Steps to do the work– People to execute the steps– People on call to fix it when things break

• Deployment document example from a Fortune 100 customer– ~15 components– 148 People– 600 lines in Excel

• We’ve seen the 10,000 step deployment plan

9

Our releases are inconsistent

• Many components in the system, but they don’t change all at once– Most changes involve more than one, but less

than all pieces

• We write a new plan for each release

10

From “Hard” to a “Non-Event”

• Why deployments are so complex• Where basic automation helps• Coordinating deployments• Increasing consistency

11

Automation is great…

• A basic script can turn 10 steps into 1 or 2 steps.

• Distributed workflow engines can easily model a component deployment

• With environment modeling, you can consistently deploy through environments.

12

Automation works with people

• People are good at creative work and problem solving

• People are bad at “consistency”.

• Good plan: – People design automation and troubleshoot– Computer consistently runs a process.

• See our webcast “Death to Manual Deployments”

13

…but we still have a coordination problem

• Distributed automation or runbook can deploy components or even systems

• But what exactly do I deploy? What goes together?

14

From “Hard” to a “Non-Event”

• Why deployments are so complex• Where basic automation helps• Coordinating deployments• Increasing consistency

15

How do we know what goes together?

• We want:– Components with changes ready to release– No components with changes not ready– All components needed by the components we

need (deployment dependencies)

16

How do we know what goes together?

• “All components needed by the components we need” is a tough requirement

• Traditionally satisfied by asking Dev and trusting their omniscience• Works most of the time

– Or we trust the ALM (Source + Bug Tracking) tools• Could work if people were perfectly disciplined• People are not

17

How do we know what goes together?

• Test it!–ALM tools and developer oral histories

suggest what to test together–Once verified, the stuff in Test is the stuff

we want in Production

18

Model based Deployment Strategy

• Model the system in a test environment

• Promote the model across environments

• Model includes:– applications, app configuration, databases,

infrastructure, settings– Aware of environment specifics as well

19

Requirements for Model Promotion

• Inventory – what is where• Recording inventory as a Model• Testing that we can trust

20

Model Promotion: Inventory

• To promote what is in Test to Production, we must know what is in test

• To know the extent of the change, we must know what is in production

• Inventory of all environments• What versions of each component make up the system• Where appropriate, treat configuration as a component

• Bonus: Inventory helps with visibility and audit

21

Model Promotion: Model Snapshots

• Need to be able to snapshot the inventory to define a desired state

• Approaches:– Spreadsheets listing component versions– Change management tickets with components as

attachments– Build it into your deployment product (we did)

22

Model Promotion: Testing we can trust

• Testing validates compatibility

• Must leave adequate time to test

• With more automated tests, we can:– Test quicker– Test more creatively

• Automated tests should run in ALL environments

23

From “Hard” to a “Non-Event”

• Why deployments are so complex• Where basic automation helps• Coordinating deployments• Increasing consistency

24

Each deployment is different

• Only a subset of components change in a release

• Sometimes we change configuration

25

Have a single deployment process

• Define the process for releasing the entire system

• Note which steps are associated with what components

• Execute only the steps for the components that are changing (which your inventory tells you)

26

Relentlessly remove variation

• Developers and architects contribute to repeatable releases

• “Special” events (like adding a new data stream) must fit into any automation or defined process

27

Use the same process in Test and Live

• The earlier the process is used, the better– It will be tested more– Earlier environments have less tolerance for

slowness

• The process or automation must take into account environmental differences

28

Key Takeaways

1. Releases are complex, and the business needs them done rapidly

29

Key Takeaways

1. Releases are complex, and the business needs them done rapidly

2. Traditional methods for determining what is released together should be used in Test

30

Key Takeaways

1. Releases are complex, and the business needs them done rapidly

2. Traditional methods for determining what is released together should be used in Test

3. Only components tested together should be released together

31

Key Takeaways

1. Releases are complex, and the business needs them done rapidly

2. Traditional methods for determining what is released together should be used in Test

3. Only components tested together should be released together

4. Simple deployment automation is very helpful, but coordination is key

32

Key Takeaways

1. Releases are complex, and the business needs them done rapidly

2. Traditional methods for determining what is released together should be used in Test

3. Only components tested together should be released together

4. Simple deployment automation is very helpful, but coordination is key

5. Getting this right is easier if you have help from developers, testers, operations and release management

33

References

http://urbancode.com/resources• Death to Manual Deployments!• Build & Deployment Automation for the Lean

Economy• ITIL Release Management and Automation

Urbancode.com/blogs/Twitter.com/UrbanCodeSoftFacebook.com/UrbanCodeSoft

34

Yes, UrbanCode sells products for this

• AnthillPro is now the DevOps Platform

• DevOps Platform– Automated build, test and deployment. Includes

UrbanDeploy.

• UrbanDeploy– Application Release Automation

35

Questions?