32
Lessons Learned From Continuous Delivery Out of the Frying Pan, Into the Fire Mandi Walls Leeds DevOps January 31, 2017

Lessons Learned from Continuous Delivery

Embed Size (px)

Citation preview

Page 1: Lessons Learned from Continuous Delivery

Lessons Learned From Continuous DeliveryOut of the Frying Pan, Into the FireMandi WallsLeeds DevOpsJanuary 31, 2017

Page 2: Lessons Learned from Continuous Delivery

Mandi Walls

Technical Community Manager for EMEA@[email protected]

Page 3: Lessons Learned from Continuous Delivery

Who Is Chef?

Page 4: Lessons Learned from Continuous Delivery

Why Continuous Delivery?

• Used as a proxy for success with DevOps transformations• Blame John Allspaw and Paul Hammond

• “10+ Deploys A Day at Flickr”

Page 5: Lessons Learned from Continuous Delivery

The rewards?*

*source: Dr. Nicole Forsgren research on DevOps

More deployments

Ship faster

Faster MTTR after issues More profits, market share, and productivity

Market cap goes up

Page 6: Lessons Learned from Continuous Delivery

Technical LessonsThe Fun Stuff

Page 7: Lessons Learned from Continuous Delivery

Lesson: Automation

• Automated, full-stack application policies• Package and service installation• Versionable, testable, repeatable workflow• Scalable application policies• Management of interdependencies across nodes

Page 8: Lessons Learned from Continuous Delivery

Lesson: Dynamic Infrastructure

• Provisioning and setting up environments • Dynamic scaling of compute resources• Migrating legacy workloads to the cloud• Multi cloud and hybrid cloud deployment• Support for heterogeneous environments

Page 9: Lessons Learned from Continuous Delivery

Do you HAVE to go to the Cloud?

• Well….• You need cloud-approximate features• APIs• Resources-on-Demand

Page 10: Lessons Learned from Continuous Delivery

Infrastructure as Code

• Programmatically provision and configure components

• Treat like any other code base• Reconstruct business from code repository,

data backup, and compute resources• Partner with dynamic infrastructure• “Cattle not pets” or “Crops not house plants”

Page 11: Lessons Learned from Continuous Delivery

How Chef Got Into DevOps

• System automation is often a first step for transitioning to DevOps work practices• With or without previous experience in Agile• Agile left Operations concerns in the dark• Config management presents practical value

• Displays easy wins, demonstrable and measurable outcomes• MTTR

Page 12: Lessons Learned from Continuous Delivery

The game changer: rapid time to value

Innovation

Quality/Compliance

DynamicInfrastructure

Infrastructure as Code

Automate the Stack

DevOps

+ +

Page 13: Lessons Learned from Continuous Delivery

Trust The Automation

Page 14: Lessons Learned from Continuous Delivery

Lesson: Revision Control

• Findings from State of DevOps report• Everyone must use version control

• Dev• Ops• You• Me• The dog your hipster office mate brings to work

Page 15: Lessons Learned from Continuous Delivery

Lesson: Automated Testing

• Difficult tasks – automating the testing of applications developed internally and COTS or contract work

• Different tools for different languages and platforms• Delivers a lot of value, but costs a lot to get going• Unfortunately met a lot of orgs that struggle with this or choose to postpone

Page 16: Lessons Learned from Continuous Delivery

Testing Infra Code Toodescribe 'apache::default' do context 'When all attributes are default, on an unspecified platform' do

let(:chef_run) do runner = ChefSpec::ServerRunner.new runner.converge(described_recipe) end

it 'converges successfully' do expect { chef_run }.to_not raise_error end

it 'installs apache' do expect(chef_run).to install_package 'apache2' end endend

Page 17: Lessons Learned from Continuous Delivery

Culture LessonsThe Messy Stuff

Page 18: Lessons Learned from Continuous Delivery

Lesson: Collaboration

• Ideological shift for Operations folks used to working alone• Also difficult for specialized techs in silos – Storage Ops, NetOps, Security• Talking to people vs just putting in tickets and waiting

• The stereotypical “throw it over the wall”

Page 19: Lessons Learned from Continuous Delivery

Lesson: CAMS to CALMS

• CAMS: Culture, Automation, Metrics, Sharing (c. 2010)• CALMS: Culture, Automation, Lean, Metrics, Sharing• Using Lean to break through the logjam of enterprise processes

Page 20: Lessons Learned from Continuous Delivery

Lean

• Eliminate non-value-added action – Don’t do things that don’t add value just because you’ve always done them

• Pull over Push – Spend time on the things customers want to use. Don’t spend time thinking about ways to force them to use things they don’t want

• Kaizen – Continuously improve people, processes, tools• Kaikaku – Be willing to change the whole thing if it isn’t working• Small Batch + Experimentation – Often learned from Agile

Page 21: Lessons Learned from Continuous Delivery

Business StuffThe “Culture + Tools = ££££” Stuff

Page 22: Lessons Learned from Continuous Delivery

Failure

• Accepting failure as part of experimentation• Practice recognizing how much risk is involved in each experiment

• The cost of an individual failure

Page 23: Lessons Learned from Continuous Delivery

Blameless Culture

• Recognizing that complex systems have complex problems• The system may be too complex for one human to fully understand• Research in other systems fields

• Aerospace and aviation, nuclear power generation, etc• “Human error” is often cited but not often the actual problem

• The humans didn’t know your system didn’t have redundant error handling• The humans didn’t know there wasn’t alerting on a subsystem they weren’t told about

Page 24: Lessons Learned from Continuous Delivery

Lesson: Business Value

• Fits in with Lean• Work on the things that will add value to users, even if they aren’t the most fun

• “Eating your vegetables”• Minimize NIH in favor of reuse

Page 25: Lessons Learned from Continuous Delivery

Managing Risk

• Small batches to manage the near-term costs• Validate with customers in short timeframes• Introducing near-term volatility to mitigate long-term risk

We spent a year building this thing but no one liked it!

Page 26: Lessons Learned from Continuous Delivery

Lesson: Skills x Contractors x Employees

• DevOps is a way of life• Getting to Continuous Delivery takes TIME• Managing the skill matrix of the people working on the project is important• Harder to do when folks are leaving after 6 months to find something more

interesting

Page 27: Lessons Learned from Continuous Delivery

TrendsWhat’s up with DevOps in 2017+

Page 28: Lessons Learned from Continuous Delivery

Containers

• Of course. • Because containers.• And microservices.• And whatever comes next as the smallest compartment of compute

Page 29: Lessons Learned from Continuous Delivery

Shifting “To the Left”

• Pushing more and more activities, discussion, acknowledgement of complexity to earlier stages of the process

• Security, compliance, testing, performance

http://www.rebalanceclinic.co.nz/wp-content/uploads/2013/05/keep-left.png

Page 30: Lessons Learned from Continuous Delivery

Tools For All Teams

• Complement collaboration, communication, transparency• Integrate dynamic infrastructure, testing, reporting

Page 31: Lessons Learned from Continuous Delivery

We Are Chef

• https://chef.io• Find us! https://events.chef.io/• learn.chef.io

Page 32: Lessons Learned from Continuous Delivery