LJC 2015 "The Crafty Consultants Guide to DevOps"

The Crafty Consultant’s Guide to …Craftily Consulting in DevOps

Daniel BryantPrincipal Consultant, OpenCredo

[email protected]@danielbryantuk

Tonight

• A tale of two parts…

• The Crafty Consultant’s take on DevOps

– Disclaimer – this is not how I actually consult!

• DevOps in real world!

– An overview

27/02/2015 @danielbryantuk

Who Am I?

• London Java Community Associate

• Adopt OpenJDK and JSR

• InfoQ Editor and DZone MVB


• Principal Consultant at OpenCredo

Agile transformations

DevOps methodologies

Microservices and Cloud

The Crafty Consulting Guide to Craftily Consulting in DevOps

Crafty Consulting Ltd.

- Not a genuine company! -- A satirical guide to DevOps -

Crafty slides hidden…

(as I’m going to reuse them!)

End of Part One…

…now let’s look at the real world of DevOps


DevOps: The Motivations

• Improve flow

• Build, measure, learn

• Accelerate development

• Automate QA

• Empower Operations

• Visibility

• Antifragility Jfokus 2015: 21st Century Software Delivery

(bit.ly/1JzlY6d)


http://bit.ly/1JzlY6d

Breaking Down Silos

Dev QA Ops

Hand-off delaysHand-off delays

Slow feedback Slow feedback

Lack of cohesion, shared understanding and joint accountability


Being agile is a prerequisite…

Organisational Challenges

• Delivering value continuously– Fear of failure

• Ownership and accountability– The DevOps team antipattern?

– Culture, support, innovation

• Complex systems– Failure should lead to enquiry, not blame


That’s the organisation…

…what about the tech?


Continuous Integration

• Goals

– Remove integration problems

– Avoid back-tracking

– Automated repeatable build

• Approach

– Determine branching strategy

– Create and follow processes

– Create a build pipeline


Build Pipeline


Example Build Pipeline

• Component Build

– Compile

– Unit Tests (surefire)

– Integration Tests (failsafe)

• Deployment onto QA Cloud

– Python Scripts + Chef to provision

– Verify success using Python


Example Build Pipeline

• Acceptance Tests– Cucumber (and Selenium)

• Performance Tests– Jmeter + Jenkins performance plugin

– Make sure environment is realistic!!

• Live Deployment?


Diving into the details…


Local Development



Local Development

• Goals– ‘One click’ setup

– Dev/Prod parity

– Easy testing (pre-canned data, service virtualisation)

• Tools– Sprout, Boxen, Homebrew, provisioning

– VMs (Vagrant), Docker (Fig)

– DbUnit, VCR, Wiremock


Provisioning

• Goals– Automated reliable environment build

– Treat servers as cattle, not pets

– Immutable infrastructure

• Tools– Puppet, chef, SaltStack, Ansible

– Packer.io, Veewee, Aminator

– Terraform, AWS Cloudformation


Testing



Testing

• Goals– Automated reliable testing

– Functional, cross-functional (NFR) covered

– Free QA team for exploratory/requirements

• Tools– Serenity BDD, Cucumber, JBehave, Spock

– Saboteur/Wiremock (bit.ly/1tU6wZj)

– JMeter, Gatling (bit.ly/1JIvOCI)


http://bit.ly/1tU6wZj

http://bit.ly/1JIvOCI

Serenity BDD


Serenity BDD


Visibility


Logging

• Goals

– Audit system/application processes

– Assist debugging

– Single point of reference (centralised)

• Tools

– Logstash, fluentd, flume

– Kibana (ELK Stack)


Kibana


Monitoring

• Goals

– Application/system-level metrics

– ‘Birds eye view’ dashboards

• Tools

– Codahale Metrics, Netflix Servo

– Graphite, InfluxDB, OpenTSDB

– Grafana, dashing.io





Alerting

• Goals– You know of problems before users

– Don’t turn a crisis into a drama

• Tools– Nagios, Zabbix, Sensu, Pagerduty

• Process– Have one, follow it, retrospect (root cause)


Putting it all Together…


Releases

• Goals– Deterministic deployment

– Reliable rollout and rollback (or fix-forward)

– Separate release from deployment

• Processes– Deploy with rolling upgrade, blue/green, canary

– Release with feature flags

– Two phase data migrations


Debugging DevOps

• Goals

– Core skills throughout team

– Appropriate tools

– Document (and follow) processes

• Tools

– grep, top, vmstat, netstat, tcpdump

– jps, jstat, jmap, jhat


The Future

• Portable application deployment– Docker and Rocket

• The operating system reinvented– CoreOS, Project Atomic, Ubuntu Snappy

• Modern cluster management– Providing a kernel for the data center

– Mesos, Kubernetes, Fleet, fabric8


The Future is Layered…


twitter.com/gabrtv/status/539805332432637952

https://twitter.com/gabrtv/status/539805332432637952

In Summary…


DevOps: The Motivations

• Improve flow

• Build, measure, learn

• Accelerate development

• Automate QA

• Empower Operations

• Visibility

• Antifragility Jfokus 2015: 21st Century Software Delivery

(bit.ly/1JzlY6d)


http://bit.ly/1JzlY6d

Bu-Dev-QA-Ops…


Thanks for Listening!

Comments and feedback are welcomed…

[email protected]

@danielbryantuk

Thanks:


mailto:[email protected]

Devoxx UK Conference

• 2014 “Moving to a DevOps Mode” bit.ly/1wdkh8K

– Daniel Bryant, Steve Poole

• Great DevOps content

• Get your tickets! www.devoxx.co.uk

– Tracks and content for every interest!


http://bit.ly/1wdkh8K

http://www.devoxx.co.uk/

Further Inspiration #shamlesspromotion

• Cloud news

– www.infoq.com/author/Daniel-Bryant

• Code and opinion

– java.dzone.com/users/daniel-bryant-uk

• Microservices

– www.voxxed.com/blog/author/danielbryantuk


http://www.infoq.com/author/Daniel-Bryant

http://java.dzone.com/users/daniel-bryant-uk

https://www.voxxed.com/blog/author/danielbryantuk/

Technology

LJC 2015 "The Crafty Consultants Guide to DevOps"