Closing the DevOps Loop

Confidential, Dynatrace, LLC

Closing the DevOps Loop

Helen BealDevOpsologist

@helenranger4

Andreas GrabnerTechnology Strategist@grabnerandi


A Short History of DevOps

2007

2008

2009 2013

Andrew ShaferAgile Conference, Toronto

Patrick Dubois

Agile System AdministratorGroup

DevOps Days Belgium#devops

John Allspaw &Paul HammondFlickR

‘Gartner ExploresDevOps’Cameron Haight

Mike Gualateri, Forrester– ‘NoOps’

Ronnie Colville of Gartner: ‘ARA is a Key to DevOps’

20142010

2011

2012

2015

2016

DevOps Days Belgium5 Year anniversary Ghent

The Phoenix Project GAME!

DevOps


CALMS


CALMS


The Three Ways


The Three Ways


The Three Ways


The Three Ways

1.Flow

2.Feedback loops

3. Experimentation and learning


Ideation

Integration

ValidationOperation

Realisation

DevO

ps

The DevOps Loop ©Ranger4


What are Metrics that Matter?


What is Digital Performance

Management?


mobile

browser

network

multi-geo3rd parties cloud

containers

services

codehosts

synthetic

logsbusinesstransaction

applications

sdnrelax

full-stack, broad, hyper-scale


unified, modern, all you need

@grabnerandihttps://dynatrace.github.io/ufo/

“In Your Face” Data!

@grabnerandi

Time of D

eployment

Availability dropped to 0%

#1: Availability -> Brand Impact

#2: UX by Device, Geo, AppVersion …

User Experience

AppVersion

(A/B Testing)

Device Geo

@grabnerandi

New Deployment + Mkt Push

Increase # of unhappy users!

Decline in Conversion Rate

Overall increase of Users!

#3: User Experience -> Conversion

Spikes in FRUSTRATED Users!

@grabnerandi

#4: Resource Cons -> Cost per Feature

4x $$$ to IaaS

@grabnerandi

#5: Performance -> Behavior

@grabnerandi

From Reactive to ProactiveYou measure it! from Dev (to)

Ops

@grabnerandi

Scaling an Online Sports Club Search Service

2015201420xx

Response Time

2016+

1) 2-Man Project 2) Limited Success

3) Start Expansion

4) Performance Slows Growth Users

5) Potential Decline?

@grabnerandi

Early 2015: Monolith Under Pressure

Can‘t scale vertically endlessly!

May: 2.68s 94.09% CPU Bound

April: 0.52s

@grabnerandi

From Monolith to Services in a Hybrid-Cloud

Front End inGeo-Distributed

Cloud

Scale Backendin ContainersOn Premise

@grabnerandi

Go live – 7:00 a.m.

@grabnerandi

Go live – 12:00 p.m.

What Went Wrong?

@grabnerandi

26.7s Load Time5kB Payload

33! Service Calls99kB - 3kB for each call!

171! Total SQL Count

Architecture ViolationDirect access to DB from frontend service

Single search query end-to-end

@grabnerandi

The fixed end-to-end use case“Re-architect” vs. “Migrate” to Service-Orientation

2.5s (vs 26.7) 5kB Payload

1! (vs 33!) Service Call5kB (vs 99) Payload!

3! (vs 177) Total SQL Count

@grabnerandi

@grabnerandi

You measure it! from Dev (to) Ops

@grabnerandi

Build 17 testNewsAlert OKtestSearch OK

Build # Use Case Stat # APICalls # SQL Payload CPU

1 5 2kb 70ms1 35 5kb 120ms

Use Case Tests and Monitors Service & App Metrics



1 4 1kb 60ms34 171 104kb 550ms

Ops# Servers Usage UX

1 0.5% 7.2s

1 63% 5.2s

1 4 1kb 60ms2 3 10kb 150ms

1 0.6% 3.2s

6 75% 2.5s

Build 35 testNewsAlert -testSearch OK

- - - -2 3 7kb 100ms

- - -

4 80% 2.0s

Continuous Innovation and Optimization

Re-architecture into „Services“ + Performance Fixes

Scenario: Monolithic App with 2 Key Features

Reduce Lead Time: Stop Bad Code Early, Automated through Better DevOps Feedback Loops

CI/CD: Level-Up Functional to to Architectural & Performance

regression detection

Perf: Focus on the tough performance issues in

Load Testing

Dev: Check-In BETTER Code

Ops: Keep your end users and environment

happy

Biz: Innovate on End User Behavior Analysis

Want to see a demonstration?

BeDevOpstastic