35
@mattstratton Matt Stratton DevOps Advocate & Thought Validator, PagerDuty THE PSYCHOLOGY OF CHAOS ENGINEERING

THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

Page 1: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

Matt StrattonDevOps Advocate & Thought Validator, PagerDuty

THE PSYCHOLOGY OF CHAOS ENGINEERING

Page 2: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

Page 3: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

let’s set some agreement

Page 4: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

Chaos Engineering is the discipline of experimenting on a system in order to build confidence in the

system’s capability to withstand turbulent conditions in production.

https://principlesofchaos.org/

Page 5: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any

problems, we can still learn the lessons about the weaknesses of our system, and build automatic recovery mechanisms to deal with them. So

next time an instance fails at 3 am on a Sunday, we won’t even notice.

Netflix Technology Blog, 2011

https://bit.ly/netflix-chaos

Page 6: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

perceptions

Page 7: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

“isn’t all engineering chaotic?”

Page 8: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

“isn’t all engineering chaotic?”

Page 9: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

It’s not about breaking things

Page 10: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

It’s not about breaking things

Page 11: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

look, I know you know this

Page 12: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

I’m gonna say it anyway

Page 13: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

these are experiments

Page 14: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

Page 15: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

we know this, Matty

Page 16: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

how we talk about things matters

Page 17: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

people get nervous

Page 18: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

“You want to do what in production??”

Page 19: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

use your monitoring like it’s for real

because it is

Page 20: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

use your monitoring like it’s for real

because it is

Something’s broken…

…it’s your fault

Page 21: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

but what about the people?

Page 22: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

how does it make you feel to know Netflix practices chaos engineering?

Page 23: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

what about your bank?

Page 24: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

Page 25: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

data, such as it is

Page 26: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

Page 27: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

Page 28: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

management can get……nervous

Page 29: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

consider your words

Page 30: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

it’s about the philosophy

Page 31: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

Page 32: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

safety first

Page 33: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

• Know your conditions - when will you shut down the experiment?

• This isn’t about causing stress on your people - be transparent

• There are humans at the other end of those numbers

Page 34: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

https://speaking.mattstratton.com

Page 35: THE PSYCHOLOGY OF CHAOS ENGINEERING · By running Chaos Monkey in the middle of a business day, in a carefully monitored environment with engineers standing by to address any problems,

@mattstratton

Pagey Says….