37
z Performance Hackathons Trulia’s obsession with speed and scale

Performance Hackathons: Trulia’s Obsession With Speed and Scale

Embed Size (px)

DESCRIPTION

As presented at FutureStack 2014 Thursday October 9, 2014 Fort Mason Center, San Francisco, CA "Dev Ops” is not just a trendy job title — it’s a philosophy whose tenets are integral to creating realistic expectations for application developers. Learn how Trulia uses New Relic as a tool in performance hackathons, where developers and ops engineers join forces to identify performance bottlenecks in its products. By identifying system optimizations, coding inefficiencies, and more, these hackathons benefit Trulia’s customers as well as its performance KPIs; most importantly, they have helped Trulia’s engineering team blur the line between operations and development.

Citation preview

Page 1: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Performance HackathonsTrulia’s obsession with speed and scale

Page 2: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Hi

Page 3: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

We’re Chris & LouisWe work together.

Page 4: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

We’re from TruliaIt’s a nice place to work.

Page 5: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Pardon the interruption

Page 6: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

© Naotake Murayama, CC BY 2.0https://www.flickr.com/photos/naotakem/8065291641/

Page 7: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Moving right along

Page 8: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Maintaining a start-up vibe as we scale2007

• 17 Engineers• 1 building

2014• Over 200 Engineers• 5 floors, 3 buildings, 2 cities

Chris Sessions
Add examples of data from then and now. (Munin, Cacti, …)
Page 9: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

R. WestrumSociologist, Professor… and Ufologist

Page 10: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Westrum’s Three Cultures ModelPathological (power-oriented)

Pattern of ResponseLow CooperationMessengers Shot

Failure Leads to Scapegoating

Page 11: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Westrum’s Three Cultures ModelBureaucratic (rule-oriented)

Pattern of ResponseModest Cooperation

Messengers NeglectedFailure Leads to Justice

Page 12: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Westrum’s Three Cultures ModelGenerative (performance-oriented)

Pattern of ResponseHigh Cooperation

Messengers TrainedFailure Leads to Inquiry

Page 13: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

What’s worked for us• Weekly release meetings• Regular tech learning sessions• Lunch roulettes, group outings, regular and

impromptu HH

Page 14: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

What’s worked for us• Innovation weeks & providing a safe environment

for risk• Post-mortems are productive• Show data everywhere• Regular performance and scalability hackathons

Page 15: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Wait…Isn’t this the New Relic track?

Page 16: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Dev/OpsA most ingenious paradox

Page 17: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

© Matt Neale, CC BY 2.0https://www.flickr.com/photos/mattneale/4686273802/

Page 18: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

fa·cil·i·ta·tor noun \fə-ˈsi-lə-ˌtā-tər\one that helps to bring about an outcome (as learning, productivity, or communication) by providing indirect or unobtrusive assistance, guidance, or supervision

— Merriam-Webster

Page 19: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Enter the hackathon

Page 20: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

InceptionSite performance is a top-line KPI at Trulia.

Page 21: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

InceptionWe have performance monitors everywhere, and all devs have access to New Relic, but…

Page 22: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

InceptionAs developers, we sometimes have trouble balancing tech initiatives and product work. Let’s block off a couple of hours each week to address this.

Page 23: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Iterate.… and then do it again.

Page 24: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Take one• This is great• We need Ops here

Page 25: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Take twoInstead of a small group, let’s invite everyone!At the same time, make it clear that…• Attendance is optional, and• Participation is mandatory

Page 26: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

What we found

Page 27: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Example 1: Legacy code

Page 28: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Example 2: Server oddities

xfs on a web server.

… really?

Page 29: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

You can do it!How to build your own

Page 30: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

… but, before we get started.Let’s take a moment to acknowledge our privilege.

Page 31: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Step 1: Get insight into your apps.We like New Relic.

Page 32: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Step 2: Set aside some timeWe started with 2 hours a week, and that feels right.

Page 33: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Step 3: Find an area of the app to work onE.g., key transactions

Page 34: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Step 4: Make it better!• Research• Hack• Create some PRs• File follow-ups

Page 35: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Step 5: IterateIt’s more important to be making changes than it is to have every change be a huge success.

(Success helps, though.)

Page 36: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Thanks

Page 37: Performance Hackathons: Trulia’s Obsession With Speed and Scale

z

Q&A