Security is Dead. Long Live Rugged DevOps: IT at Ludicrous Speed

  • View

  • Download

Embed Size (px)


Slides from Gene Kim and Joshua Corman's

Text of Security is Dead. Long Live Rugged DevOps: IT at Ludicrous Speed

  • 1.Security is Dead.Long Live Rugged DevOps:IT at Ludicrous SpeedJoshua Corman & Gene KimSession ID: CLD-106Session Classification: Intermediate

2. About Joshua Corman Director of Security Intelligence for Akamai Technologies Former Research Director, Enterprise Security [The 451 Group] Former Principal Security Strategist [IBM ISS] Industry: Expert Faculty: The Institute for Applied Network Security (IANS) 2009 NetworkWorld Top 10 Tech People to Know Co-Founder of Rugged Software BLOG: Things Ive been researching: Compliance vs Security Disruptive Security for Disruptive Innovations Chaotic Actors Espionage Security Metrics2 3. About Gene Kim Researcher, Author Industry: Invented and founded Tripwire, CTO (1997-2010) Co-author: Visible Ops Handbook(2006), Visible Ops Security (2008) Co-author: When IT Fails: The Novel, The DevOps Cookbook (ComingMay 2012) Things Ive been researching: Benchmarked 1300+ IT organizations to test effectiveness of IT controls vs.IT performance DevOps, Rugged DevOps Scoping PCI Cardholder Data Environment (#FAIL)3 4. Agenda Problem statement What is DevOps? What is Rugged? What is Rugged DevOps? Things you can do right away4 5. Potentially Unfamiliar Words You Will See Kanban Andon cord Sprints Rugged DevOps Bottleneck Systems thinking Controls reliance5 6. Problem Statement 6 7. Ludicrous Speed? 7 8. Ludicrous Speed8 9. Ludicrous Speed! 9 10. Ludicrous Fail?! 10 11. What Is DevOps? 11 12. Source: John Allspaw 13. Source: John Allspaw 14. Source: John Allspaw 15. Source: John Allspaw 16. Source: Theo Schlossnagle 17. Source: Theo Schlossnagle 18. Source: Theo Schlossnagle 19. Source: John Jenkins, 20. High Performing IT Organizations High performers maintain a posture of compliance Fewest number of repeat audit findings One-third amount of audit preparation effort High performers find and fix security breaches faster 5 times more likely to detect breaches by automated control 5 times less likely to have breaches result in a loss event When high performers implement changes 14 times more changes One-half the change failure rate One-quarter the first fix failure rate 10x faster MTTR for Sev 1 outages When high performers manage IT resources One-third the amount of unplanned work 8 times more projects and IT services 6 times more applicationsSource: IT Process Institute, 2008 Source: IT Process Institute, 2008 21. 2007: Three Controls Predict 60% OfPerformance To what extent does an organization define,monitor and enforce the following? Standardized configuration strategy Process discipline Controlled access to production systemsSource: IT Process Institute, 2008 22. What Is Rugged? 23 23. Rugged Software DevelopmentJoshua Corman, David Rice, Jeff Williams2010 24. RUGGED SOFTWARE 25. so software not only needs to be 26. FAST 27. AGILE 28. Are You Rugged? 29. HARSH 30. UNFRIENDLY 31. THE MANIFESTO 32. I recognize that my code will be used in ways Icannot anticipate, in ways it was not designed, and for longer than it was ever intended. 33. CrossTalk 34. What Is Rugged DevOps? 39 35. Source: James Wickett 36. Source: James Wickett 37. Survival Guide/ Defensible Infrastructure 38. Survival Guide/Pyramid Operational Discipline Defensible Infrastructure 39. Survival Guide/Pyramid Situational Awareness Operational Discipline Defensible Infrastructure 40. Survival Guide/Pyramid Countermeasures Situational Awareness Operational DisciplineDefensible Infrastructure 41. Zombie Proof Housing s Situational Awareness Operational DisciplineDefensible Infrastructure 42. Zombie Proof Housing s Situational Awareness Operational DisciplineDefensible Infrastructure 43. Zombie Proof Housing s Situational Awareness Operational DisciplineDefensible Infrastructure 44. Zombie Proof Housing s Situational Awareness Operational DisciplineDefensible Infrastructure 45. Zombie Proof Housing s Situational Awareness Operational DisciplineDefensible Infrastructure 46. Zombie Proof Housing s Situational Awareness Operational DisciplineDefensible Infrastructure 47. Zombie Proof Housing s Situational Awareness Operational DisciplineDefensible Infrastructure 48. Source: James Wickett 49. DevOps: Its A Real Movement I would never do another startup that didntemploy DevOps like principles Its not just startups its happening in theenterprise and in public sector, too I believe working in DevOps environments willbe a necessary skillset 5 years from now 50. How Do You DoRugged DevOps?57 51. The Prescriptive DevOps Cookbook DevOps Cookbook Authors Patrick DeBois, Mike Orzen,John Willis Goals Codify how to start and finishDevOps transformations How does Development, ITOperations and Infosecbecome dependable partners Describe in detail how toreplicate the transformationsdescribe in When IT Fails: TheNovel 52. Arc 1: Decrease Cycle Time Of Releases Create determinism in the release process Move packaging responsibility to development Release early and often Decrease cycle time Reduce deployment times from 6 hours to 45 minutes Refactor deployment process that had 1300+ steps spanning 4 weeks Never again fix forward, instead roll back, escalating anydeviation from plan to Dev Ensure environments are properly built before deployment begins Control code and environments down the preproduction runways Hold Dev, QA, Int, and Staging owners accountable for integrity 53. Arc 2: Increase Production Resilience To preserve and increase throughput, elevate preventiveprojects and maintenance tasks Document all work, changes and outcomes so that it isrepeatable Protect the flow of planned work (e.g., tickets bouncingaround for weeks, causing features to slip into next sprint) Ops builds Agile standardized deployment stories Maintains adequate situational awareness so that incidentscould be quickly detected and corrected Standardize unplanned work and escalations Continually seek to eradicate unplanned work and increasethroughput 54. Arc 3: Remove Complexity, Attack Surface AndWaste Elective complexity adds to technical debt Infosec (and everyone) wins when we take workout of the system Understand where controls reliance is placedand what matters to the business 61 55. Meeting The DevOps Leadership Team Typically led by Dev, QA, IT Operations andProduct Management Our ultimate goal is to add value at every step inthe flow of work See the end-to-end value flow Shorten and amplify feedback loops Help break silos (e.g., server, networking, database) 56. Definition: Agile Sprints The basic unit of development in Agile Scrums,typically between one week and one month At the end of each sprint, team should havepotentially deliverable productAha Moment: shipping product implies not just code its the environment, too!63 57. Help Dev And Ops Build Code AndEnvironments Dev and Ops work together in Sprint 0 and 1 tocreate code and environments Create environment that Dev deploys into Create downstream environments: QA, Staging,Production Create testable migration procedures from Dev all theway to production Integrate Infosec and QA into daily sprintactivities 58. Definition: Andon Cord 65 59. Integrate Ops Into Dev Embed Ops person into Dev structure Describes non-functional requirements, use casesand stories from Ops Responsible for improving quality at the source(e.g., reducing technical debt, fix known problems,etc.) Has special responsibility for pulling the Andon cord 60. Integrate Dev Into Ops MobBrowser case study: Waking up developersat 3am is a great feedback loop: defects getfixed very quickly Goal is to get Dev closer to the customer Infosec can help determine when its too close (andwhen SOD is a requirement) 61. Keep Shrinking Batch Sizes Waterfall projects often have cycle time of oneyear Sprints have cycle time of 1 or 2 weeks When IT Operations work is sufficiently fast andcheap, we may decide to decouple deploymentsfrom sprint boundaries (e.g., Kanbans) 62. Definition: Kanban Board Signaling tool to reduce WIP and increase flow69 63. IT Operations Increases Process Rigor Standardize deployment Standardize unplanned work: make it repeatable Modify first response: ensure constrainedresources have all data at hand to diagnose Elevate preventive activities to reduce incidents 64. Help Development Help them see downstream effects Unplanned work comes at the expense of plannedwork Technical debt retards feature throughput Environment matters as much as the code Allocate time for fault modeling, asking whatcould go wrong? and implementingcountermeasures 65. Help QA Ensure test plans cover not only codefunctionality, but also: Suitability of the environment the code runs in The end-to-end deployment process Help find variance Functionality, performance, configuration Duration, wait time and handoff errors, rework, 66. Help IT Operations The best way to avoid failure isto fail constantly Harden the productionenvironment Have scheduled drills to crashthe data center Create your chaos monkeys tointroduce faults into the system(e.g., randomly kill processes,take out servers, etc.) Rehearse and improveresponding to unplanned work NetFlix: Hardened AWS service StackOverflow Amazon firedrills (Jesse Allspaw)