Building Scalable Organizations

  • Published on
    28-Nov-2014

  • View
    568

  • Download
    0

Embed Size (px)

DESCRIPTION

AKF Partners' presentation to NYC CTO Club on scaling organizations. The premise is that organizational scale is just as important as architectural scale.

Transcript

<ul><li> 1. Building Scalable Organizations Marty Abbott Mike Fisher Partners AKF Partners We wrote the book on scalability www.akfpartners.com </li> <li> 2. 2 Our Business Focused on helping companies scale their architecture, organization, and processes 11 Partners, Associates, and Consultants Over 350 Clients in 12 countries We wrote the book on scalability www.akfpartners.com </li> <li> 3. The Power of Customer Misbehavior: Drive Growth and Innovation by Learning From Your Customers 3 Published Books Scalability Rules: 50 Principles for Scaling Web Sites Addison-Wesley Professional (2011) Translated into Japanese and simplified Chinese The Art of Scalability: Scalable Web Architecture, Processes, and Organizations for the Modern Enterprise Addison-Wesley Professional (2009) Translated into Korean, simplified Chinese, and English for India Palgrave Macmillan (2013) We wrote the book on scalability www.akfpartners.com </li> <li> 4. Drivers of Innovation Context for the Agile Organization We wrote the book on scalability www.akfpartners.com </li> <li> 5. Innovation 5 Cognitive Conflict As cognitive conflict increases (what and why), so does innovation + **De Dreu and Weingart 2003 Cognitive Conflict We wrote the book on scalability www.akfpartners.com </li> <li> 6. As affective conflict increases (who and how), innovation decreases Innovation Affective Conflict Cognitive Conflict - + **Amason and Mooney, 1999; De Dreu and Weingart 2003 6 Affective Conflict We wrote the book on scalability www.akfpartners.com </li> <li> 7. Innovation 7 Empowerment Feelings of empowerment increase innovation Sense of Empowerment Affective Conflict Cognitive Conflict + - + **Spreitzer, De Janasz, 1999 We wrote the book on scalability www.akfpartners.com </li> <li> 8. Innovation 8 Org Boundaries Organizational boundaries across which collaboration must happen increase affective conflict Sense of Empowerment + Affective Conflict Cognitive Conflict - + Organizational Boundaries + ** Tajfel 1974; Hogg and Terry 2000; Cummings and Kiesler 2005 We wrote the book on scalability www.akfpartners.com </li> <li> 9. Both cognitive and affective conflict driven by diversity in experiences Innovation Sense of Empowerment + Affective Conflict - + + ** Burt 2001; Bassett Jones 2005; Abbott, Lyytinen et al. 2010 9 Diverse Experience Cognitive Conflict + Organizational Boundaries Experiential Diversity We wrote the book on scalability www.akfpartners.com </li> <li> 10. Best when high number of non-overlapping network links (loose acquaintances) outside of the organization Agile Team ** Burt 2001; Bassett Jones 2005; Abbott, Lyytinen et al. 2010 10 Network Diversity Network Diversity We wrote the book on scalability www.akfpartners.com </li> <li> 11. The greater the network diversity, the higher the innovation Network Diversity Innovation 11 Network Diversity Sense of Empowerment + Affective Conflict Cognitive Conflict - + + + ** Burt 2001 Organizational Boundaries Experiential Diversity We wrote the book on scalability www.akfpartners.com + </li> <li> 12. Barriers to Success Context for the Agile Organization We wrote the book on scalability www.akfpartners.com </li> <li> 13. 13 Monolithic Applications Product: Foo.com Agile Tm 1 Agile Tm 2 Agile Tm N Conflict! Network Diversity Innovation Empowerment Affective Conflict Cognitive Conflict Organizational Boundaries Experiential Diversity We wrote the book on scalability www.akfpartners.com </li> <li> 14. 14 Functional Organizations Conflict! Conflict! Biz PM Eng QA INF Ops Conflict! Network Diversity Innovation Conflict! Empowerment Affective Conflict Cognitive Conflict Agile Team 1 Agile Team N Conflict! Organizational Boundaries Experiential Diversity We wrote the book on scalability www.akfpartners.com </li> <li> 15. Innovation Decreases: 1) No Empowerment/Ownership 2) Does not leverage network 3) Does not leverage experience 15 Top Down Innovation Network Diversity Innovation Empowerment Affective Conflict Cognitive Conflict Boss Organizational Boundaries Experiential Diversity Product Teams We wrote the book on scalability www.akfpartners.com </li> <li> 16. Moving Forward Steps to Create a Truly Agile Organization We wrote the book on scalability www.akfpartners.com </li> <li> 17. Agile Product Teams aligned with Services Multi-Disciplinary Agile 17 To Succeed We Must Move From This: Monolithic Products To This: Service or Resource Oriented Systems Silod Organizations with Agile Overlays Agile within only software development Teams We wrote the book on scalability www.akfpartners.com </li> <li> 18. Learning Organizations 18 To Succeed We Must Move From This: Moving from Fire to Fire To This: Hubris and Wicked Smart People Humility and Great Performing Teams Top Down, Sparse Innovation Organization Wide Innovation and Ownership We wrote the book on scalability www.akfpartners.com </li> <li> 19. Agile Processes, Agile Orgs, and Innovation The Future We wrote the book on scalability www.akfpartners.com </li> <li> 20. 20 Yesterday Monolithic Application Waterfall Process Functional Organizations Product: Foo.com Product Engineering QA Operations We wrote the book on scalability www.akfpartners.com </li> <li> 21. 21 Today Agile teams dont perform well with monolithic apps (conflicts). Teams still engage in affective conflict (bad) about who and how. Innovation increases due to experiential diversity but is still sub-optimized. Product Engineering QA Operations Product: Foo.com We wrote the book on scalability www.akfpartners.com </li> <li> 22. 22 The Fix Services or Resource Based Architectures Multidisciplinary Development and Deployment Teams Search Search Team Registration Registration Team We wrote the book on scalability www.akfpartners.com </li> <li> 23. 23 The Reasons Teams that own a product (or service) perform better. Functional roles and managers add little value in Agile. The managers dont see enough of the individual performance to be worthwhile. Conflict across organizations about conflicting goals and ownership do NOT increase value added cognitive conflict. Innovation works best when there is a single team identity not identity conflict. We wrote the book on scalability www.akfpartners.com </li> <li> 24. NoOps Organization Model: The NoOps model is most often associated with Netflix and their near 100% use of the AWS Cloud. High scalability and fast Time to Market are primary goals of this model. Cost management is secondary. Budget and responsibility of figuring out how to use public cloud IaaS was given directly to the Development organization after failed Relies heavily on near 100% automation for all critical development and releases processes. The concept of ITOps and private data centers exist at Netflix however they support the physical Netflix service of delivering DVDs. Netflix has a developer-oriented culture starting with the CEO Reed Hastings who was a developer. 24 NoOps Netflix Model Services Teams CORE Team (Cloud Operations Reliability Engineering) Responsible for: - Monitoring - Application Performance Management attempts to quickly scale out data centers during periods of extreme growth. A handful of DevOps engineers are coding and running the build tools and bakery, and updating the base AMI from time to time. Several hundred development engineers use these tools to build code, run it in a test account in AWS, then deploy it to production themselves. They never have to have a meeting with ITops, or file a ticket asking someone from ITops to make a change to a production system, or request extra capacity in advance... This is part of what we call NoOps. The developers used to spend hours a week in meetings with Ops discussing what they needed, figuring out capacity forecasts and writing tickets to request changes for the datacenter. Now they spend seconds doing it themselves in the cloud There is no central control, the teams are responsible for figuring out their own dependencies and managing AWS security groups that restrict who can talk to who. Adrian Cockcroft - http://perfcap.blogspot.com/2012/03/ops-devops-and-noops-at-netflix. html We wrote the book on scalability www.akfpartners.com </li> <li> 25. Full Stack Organization Model: Minimizing risk and moving fast are primary goals of this model. Teams responsible for services have all of the necessary skill to be innovative and release frequently. They have knowledge of UI/UX, Mid- Facebook emphasizes Full Stack development teams that work together to deliver services. Facebook also has a sizeable Release Engineering team that assists with releases multiple times daily and weekly. Facebook Infrastructure Engineering is a separate organization that is responsible for data centers, capacity planning, etc. Facebook has a developer-oriented culture starting with the CEO who was a developer. Release Engineering Responsible for: - Controlling releases - Daily pushes - Weekly pushes 25 Full Stack Facebook Model Full Stack Dev Team Infrastructure Engineering - Data Centers - Server build out - Capacity - Performance - Incident Management Tier services, Databases, Hosting environments, Networks, etc. None of the previous principles work without operations and engineering teams that work together seamlessly, and can handle problems together as a team. The person responsible for something must have control over it. At Facebook we push code to the site every day, and the person who wrote that code is there to take responsibility for it. .. We have three people working on photos. Each of those three people knows photos inside and out, and can make decisions about it. So when something needs to change in photos they get it done quickly and correctly. Peter Deng - https://www.facebook.com/note.php?note_id=409881258919 We wrote the book on scalability www.akfpartners.com </li> <li> 26. Hybrid Infrastructure Core &amp; Services: The comparable SaaS company transitioned from a 3-tiered Engineering/Services Delivery (similar to Prod Ops at Citrix)/Infrastructure to a model that combined Engineering and Services Delivery into a single team while dividing Infrastructure into two team dedicated to core services and delivery services. This model allows for flexibility based on product line. Mature, slow-growth product lines can leverage shared Infrastructure teams and do not need dedicated Infrastructure team members in the agile teams. High-change/High-Growth services need dedicated Infrastructure service team members embedded within the Agile team for fast time-to-market and innovation. 26 Intuit Model Motivation For Change We wrote the book on scalability www.akfpartners.com </li> <li> 27. Spotify Model A realworld example of a service oriented, crossfunctional team can be found at Spotify. Their organizational structure is not functionally aligned but rather is organized into small Agile teams, which they refer to as Squads. These are selfcontained teams organized around a deliverable or service which the business provides. The following image and descriptions are taken from Henrik Kniberg and Ande...</li></ul>