Building Next Gen Applications and Microservices

  • Published on
    16-Apr-2017

  • View
    148

  • Download
    1

Transcript

PowerPoint Presentation

Building Next Gen Applications and MicroservicesSteve Mirman, IBM Bluemix Architect@SteveMirmanJune 15, 2016

Progression to Microservices2

Continuous IntegrationAgileHeavyweight developmentContinuous DeploymentDevOpsMicroservices

http://www.infoworld.com/article/3075880/application-development/microservice-architecture-is-agile-software-architecture.html

Since the term microservices hit the software industry like a bolt of lightning in 2014, technical professionals of all stripes have been analyzing this new architectural style from their own frames of reference. Having lived through the rise and fall of service-oriented architecture, I had the same reaction as many others: How does microservice architecture differ from SOA? The more I learned about the case studies that led to the creation of the term microservices, the more I recognized that this question would not capture the essence of this new software movement.

The first thing to recognize about the microservice movement is that it has been empirically defined. Microservice architecture emerged from a common set of patterns evident in companies like Amazon, Netflix, SoundCloud, and Gilt (now part of HBC Digital). Applications at these companies that were monolithic evolved over time into decomposed services that communicated via RESTful APIs and other network-based messaging protocols.

However, the commonalities were not restricted to architectural patterns. The companies at the forefront of microservices also shared a common approach to software development, had similar organizational structures and cultural practices, and shared an affinity for cloud-based infrastructure and automation. Many companies that succeed with microservices have followed a similar progression driven by a desire for development speed and scalability.

The agile progressionIn early 2001, a group of software professionals published the Agile Manifesto as a statement of values on how to improve software development. Although the principles stated were not new -- they were a consolidation of ideas from extreme programming, scrum, lean, and more -- the unified voice caught the industrys attention. Just as microservice architecture is frequently defined in contrast to monolithic architecture, the manifesto differentiates agile software development from documentation-driven, heavyweight software development processes. The agile approach sought to remove the overhead and risk of large-scale software development by using smaller work increments, frequent iterations, and prototyping as a means of collaboration with users. The adoption of agile methods in the industry grew consistently following the publication of the manifesto.

The spread of agile methods also led to the popularization of continuous integration (CI) in the software industry, a common practice from extreme programming. CI sought to combine software components as early in the lifecycle as possible in order to minimize the impact of code integration issues. However, many of the early agile adopters found that once they had removed the bottlenecks in the coding, they hit snags in releasing the software. These difficulties were only amplified by the popularization of SaaS as an increasingly preferred deployment option.

To address the need for more frequent software releases, the practice of continuous delivery (CD) started to gain traction in 2006, taking the internal CI concept and applying it to the external view of software deliverables. CD takes scrums quality-focused potentially shippable product increment literally, defining a deployment pipeline to bring changes to production as quickly as possible. Virtualization and cloud computing removed technological barriers to CD, and new tools emerged to institutionalize CD practices. The combination of agile and CD was improving both the speed of production and the quality of the software produced.

Still, there were bottlenecks. Agiles primary scope was on the development of software, while CD extended that scope to include production deployment, an operations task. In most organizations, development and operations were consciously divided in both reporting and mission. In 2009, John Allspaw and Paul Hammond from Flickr gave an influential talk at the OReilly Velocity conference detailing how they had bridged this gap. From experiences like theirs, the devops movement arose to address this cultural divide.

Organizations found that combining development and operations responsibilities in the same team led to highly effective continuous delivery practices. As collaboration increased between dev and ops, so did empathy. Developers designed solutions that included an operational perspective from the outset, and operations people used an engineering approach to tackle problems that were previously dealt with procedurally. Greater use of automation in day-to-day tasks resulted in greater system stability and resilience. Netflix's Simian Army approach to testing the resilience of production systems is an extreme example of this.

The organizations that followed this "agile progression" -- from addressing software development to deployment to organizational structure -- now had alignment in these areas. Many of these agile pioneers were Web native and provided their software solutions in a single application stack. As the complexity and scale of their businesses increased, they found that this architecture not only became an impediment to new feature delivery, but caused stability issues due to brittleness and lack of scalability. In parallel, several companies -- such as SoundCloud -- discovered that breaking their monolithic applications into discrete, business-focused services was more suitable to their agile delivery methodology and devops culture. This is the true origin of microservice architecture. Microservices are the architectural phase of the agile progression.

In search of agile software architectureIn a 2013 post on his blog Coding the Architecture, software architect Simon Brown speculated about what an agile software architecture would look like. He points out that an agile architecture does not naturally emerge from agile development practices. Rather, it must be consciously sought. Note that his description of agile software architecture is a perfect match for microservice architecture (emphases mine): If we look at the characteristics of an agile software architecture, we tend to think of something that is built using a collection of small, loosely coupled components/services that collaborate together to satisfy an end-goal. This style of architecture provides agility in a number of ways. Small, loosely coupled components/services can be built, modified and tested in isolation, or even ripped out and replaced depending on how requirements change. This style of architecture also lends itself well to a very flexible and adaptable deployment model, since new components/services can be added and scaled if needed.

Companies like Amazon, Netflix, SoundCloud, and Gilt encountered an architectural bottleneck when they reached a certain scale. This barrier motivated them to focus on the architecture, as Brown encourages, and they landed on microservices.

There are important lessons to be gleaned from tracking this agile progression through to its architectural phase. First of all is that agile software development, continuous delivery, devops culture, and microservice architecture are all bound by a common set of goals: to be as responsive as possible to customer needs while maintaining high levels of software quality and system availability. Although these phases evolved in a particular order from the industry perspective, there is no right sequence for an individual organization to follow. For example, Amazon adopted an architecture that forced changes to its organization. By contrast, SoundCloud evaluated its delivery methodology and made changes to its team structure and architecture as a result.

If you are evaluating how you can adopt microservices, it is important to understand where your organization is on the agile progression. Are you an agile shop? If so, who is looking after the architecture of your applications? If not, are you on a path to adopt agile practices? Do you have CD and deployment pipelines in place? What is the relationship between your development and operations teams, and who owns those responsibilities? Weighing the answers to these questions against your primary goals for adopting microservices will help you chart the right course to success that includes incremental wins along the way.

2

1. From Heavyweight Development to Agile3A consolidation of ideas from Extreme Programming, Scrum, Lean, etc.Tried to remove the overhead and risk of large scale software development by having:Smaller work incrementsFrequent iterationsRapid prototyping

Continuous IntegrationAgileHeavyweight developmentContinuous DeploymentDevOpsMicroservices

2. From Agile to Continuous Integration (CI)4Sought to combine software components as early in the lifecycle as possible in order to minimize the impact of code integration issues.Virtualization and automated testing removed technological barriers to CI.The adoption of Agile led to a growth in CI, which was a common practice in Extreme Programming.

Continuous IntegrationAgileHeavyweight developmentContinuous DeploymentDevOpsMicroservices

The spread of agile methods also led to the popularization of continuous integration (CI) in the software industry, a common practice from extreme programming. CI sought to combine software components as early in the lifecycle as possible in order to minimize the impact of code integration issues. However, many of the early agile adopters found that once they had removed the bottlenecks in the coding, they hit snags in releasing the software. These difficulties were only amplified by the popularization of SaaS as an increasingly preferred deployment option.

4

3. From CI to Continuous Delivery (CD)5CD defines a deployment pipeline to bring changes to production as quickly as possible.Is an instantiation of Scrums potentially shippable product increment.

Continuous IntegrationAgileHeavyweight developmentContinuous DeploymentDevOpsMicroservices

CD takes scrums quality-focused potentially shippable product increment literally, defining a deployment pipeline to bring changes to production as quickly as possible. Virtualization and cloud computing removed technological barriers to CD, and new tools emerged to institutionalize CD practices. The combination of agile and CD was improving both the speed of production and the quality of the software produced.

5

4. From CD to DevOps6In most organizations development and operations were separate.Organizations where operations and development were together were more successful at establishing CD practices.Engineering approaches were taken to problems that were previously dealt with procedurally.This led to:Higher automation in day to day tasksGreater stability and resilience

Continuous IntegrationAgileHeavyweight developmentContinuous DeploymentDevOpsMicroservices

5. From DevOps to Microservices7The architectural phase where large monolithic applications are broken down into discrete, business focused services.This led to:Higher development parallelizationGreater scalability and utilization

Continuous IntegrationAgileHeavyweight developmentContinuous DeploymentDevOpsMicroservices

7

Microservice OutcomesIncrease SpeedReduce CostImprove ResilienceEnable Visibility8

Microservice Keys to Success9

Architecture

Methodology

Technology

Organization

Organizational SuccessAre teams aligned to business or technology?How are responsibilities divided between teams?At what level of organization are development and operations divided? DevOps or Development and Operations?What are team sizes and skills?Dependencies and cross-team communications?Power distribution between teams?10

Architecture

Methodology

Technology

Organization

10

Methodological SuccessProduct or projects?Agile or waterfall?Who controls business requirements?Fear of change, or continuous delivery?Degree of automation in deployment and operations?11

Architecture

Methodology

Technology

Organization

The waterfall model is one in which each phase of a products life cycle takes place in sequence, so that progress flows steadily downwards through these phases like a waterfall.

Nobody invented the waterfall method. Rather it was inherited by enterprise software developers from other industries where, once a particular phase of production is complete (like laying the foundations of a building for example), it was incredibly costly or impractical to go back and make changes. The waterfall was only codified when people subsequently realised that it wasnt the only way of doing things.

Pros of the waterfall methodPotential issues that would have been found during development can be researched and bottomed out during the design phase. If appropriate meaning an alternate solution is selected before any code is written.The development process tends to be better documented since this methodology places greater emphasis on documentation like requirements and design docs. Many organisations find this reassuring.Because the waterfall process is a linear one it is perhaps easier to understand, especially for non-developers or those new to software development. Often teams feel more comfortable with this approach.Cons of the waterfall methodOften the people were building software for (the client) dont know exactly what they need up front and dont know whats possible with the technology available. This way of working doesnt handle this well.Solution designers often arent able to foresee problems that will arise out of the implementation of their designs.Changes to requirements (e.g. like those resulting from new technologies, changes in a market or changes to business goals) cant easily be incorporated with the waterfall method and there are often laborious change control procedures to go through when this happensThe process doesnt have its own momentum

An Agile software development methodology such as Scrum is one which eschews a linear, sequential approach in favour of an incremental, iterative one.

Instead of extensive planning and design up front, Agile methodologies allow for changing requirements over time by using cross-functional teams incorporating planners, designers, developers and testers which work on successive iterations of the product over fixed time periods (timeboxes). The work is organised in to a backlog that is prioritised in to exact priority order based on business (or user) value.These teams are self-organising, include a representative of the business (the product owner). The emphasis is on efficient f...

Recommended

View more >