Evolving SoftwareHow to manage explosive growth

without exploding code

Why does software change?

Adding a featureGrowth

Behavior is modifiedFixing a bugRepair

Improving structureRefactoring Behavior stays the

sameImproving performanceOptimization

The balance of value

! Creating features and fixing bugs adds value directly to the product! We can measure the success of these activities by

monitoring conversions and sales! Performance optimization is empirical

! We get hard data on whether we did it right or not! Structural change is difficult to quantify

! Impacts on features, bugs, and performance! Highly subjective

Why structural change matters

! Software growth is not constant! Bursts of activity; release and QA cycles! We can’t anticipate all future requirements

! We want change with minimum cost and effort! Requires code to be supple and adaptable

! We want predictable results from change! Requires effects of code changes to be localized! Requires loosely coupled, easily testable objects

Measuring Programming

! Creative activity: no statistical feedback to tell us whether we are reaching our goal in code

! Little or no correlation between time spent coding and quality of the result

! We can mine data from code syntax; version control history; bug trackers! But the same metrics have different meanings,

depending on the particular team and codebase

Looking at code on a larger scale

! Methods per class! God class

! Lines per method! Complex method

! Revisions per code file! Churn

! Bug fixes per code file! Instability

0

10

20

30

40

50

60

70

Metric

Statistical Similarities

! All large code bases seem to share common statistical features

! The shapes of many software metrics conform to power law or log-normal distributions

! AKA: 80/20 rule, Pareto Principle, ‘long tail effect’

The shape of software

! These statistical discoveries give us a new way of looking at widely held ideas about software engineering:! The ‘Big Ball of Mud’ is the wrong metaphor — even

messy legacy systems have a definite shape! The ‘Lego Hypothesis’ is invalid — object oriented

software is not constructed by connecting reusable, modular components

What we say versus what we do

! Traditionally, we describe effects like overly long methods and complex classes as ‘code smells’

! But these effects seem to be common to every large code base in existence

! As software engineers, we have to account for this in our methodology

Behavioral Economics

! Psychological explanation for why code tends towards certain shapes

! Stronger incentives for getting something working quickly than sacrificing time and mental effort to create new abstractions

! Easier to reason about code changes in one place! Naming things is one of the most difficult problems

in software development

Actor Networks

! Sociological explanation for why code tends towards certain shapes

! Conway’s Law: structure of systems tends to mirror that of the organization building the system

! The boundary of an application is an entirely social construction

! Rationale for coding decisions is lost when developers leave the team or new requirements overwhelm an existing system

Accidental Complexity

! A by-product of the technology used to solve the problem

! Cost of change increases with the volume of code! Code that is difficult to change

! Unavoidable! Indicator of code not evolving to fit changing

requirements! Indicator of a mismatch between chosen abstractions

and the shape of the problem domain

Granularity

! Choice of programming language matters! Translating a high-level concept into actual code

! Coarse grained APIs are inflexible and require heavy customization and workarounds

! Fine grained APIs are hard to use and require large amounts of wrapping code and sequential calls

! Architectural style:! Convention or Configuration! Library or Framework

How Buildings Learn

! Layers of a building grouped according to different rates of change:! STUFF! SPACE-PLAN! SERVICES! SKIN! STRUCTURE! SITE

How Applications Learn

! Layered architecture! Lower levels wrap infrastructure that changes less

frequently! Upper levels wrap business logic that changes

constantly! Domain driven design

! Minimize accidental complexity through a rigorous approach to language

! Look for implicit models hidden in existing code and flesh them out

Ubiquitous Language

! A living document of all the terms, concepts, and relationships that drive product features

! Bridging product level thinking and technical architecture

! Models behaviour from a software point of view! Encapsulates essential complexity! Changes and evolves in sync with the product

Relentless Refactoring

! Refine the model; improve the language! Discover new organizing concepts

! Focus on the relationships between components! Each object has a single responsibility! Small pieces, loosely joined

! Break apart entanglements! Extract new classes and methods from existing code! Extract new applications from existing code

Component Boundaries

Domain Model

Data Store

! Dependencies are transitive

! Lower level packages should never depend on higher level packages

Application Logic

Evolving Architecture Patterns

! Strangler App! Introduce new architectural abstractions that wrap the

existing system like a creeping vine that eventually overtakes the host tree

! Parallel Rewrite! New architecture develops side by side with the old

system within a single app! Extract App

! New applications and service layers are spawned off the original system