Deepak Nadig

Distinguished Architect, Intuit

New York Life Build BlueMay 19, 2017

DesigningAPI Platforms that Developers Love

2

3

Consumers Small Businesses Accounting Professionals

Who we serve:

4

Era of the CloudEra of Windows Era of WebEra of DOS

2010+1980s 1990s 2000s

Proven track record: capitalizing on change

Intuit Founded

Quicken DOS

IPO

Quicken Window

s

QuickBooks DOS

TurboTax Online

QuickBooks Windows

TurboTax WindowsQuickBooks

MobileQuickBooks Online

QuickBooks Online Global

• Employees: 150• Customers: 1.3M• Revenue: $33M• INTU + 1,400%, NASDAQ + ~500%

• Employees: 4,500 • Customers: 5.6M • Revenue: $1.04B• INTU - 20%, NASDAQ - 40%

• Employees: 7,700• Customers: 50M• Revenue: $3.1B• INTU +160%, NASDAQ 90%

INTU

sh

are

pri

ceSnapTax

Competitor A

Competitor A Competitor C

Competitor ACompetitor A

Competitor B

Competitor D

Competitor E

5

# 1 Small Business Management. 7700 employees. $4.7 Billion in revenue.

QuickBooks has more than 5 million customers. Online is 1.5 million (41% growth YoY).

26K developers. 2500 third-party apps. 15% of customers use at least one third-party app.

Yearly, 100 million invoices; $1.5 trillion in commerce; payroll for more than 1 million employees (1/12 in US).

Small Business Owners spend 4 hours online every day.

QuickBooks is in 112 countries.

Intuit and QuickBooks by the numbers

6

The capability of rapidly and cost efficiently adapting to changes

- Wikipedia

The ability of an organization to sense environment change and

to respond efficiently and effectively to it

- Gartner

Challenge for businesses today = Agility

7

Architecture Business Cycle

to place them into a system-development context.

We have organized the set of forces into the following seven categories as follows:

1. stakeholder needs

2. business management issues

3. legal/contractual issues

4. commercial/competitive pressures

5. technical environment

6. political issues

7. life-cycle issues

The ABC RevisitedTo represent the ways in which a force may affect a design, we have adopted andadapted the Architecture Business Cycle (ABC) [Bass 03]. The ABC was originallyenvisioned as a means of depicting the influences on an architect and on how anarchitecture can eventually influence the forces that influenced the architecture, thuscreating a cycle. The original influences were the stakeholders and the developingorganization (who, together, fashioned the quality-attribute requirements), the technicalenvironment, and the architect’s experience.

In our adaptation of the ABC, the “influences” have been replaced by a set of forces thatcollectively shape the requirements and, over time, the architect’s experience. Theextended ABC is depicted in Figure 1.

Figure 1: Extending the Architecture Business Cycle to Include All Design Forces

Notice that this extended ABC shows the requirements as being categorized into threeoverlapping sets: quality-attribute requirements (e.g., the system should initialize itselfin 60 seconds), business requirements (e.g., version 1.0 must be available to ourresellers by August 1), and functional requirements (e.g., the system must providemonthly sales summaries, broken down by region). These requirements, along with thearchitect’s experience, are the inputs to the architect and to the architecture-designprocess.

Capturing a ForceIf we are to propose forces that designers and design methods can consider whendesigning a system, we should be able to describe these forces clearly. Our goal is toanswer this question:

What do we need to communicate about a force so that everyone will understand it?

We propose the following scheme:

1. stimulus source: who brings the force to bear?

2. stimulus: what is the force?

3. artifact stimulated: what does the force act on?

4. environment: when is the force brought to bear (e.g., during development)?

5. response: what does the stimulus source desire as an outcome?

Software Engineering Institute http://sei.cmu.edu/

Architecture Business Cycle has impact on Architecture, Business and Culture

8

• Architecture journey

• Business journey

• Culture journey

• Current à Target ABC

S-Curve

BusinessPerformance

Time

Kickoff

HighGrowth

LimitsReached

ABC

ABC

ABC

(*) ABC = Architecture Business Cycle

9

Flexibility and Extensibility

Flexibility

Ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed

Extensibility

Ease with which a system or component can be modified to increase its functional capacity

IEEE Standard Glossary of Software Engineering Technology

10

Agility is enabled by Flexibility and Extensibility

APIs enable flexibility

Microservices enable extensibility

Why APIs and Microservices?

11

Journey begins as one of the following …

UI and Business Logic

Web Browser

Web UI

Web Browser

Business Logic

Web UI

Web Browser

Business Logic

Mobile UI

Mobile

Web UI

Web Browser

Business Logic

Mobile UI

Mobile

We need microservices!Why are we still slow?

We need mobile!No time to refactorDevelop feature N times

Just want to tweak UI!How do we organize?

Yay! Growing fastOne language

Slowing releases

12

Target State

API Definition Internal or External Universal

API Discovery Painful Developer Portal

API Design Project specific API as a Product

Architecture Tightly coupled SOA Loosely coupled SOA

Technology Proprietary Standards based

Integration Expensive TTFHW* < 5 min

(*) TTFHW = Time To First Hello World

13

Portfolio of APIs

aligned by business capabilities,

realized by isolated and encapsulated services,

that can be used by internal and external developers

to develop applications and integrations

quickly and cost effectively

API Platform

14

API Platform Principles

API First

API Design

• Think API before experience• API is a product

• Externalize-able• Domain model & Integration scenarios

Developer Experience• Easy to learn, integrate, diagnose• Be the ‘developer’

API Quality Attributes• Response-time• Availability

Service Architecture• Encapsulated, Isolated• Craftsmanship

Best Practices

API as a productAPI designExperience vs Capability APIAPI specificationAPI documentationService granularityService developmentAPI testingAPI status & supportDeveloper Portal

16

End usersUnderstand Segments

Their ExperiencesAnd Expectations

DevelopersTools and processes

Their Integration needsBe Transparent on usage

API as a Product

17

API PortfolioDefine bounded contexts/capabilities

Organize cohesive conceptsUse orthogonality or distance to separate

API DesignUse problem space for domain model

Encompass all interactions – Events, Batch

Dog food API (and events)Share design guidelines (apistylebook.com)

API Design

id

id

/invoicing /payments

/risk

Domain Model à API• Entities, Attributes• Verbs• Relationships• State machine

18

Capability APIWhat API teams deliver

Optimized for API providerREST, Batch, …

Experience APIPersonalized for end user experience

Make it self-service for developer

Plan for multiple - GraphQL, Groovy

Experience vs Capability API

API Facade

Invoicing Payments Customer

Experience API

Capability API

19

Many choicesOAS*, JSON Schema, RAML

1 domain model, N experience APIsGenerate Docs, SDK, Tests

Maintain specification as codeGitHub, GitFlow (agile versioning)

Fail builds – enforce validation

Pull Request - customer review

API Specification

(*) Open API Specification

20

By developers for developersDevelopers contribute documentation

Technical writers play editor roleInclude in release process

Maintain documentation as codeGitHub, GitFlow (version w/specification)Fail builds – automated validation

Open it for community contributions

API Documentation

21

Why you should care?Extensibility

Granularity PrinciplesTeam boundary

SecurityRate of change

Scalability

Cohesion

Service Granularity

Credit: Spotify Engineering Culture

22

Getting to microservicesCode complexity vs. change

Pick the first, complete, repeatTeam autonomy, diagnosing issues

New microservicePlan for multiple languages

Plan for CI

Team (4) > Team (2) (45% more time)*

Service Development

Com

ple

xity

Low

High

High

Low

Least priority

Low priority

Impacting cycle time

Easier to decompose

Start here

Impacting cycle time most

Decompose,then refactor

Change(*) The team scaling fallacy: Underestimating the declining efficiency of larger teams

23

API TestingTest independently

Use API specification for automationComplete scenarios (with Webhooks)

SandboxSandbox = ProductionFor internal and external developers

Automated testing for external developers

API Testing

24

API StatusDevelopers need to know

Health dashboardReport status promptly on issues

Diagnosing issuesTreat developers alikeProvide API logs with messages

Be proactive using API analytics

API Status & Support

25

Complete developer experience

For 1st and 3rd party developers

Time To First Hello World < 5 min

Developer Portal

Culture

27

A way of thinking, behaving, or working

that exists in a place or organization

- Merriam-Webster

Behavior of humans within an organization and

the meaning that people attach to those behaviors.

- Wikipedia

What is culture?

28

Application developmentWeb page design Process

Site design/Portfolio ManagementEngineers work with page designs

API developmentAPI Design Process

API Portfolio Management

Engineers work with API designs

API Engineering Processes

App. ProductManager UED*

ApplicationEngineering

API ProductManager

API Designer

ServiceEngineering

(*) UED = User Experience Design

29

Be aware of Conway’s law

Namespace != Organization

Reuse = Coupling

Agreements based on standards

Teams evolve at different rates

Use maturity model to measure

Organization Evolution

How do committees invent?

30

Build something in 24 hours

Internal hackathons

Usability testing for APIs

Pride of ownership for API teams

Developer advocacy

Hackathons

31

• Agility is a business imperative

• Agility comes from flexible and extensible architectures

• APIs enable flexibility

• Microservices enable extensibility

• Architecture and culture need to evolve together

Summary

Thank You@deepak_nadig