NGINX MRA AMA #7:

The Circuit Breaker Pattern

Wednesday, January 10th, 2018

1

MORE INFORMATION AT NGINX.COM

Who Are We?

Floyd SmithDirector, Content

Marketing

Formerly with Apple,

Alta Vista, Google,

startups, and an

author

Chris StetsonChief Architect & Sr.

Director, Microservices

Engineering

Formerly with Razorfish

and Huge

Charles PretzerTechnical Architect

Former software

architecture

consultant and

engineering lead

MORE INFORMATION AT

NGINX.COM

Agenda

1. Introducing NGINX and NGINX Plus

2. The Circuit Breaker pattern

3. Embracing Failure

4. Understand Your Failure Profile

5. NGINX Plus as the Circuit Breaker

6. Other Resiliency Options

7. Questions

3 3

MORE INFORMATION AT

NGINX.COM

1. Introducing NGINX and

NGINX Plus

MORE INFORMATION AT NGINX.COM

About NGINX, Inc.

• NGINX OSS released 2003

• NGINX Plus first released in 2013

• NGINX, Inc. is VC-backed by leading investors in enterprise

software

• Offices in SF, Sunnyvale, Singapore, Cork, Cambridge, & Moscow

• 1,200+ commercial customers

• 180+ employees

>50%of the top 100,000

busiest websites

6Source: W3Techs Web Technology Survey

>40%of sites on AWS

7Source: SumoLogic

8

Internet

Web ServerServe content from disk

Application GatewayFastCGI, uWSGI, Passenger…

Reverse ProxyCaching, load balancing…

HTTP traffic

Where NGINX Plus Fits

MORE INFORMATION AT NGINX.COM

Public/Private/Hybrid CloudBare Metal Containers

10

NGINX Plus Works in All Environments

MORE INFORMATION AT NGINX.COM

NGINX Products and Roadmap

• Start with Gus Robertson keynote at nginx.conf 2017

• See Owen Garrett’s Roadmap presentation….

• …and Chris and Rachael’s Controller demo

• Also, Chris Richardson series, Intro to Microservices…

• …and Chris Stetson series, NGINX MRA

• …and much more; contact Sales for free evaluation

MORE INFORMATION AT

NGINX.COM

2. The Circuit Breaker Pattern

MORE INFORMATION AT

NGINX.COM

Architectural

Changes:

Monolith to

Microservices 1

MORE INFORMATION AT

NGINX.COM

Architectural

Changes:

Monolith to

Microservices 2

MORE INFORMATION AT

NGINX.COM

Microservices

Reference

Architecture

• Docker containers

• Polyglot services

• Design incorporates

Twelve-Factor App

principles

MORE INFORMATION AT

NGINX.COM

Circuit Breaker

• Active health checks

• Avoid catastrophic

failure

• Greater resiliency

MORE INFORMATION AT

NGINX.COM

3. Embracing Failure

Your App

Will Fail18

MORE INFORMATION AT

NGINX.COM

Layer Cake

• Cloud VMs

• Docker containers

• Application libraries

• Dynamic networking

You Can’t

Prevent Failure

20

MORE INFORMATION AT

NGINX.COM

Resizer

• Compute & memory

intensive

• Interfaced through

uploader

MORE INFORMATION AT

NGINX.COM

Resizer

• Compute & memory

intensive

• Interfaced through

uploader

File uploaded

Image expanded in memory 4k x 3k

Rotated and/or flipped

Resized to 3 sizes

MORE INFORMATION AT

NGINX.COM

At high volume

• Docker image crashed

• Taking out the underlying

VM

• Causing cluster to

become unstable

MORE INFORMATION AT

NGINX.COM

Health Check

• Uploader interfaces with

resizer

• Routes through N+

• Health check prevents

failure

MORE INFORMATION AT

NGINX.COM

4. Understand Your Failure Profile

Understand

Your Service

26

MORE INFORMATION AT

NGINX.COM

Dimensions of

failure• Memory

• CPU

• Disk

Memory

CPU

Disk

MORE INFORMATION AT

NGINX.COM

Develop the

failure profile• Establish thresholds

• Know the triggers

• Understand the

metrics

MORE INFORMATION AT

NGINX.COM

5. NGINX Plus as the Circuit Breaker

MORE INFORMATION AT

NGINX.COM

Circuit

Breakers

• Active health checks

• Retry

• Caching

MORE INFORMATION AT

NGINX.COM

Active Health

Check

• Check regularly

• Don’t over do it

• Internal

location /health-check-resizer {

internal;

health_check uri=/health match=conditions fails=1 interval=3s;

proxy_pass https://resizer;

proxy_ssl_protocols TLSv1.2; proxy_http_version

1.1;

}

MORE INFORMATION AT

NGINX.COM

Match

• Get specific data match conditions {

status 200-399;

header Content-Type ~ "application/json";

body ~ '{

"deadlocks":{"healthy":true},

"Disk":{"healthy":true},

"Memory":{"healthy":true}

}';

}

MORE INFORMATION AT

NGINX.COM

Don’t kill a

recovering

service

• Use the slow start

feature to allow a

service to recover

upstream resizer {

server resizer slow_start=30s;

zone backend 64k;

least_time last_byte;

keepalive 300;

}

MORE INFORMATION AT

NGINX.COM

6. Other Resiliency Options

MORE INFORMATION AT

NGINX.COM

Moderated

Delivery

• Use rate limiting to

keep your services

from failing in the first

place

http {

# Moderated delivery limit_req_zone $server_addr

zone=moderateReqs:1m rate=100r/s;

# ...

server {

# ...

limit_req zone=moderateReqs burst=150; limit_req_status 503;

# ...

}

}

MORE INFORMATION AT

NGINX.COM

Caching to

mitigate failure

• Providing service

continuity

• Read-oriented

services• proxy_cache_use_

stale

MORE INFORMATION AT

NGINX.COM

Backup server

• Another service

continuity option

• Generic over customupstream personal-ad-server {

server personal-ad-server;

server generic-ad-server backup;

zone backend 64k;

least_time last_byte;

keepalive 300;

}

MORE INFORMATION AT

NGINX.COM

7. Q&A

Thank You

39