Cloud Native Go - QCon New York · 2017-08-14 · Cloud Native Go Building Scalable, Resilient ......

6/28/17, 11)02 AMCloud Native Go

of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1

Cloud Native Go

Building Scalable, Resilient Microservices for the Cloudin Go

1 / 29

Agenda1. Introduction2. Boring Stuff3. Live Demos4. Q & A

2 / 29

Cloud Native Go

Building Scalable, Resilient Microservices for the Cloudin Go

1 / 29

Introduction

Wrote a few books (2 fantasy, 15+ .NET/C#, Go)Maker, Tinkerer, LinguistTaught people how migrate/build cloud native for PivotalLead Software Engineer for Capital OneTwitter: @KevinHoffman, github autodidaddict

Go to gopherize.megopherize.me for your Gopher Avatar

3 / 29

Agenda1. Introduction2. Boring Stuff3. Live Demos4. Q & A

2 / 29

Why Go?FastLow memory, CPU, and Disk footprintDocker images from SCRATCHFit more containers per node/VMSingle, no-dependency binaryBeautiful, simple languageShort learning curve

4 / 29

Introduction

Wrote a few books (2 fantasy, 15+ .NET/C#, Go)Maker, Tinkerer, LinguistTaught people how migrate/build cloud native for PivotalLead Software Engineer for Capital OneTwitter: @KevinHoffman, github autodidaddict

Go to gopherize.megopherize.me for your Gopher Avatar

3 / 29

What is Cloud Native?API FirstDependency ManagementDesign, Build, Release, RunConfig, Credentials, and CodeLogsDisposabilityBacking ServicesEnvironment ParityAdministrative ProcessesPort BindingStateless ProcessesConcurrencyTelemetryAuthentication and Authorization

5 / 29

Why Go?FastLow memory, CPU, and Disk footprintDocker images from SCRATCHFit more containers per node/VMSingle, no-dependency binaryBeautiful, simple languageShort learning curve

4 / 29

Microservices Shopping ListHTTP ServerRouting

URL path variablesQuery strings

JSON Encode/DecodeMiddleware (logging, security, etc)

6 / 29

What is Cloud Native?API FirstDependency ManagementDesign, Build, Release, RunConfig, Credentials, and CodeLogsDisposabilityBacking ServicesEnvironment ParityAdministrative ProcessesPort BindingStateless ProcessesConcurrencyTelemetryAuthentication and Authorization

5 / 29

HTTP Server package main

import (...)

func main() { port := os.Getenv("PORT") if len(port) == 0 { port = "8080" }

mux := http.NewServeMux() mux.HandleFunc("/", hello)

n := negroni.Classic() n.UseHandler(mux) hostString := fmt.Sprintf(":%s", port) n.Run(hostString)}

func hello(res http.ResponseWriter, req *http.Request) { fmt.Fprintln(res, "Hello from Go!")}

7 / 29

Microservices Shopping ListHTTP ServerRouting

URL path variablesQuery strings

JSON Encode/DecodeMiddleware (logging, security, etc)

6 / 29

Routing Routing with Gorilla Mux

r := mux.NewRouter()r.HandleFunc("/products/{key}", ProductHandler)r.HandleFunc("/articles/{category}/", ArticlesCategoryHandler)r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler)

s := r.PathPrefix("/zombies").Subrouter()s.HandleFunc("/", ZombiesHandler)s.HandleFunc("/sightings", NewSightingHandler).Methods("POST")s.HandleFunc("/{key}", QueryZombieHandler).Methods("GET")

/zombies/zombies/sightings (POST)/zombies/bob (GET)

8 / 29

HTTP Server package main

import (...)

func main() { port := os.Getenv("PORT") if len(port) == 0 { port = "8080" }

mux := http.NewServeMux() mux.HandleFunc("/", hello)

n := negroni.Classic() n.UseHandler(mux) hostString := fmt.Sprintf(":%s", port) n.Run(hostString)}

func hello(res http.ResponseWriter, req *http.Request) { fmt.Fprintln(res, "Hello from Go!")}

7 / 29

Routing Accessing Route Data

vars := mux.Vars(req)zombieID := vars["zombie-key"]

Building URLs

r := mux.NewRouter()r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler). Name("article")...url, err := r.Get("article").URL("category", "technology", "id", "42")

Builds - /articles/technology/42

9 / 29

Routing Routing with Gorilla Mux

r := mux.NewRouter()r.HandleFunc("/products/{key}", ProductHandler)r.HandleFunc("/articles/{category}/", ArticlesCategoryHandler)r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler)

s := r.PathPrefix("/zombies").Subrouter()s.HandleFunc("/", ZombiesHandler)s.HandleFunc("/sightings", NewSightingHandler).Methods("POST")s.HandleFunc("/{key}", QueryZombieHandler).Methods("GET")

/zombies/zombies/sightings (POST)/zombies/bob (GET)

8 / 29

JSON Marshaling Read JSON from Body:

payload, _ := ioutil.ReadAll(req.Body)var newMatchRequest newMatchRequesterr := json.Unmarshal(payload, &newMatchRequest)

Write JSON to Response:

var mr newMatchResponsemr.copyMatch(newMatch)w.Header().Add("Location", "/matches/"+newMatch.ID)formatter.JSON(w, http.StatusCreated, &mr)

10 / 29

Routing Accessing Route Data

vars := mux.Vars(req)zombieID := vars["zombie-key"]

Building URLs

r := mux.NewRouter()r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler). Name("article")...url, err := r.Get("article").URL("category", "technology", "id", "42")

Builds - /articles/technology/42

9 / 29

Middleware apiRouter := mux.NewRouter()apiRouter.HandleFunc("/api/post", apiPostHandler(formatter)).Methods("POST")

router.PathPrefix("/api").Handler(negroni.New( negroni.HandlerFunc(isAuthorized(formatter)), negroni.Wrap(apiRouter),))

func isAuthorized(formatter *render.Render) negroni.HandlerFunc { return func(w http.ResponseWriter, r *http.Request, next http.HandlerFunc) { providedKey := r.Header.Get(APIKey) if providedKey == "" { formatter.JSON(w, http.StatusUnauthorized, struct{ Error string }{"Unauthorized." } else if providedKey != apikey { formatter.JSON(w, http.StatusForbidden, struct{ Error string }{"Insufficient credentials." } else { next(w, r) } }}

11 / 29

JSON Marshaling Read JSON from Body:

payload, _ := ioutil.ReadAll(req.Body)var newMatchRequest newMatchRequesterr := json.Unmarshal(payload, &newMatchRequest)

Write JSON to Response:

var mr newMatchResponsemr.copyMatch(newMatch)w.Header().Add("Location", "/matches/"+newMatch.ID)formatter.JSON(w, http.StatusCreated, &mr)

10 / 29

Shopping List ✅ HTTP Server✅ Routing

✅ URL path variables✅ Query strings

✅ JSON Encode/Decode✅ Middleware (logging, security, etc)

12 / 29

Middleware apiRouter := mux.NewRouter()apiRouter.HandleFunc("/api/post", apiPostHandler(formatter)).Methods("POST")

router.PathPrefix("/api").Handler(negroni.New( negroni.HandlerFunc(isAuthorized(formatter)), negroni.Wrap(apiRouter),))

func isAuthorized(formatter *render.Render) negroni.HandlerFunc { return func(w http.ResponseWriter, r *http.Request, next http.HandlerFunc) { providedKey := r.Header.Get(APIKey) if providedKey == "" { formatter.JSON(w, http.StatusUnauthorized, struct{ Error string }{"Unauthorized." } else if providedKey != apikey { formatter.JSON(w, http.StatusForbidden, struct{ Error string }{"Insufficient credentials." } else { next(w, r) } }}

11 / 29

We're Doing This All Wrong

13 / 29

Shopping List ✅ HTTP Server✅ Routing

✅ URL path variables✅ Query strings

✅ JSON Encode/Decode✅ Middleware (logging, security, etc)

12 / 29

Do One Thing Great

A fox knows many things, but a hedgehog one important thing -Archilochus

14 / 29

We're Doing This All Wrong

13 / 29

Hexagonal ArchitecturePorts and Adapters

15 / 29

Do One Thing Great

A fox knows many things, but a hedgehog one important thing -Archilochus

14 / 29

Ports and Adapters in GoBuild your Core Functionality FirstDecouple HTTP, JSON, Headers, Security from Core FunctionShould be able to test your core function in isolationCode is blissfully ignorant of source and nature of inputs and ultimatedestination of replies

16 / 29

Hexagonal ArchitecturePorts and Adapters

15 / 29

Ports & Adapters in a μService Ecosystem

17 / 29

Ports and Adapters in GoBuild your Core Functionality FirstDecouple HTTP, JSON, Headers, Security from Core FunctionShould be able to test your core function in isolationCode is blissfully ignorant of source and nature of inputs and ultimatedestination of replies

16 / 29

Ports & Adapters in a μService EcosystemAPI Gateways

17 / 29

Ports & Adapters in a μService Ecosystem

17 / 29

Ports & Adapters in a μService EcosystemAPI GatewaysAggregator Services

17 / 29

Ports & Adapters in a μService EcosystemAPI Gateways

17 / 29

Ports & Adapters in a μService EcosystemAPI GatewaysAggregator ServicesMessage Brokers

TopicsPub/Sub

17 / 29

Ports & Adapters in a μService EcosystemAPI GatewaysAggregator Services

17 / 29

TopicsPub/Sub

ACL / Translators

17 / 29

TopicsPub/Sub

17 / 29

TopicsPub/Sub

ACL / TranslatorsDynamic Service Discovery

17 / 29

TopicsPub/Sub

ACL / Translators

17 / 29

TopicsPub/Sub

ACL / TranslatorsDynamic Service DiscoveryExternal Configuration

17 / 29

TopicsPub/Sub

ACL / TranslatorsDynamic Service Discovery

17 / 29

TopicsPub/Sub

ACL / TranslatorsDynamic Service DiscoveryExternal ConfigurationAdapter paths through system

Gateways, ProxiesTranslators on Data Streams / Queues

17 / 29

TopicsPub/Sub

ACL / TranslatorsDynamic Service DiscoveryExternal Configuration

17 / 29

TopicsPub/Sub

Hexagonal arch inside and outside of services

17 / 29

TopicsPub/Sub

17 / 29

The Shopping List We IgnoreSecurityMonitoringLog AggregationZero Downtime DeploymentBuild/Deploy AutomationTracingMetricsAuditAutomatic / Dynamic Horizontal ScalingConfigurationSecrets ManagementDiscovery

18 / 29

TopicsPub/Sub

Hexagonal arch inside and outside of services

17 / 29

Starting at the CoreDefine what we're buildingDefine why we're building itWho are we building it for?Build the coreAdd onion layers around the core

Ports and AdaptersRemember we're building an ecosystem, not hello world

19 / 29

The Shopping List We IgnoreSecurityMonitoringLog AggregationZero Downtime DeploymentBuild/Deploy AutomationTracingMetricsAuditAutomatic / Dynamic Horizontal ScalingConfigurationSecrets ManagementDiscovery

18 / 29

The Go Shopping Sample

20 / 29

Starting at the CoreDefine what we're buildingDefine why we're building itWho are we building it for?Build the coreAdd onion layers around the core

Ports and AdaptersRemember we're building an ecosystem, not hello world

19 / 29

Go Shopping Sample - Techgo micro Framework

https://micro.mu/Ports and Adapters are services

Not boilerplate inside serviceEcosystem-aware from bottom-upDynamic Service DiscoverygRPC TransportRESTful Aggregate API... and much more

21 / 29

The Go Shopping Sample

20 / 29

Framework vs LibraryOpinionated vs boilerplateSpend boilerplate to insulate from opinions

Mitigate ceremony with code generation?Go-kit - suite of libraries, incur BP costGo-micro - opinionated, less BP

Spring Boot - example of opinionated framework

22 / 29

Go Shopping Sample - Techgo micro Framework

https://micro.mu/Ports and Adapters are services

Not boilerplate inside serviceEcosystem-aware from bottom-upDynamic Service DiscoverygRPC TransportRESTful Aggregate API... and much more

21 / 29

API FirstAPI First is NOT:

RESTful Endpoint FirstSwagger-FirstOne API to Rule them All

API First is:

A strict, semantically versioned, interaction specification.A team and organization-wide disciplineDifferent consumers, channels may need different APIs

Ports and Adapters

23 / 29

Framework vs LibraryOpinionated vs boilerplateSpend boilerplate to insulate from opinions

Mitigate ceremony with code generation?Go-kit - suite of libraries, incur BP costGo-micro - opinionated, less BP

Spring Boot - example of opinionated framework

22 / 29

Defining the Warehouse Service APIProtocol Buffer IDLgRPC Service

syntax = "proto3";

package warehouse;

service Warehouse { rpc GetWarehouseDetails(DetailsRequest) returns (DetailsResponse);}

message DetailsRequest { string sku = 1;}

message DetailsResponse { WarehouseDetails details = 1;}

message WarehouseDetails { string sku = 1; uint32 stock_remaining = 2; string manufacturer = 3; string model_number = 4;}

24 / 29

API FirstAPI First is NOT:

RESTful Endpoint FirstSwagger-FirstOne API to Rule them All

API First is:

A strict, semantically versioned, interaction specification.A team and organization-wide disciplineDifferent consumers, channels may need different APIs

Ports and Adapters

23 / 29

Service Implementation

package service

import ( "github.com/autodidaddict/go-shopping/warehouse/proto" "golang.org/x/net/context")

type warehouseService struct{}

// NewWarehouseService returns an instance of a warehouse handlerfunc NewWarehouseService() warehouse.WarehouseHandler { return &warehouseService{}}

func (w *warehouseService) GetWarehouseDetails(ctx context.Context, request *warehouse.DetailsRequest, response *warehouse.DetailsResponse) error {

response.Manufacturer = "TOSHIBA" response.ModelNumber = "T-1000" response.SKU = request.SKU response.StockRemaining = 35 return nil}

25 / 29

Defining the Warehouse Service APIProtocol Buffer IDLgRPC Service

syntax = "proto3";

package warehouse;

service Warehouse { rpc GetWarehouseDetails(DetailsRequest) returns (DetailsResponse);}

message DetailsRequest { string sku = 1;}

message DetailsResponse { WarehouseDetails details = 1;}

message WarehouseDetails { string sku = 1; uint32 stock_remaining = 2; string manufacturer = 3; string model_number = 4;}

24 / 29

Let's do some Live DemosWhat could possibly go wrong?

26 / 29

Service Implementation

package service

import ( "github.com/autodidaddict/go-shopping/warehouse/proto" "golang.org/x/net/context")

type warehouseService struct{}

// NewWarehouseService returns an instance of a warehouse handlerfunc NewWarehouseService() warehouse.WarehouseHandler { return &warehouseService{}}

func (w *warehouseService) GetWarehouseDetails(ctx context.Context, request *warehouse.DetailsRequest, response *warehouse.DetailsResponse) error {

response.Manufacturer = "TOSHIBA" response.ModelNumber = "T-1000" response.SKU = request.SKU response.StockRemaining = 35 return nil}

25 / 29

Lessons LearnedQuestion EverythingBeware of Protobuf Default ValuesService-first developmentCode is the easiest and simplest part of Microservices

NFRs are everything - logging, metrics, alerts, discovery, health,autoscale, async messaging, etc.

Decide where you want to be on the opinion vs boilerplate curveGo is an ideal microservices development language

27 / 29

Let's do some Live DemosWhat could possibly go wrong?

26 / 29

Next Steps Create ServicesTest everythingAutomate everythingDeploy all the timeTry out go-microTry out go-kitGo Shopping http://github.com/autodidaddict/go-shoppingThese slides https://github.com/autodidaddict/cng-presentation

28 / 29

Lessons LearnedQuestion EverythingBeware of Protobuf Default ValuesService-first developmentCode is the easiest and simplest part of Microservices

NFRs are everything - logging, metrics, alerts, discovery, health,autoscale, async messaging, etc.

Decide where you want to be on the opinion vs boilerplate curveGo is an ideal microservices development language

27 / 29

29 / 29

Next Steps Create ServicesTest everythingAutomate everythingDeploy all the timeTry out go-microTry out go-kitGo Shopping http://github.com/autodidaddict/go-shoppingThese slides https://github.com/autodidaddict/cng-presentation

28 / 29

29 / 29

Cloud Native Go - QCon New York · 2017-08-14 · Cloud Native Go Building Scalable, Resilient ......

Documents

CNA — Cloud-native applications · TSM-ClComp-EN Cloud Computing | Cloud-native applications | Academic year 2017/2018 Characteristics of CNA A cloud-native application is an application

Cloud Native Go - QCon New York · PDF fileStateless Processes Concurrency Telemetry Authentication and Authorization 5 / 29 Why Go? Fast Low memory, CPU, and Disk footprint ... Cloud

Cloud Native Java with Spring Cloud Services

Starting Cloud Native

Cloud-Native Workshop NYC - The Cloud-Native Landscape

Migrating to Cloud-Native Application Architecturesdownload3.vmware.com/vmworld/2015/downloads/oreilly-cloud-native... · Why Cloud-Native Application Architectures? First we’ll

Cloud Native Networking with FD.io/VPPbos.itdks.com/23e6f3d8cfbf4316a1ec5177ac087b0d.pdf• Cloud-Native Infrastructure –a Golang platform for building cloud-native microservices

Enterprise Cloud Native

for Cloud Native Applications SOLID Principles€¦ · Cloud Native Cloud Native Scope Cloud Native is structuring teams, culture and technology to utilize automation and architectures

Cloud-native Data

Cloud Native Apps & Continuous Delivery · Cloud Native Apps & Continuous Delivery. THE OPEN SOURCE INTEGRATOR FOR ENTERPRISE Cloud Native Apps (1) ... Cloud Controller Router DEA

Eseguire Applicazioni Cloud-Native con Pivotal Cloud Foundry su Google Cloud Platform (Pivotal Cloud-Native Workshop: Milan)

Cloud-Native Application Design - ZHAW Blogs · Cloud-Native Application What is a Cloud-Native Application? Application optimized to run in the cloud. Takes advantage and considers

) Cloud Native Installation Guide - Oracle · Cloud Native Installation Guide Release 1.0 F12352-01 June 2019. Oracle Communications Service Communication Proxy (SCP) Cloud Native

SED316 - Cloud Native Computingsoftwareengineeringdaily.com/.../2017/04/SED316-Cloud-Native-Com… · Then, there’s this notion of a cloud native stack that we’ve been delving

for Cloud Native Applications SOLID Principles...for Cloud Native Applications Bilgin Ibryam Principal Architect 10 May 2018 Cloud Native Cloud Native Scope Cloud Native is structuring

Cloud Native Future

Cloud Native Systems

cloud-native-integration - USbiztalk360.blob.core.windows.net/...Seroter__Cloud_Native_Integratio… · #3 Delivering cloud-native integration. @rseroter What is “cloud-native”

Cloud Native Microservices and Orchestrationbos.itdks.com/78645ac0090f441e9294fe5f19020127.pdf · Topics Cloud Native Microservices Orchestration - Spring Cloud Data Flow Continuous