Upload
rauel-perez
View
194
Download
2
Embed Size (px)
Citation preview
NATS in actionA Real time Microservices Architecture handled by NATS
1
Who am I?Raül PérezSenior Engineer at Carrenza Ltd.
Twitter: @repejotaGithub: https://github.com/repejotaEmail: [email protected]
2
Who am I?Raül PérezSenior Engineer at Carrenza Ltd.
Twitter: @repejotaGithub: https://github.com/repejotaEmail: [email protected]
3
● Almost 4 years working on devop & distributed projects.
Who am I?Raül PérezSenior Engineer at Carrenza Ltd.
Twitter: @repejotaGithub: https://github.com/repejotaEmail: [email protected]
4
● Almost 4 years working on devop & distributed projects.● Still more a dev than op dude.
Who am I?Raül PérezSenior Engineer at Carrenza Ltd.
Twitter: @repejotaGithub: https://github.com/repejotaEmail: [email protected]
5
● Almost 4 years working on devop & distributed projects.● Still more a dev than op dude.● Proud to be a Gopher after a looong PHP/Ruby/Python/Node.js past
experience. :P
Who am I?Raül PérezSenior Engineer at Carrenza Ltd.
Twitter: @repejotaGithub: https://github.com/repejotaEmail: [email protected]
6
● Almost 4 years working on devop & distributed projects.● Still more a dev than op dude.● Proud to be a Gopher after a looong PHP/Ruby/Python/Node.js past
experience. :P● Based in Barcelona.
Who am I?Raül PérezSenior Engineer at Carrenza Ltd.
Twitter: @repejotaGithub: https://github.com/repejotaEmail: [email protected]
7
● Almost 4 years working on devop & distributed projects.● Still more a dev than op dude.● Proud to be a Gopher after a looong PHP/Ruby/Python/Node.js past
experience. :P● Based in Barcelona.● Love startups & love remote work!
Microservices
8
Microservices● Small autonomous services that work together.
9
● Divide and conquer approach.
Microservices● Small autonomous services that work together.● Communicate with lightweight mechanisms.
10
● Divide and conquer approach.● HTTP/REST microservices. ( Request / Response )
Microservices● Small autonomous services that work together.● Communicate with lightweight mechanisms.
11
● Divide and conquer approach.● HTTP/REST microservices. ( Request / Response )● Message Systems for asynchronous communication. ( Publish/Subscribe )
Microservices● Small autonomous services that work together.● Communicate with lightweight mechanisms.● The platform is composed of one or more stateless processes.
12
● Divide and conquer approach.● HTTP/REST microservices. ( Request / Response )● Message Systems for asynchronous communication. ( Publish/Subscribe )● There is no shared data between different processes.
Microservices● Small autonomous services that work together.● Communicate with lightweight mechanisms.● The platform is composed of one or more stateless processes.● Designed to be distributed.
13
● Divide and conquer approach.● HTTP/REST microservices. ( Request / Response )● Message Systems for asynchronous communication. ( Publish/Subscribe )● There is no shared data between different processes.● Scaling is easy, each microservice can be executed in a different server.
Microservices … Why?● It is quick to (re) write and to deploy.
14
● Simple application can be handled by less developers.
Microservices … Why?● It is quick to (re) write and to deploy.
15
● Simple application can be handled by less developers.● Developers do not need to know the entire platform to be productive.
Microservices … Why?● It is quick to (re) write and to deploy.● It knows that failures exist and how to handle them.
16
● Simple application can be handled by less developers.● Developer do not need to know the entire platform to be productive.● Defensive programming, services should not break.
Microservices … Why?● It is quick to (re) write and to deploy.● It knows that failures exist and how to handle them.● KISS: Keep It Simple and Stupid.
17
● Simple application can be handled by less developers.● Developer do not need to know the entire platform to be productive.● Defensive programming, services should not break.● It must provide one functionality.
Microservices … Why?● It is quick to (re) write and to deploy.● It knows that failures exist and how to handle them.● KISS: Keep It Simple and Stupid.● It must provide an API that other services can use.
18
● Simple application can be handled by less developers.● Developer do not need to know the entire platform to be productive.● Defensive programming, services should not break.● It must provide one functionality.● Rely on best practices.
NATS
19
What is NATS?● Cloud-native message platform.
20
● Designed to natively support modern cloud architectures.
What is NATS?● Cloud-native message platform.● Highly performant.
21
● Designed to natively support modern cloud architectures.● It is fast! Can handle millions of messages per second.
What is NATS?● Cloud-native message platform.● Highly performant.● Extremely lightweight.
22
● Designed to natively support modern cloud architectures.● It is fast! Can handle millions of messages per second.● A single binary, easy to deploy without unnecessary complexity.
What is NATS?● Cloud-native message platform.● Highly performant.● Extremely lightweight.● Support for various messaging models and use cases.
23
● Designed to natively support modern cloud architectures.● It is fast! Can handle millions of messages per second.● A single binary, easy to deploy without unnecessary complexity.● Request/Response, Publish/Subscribe, Queues.
What is NATS?● Cloud-native message platform.● Highly performant.● Extremely lightweight.● Support for various messaging models and use cases.
24
● Designed to natively support modern cloud architectures.● It is fast! Can handle millions of messages per second.● A single binary, easy to deploy without unnecessary complexity.● Request/Response, Publish/Subscribe, Queues.● Simple text based protocol.
ErnestIAAS+PAAS Hybrid Cloud
Platform
25
Ernest : IAAS+PAAS Hybrid Cloud Platform● Manage your infrastructure. (IAAS)
26
● Manage servers, networks, balancers etc ... of your platform.
Ernest : IAAS+PAAS Hybrid Cloud Platform● Manage your infrastructure. (IAAS)● Manage your platform. (PAAS)
27
● Manage servers, networks, balancers etc ... of your platform.● Deploy and upgrade your application to the desired servers.
Ernest : IAAS+PAAS Hybrid Cloud Platform● Manage your infrastructure. (IAAS)● Manage your platform. (PAAS)● Scaling.
28
● Manage servers, networks, balancers etc ... of your platform.● Deploy and upgrade your application to the desired servers.● Automate everything.
Ernest : IAAS+PAAS Hybrid Cloud Platform● Manage your infrastructure. (IAAS)● Manage your platform. (PAAS)● Scaling.
29
● Manage servers, networks, balancers etc ... of your platform.● Deploy and upgrade your application to the desired servers.● Automate everything.● Evolve your platform when you need and redeploy your platform.
Ernest : IAAS+PAAS Hybrid Cloud Platform● Manage your infrastructure. (IAAS)● Manage your platform. (PAAS)● Scaling.● Hybrid (Multiple Providers)
30
● Manage servers, networks, balancers etc ... of your platform.● Deploy and upgrade your application to the desired servers.● Evolve your infrastructure when you need and redeploy your platform.● Automate everything.● Do not rely on a single provider, use different ones with a single interface.
Architecture of Ernest
31
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
PublicAPI
&
UI
Monitor
Provider
Architecture of Ernest
32
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
PublicAPI
&
UI
Monitor
Provider
REST API
Architecture of Ernest
33
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
PublicAPI
&
UI
Monitor
Provider
REST API
Events + State
Architecture of Ernest
34
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
PublicAPI
&
UI
Monitor
Provider
REST API
Events + State
Business LogicReal Time (almost)
Architecture of Ernest
35
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
PublicAPI
&
UI
Monitor
Provider
REST API
Events + State
Business LogicReal Time (almost)
VMWare
Architecture of Ernest
36
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
PublicAPI
&
UI
Monitor
Provider
REST API
Events + State
Business LogicReal Time (almost)
VMWare
AWS
Architecture of Ernest
37
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
PublicAPI
&
UI
Monitor
Provider
REST API
Events + State
Business LogicReal Time (almost)
VMWare
AWS
Others...
Ernest + NATS
38
Ernest + NATS● NATS is the central communication system for Ernest.
39
● Ernest core is a series of microservices talking each other through NATS.
Ernest + NATS● NATS is the central communication system for Ernest.● Ernest executes workflows defined by the user to build environments.
40
● Ernest core is a series of microservices talking each other through NATS.● Users posts a workflow written in a YAML file to be processed.
Workflow example
41
---name: my-awesome-examplenetworks: - name: network-web subnet: 10.2.0.0/24instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run
● Simple example.
Workflow example
42
---name: my-awesome-examplenetworks: - name: network-web subnet: 10.2.0.0/24instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run
● Simple example.● Each environment has an unique name.
Workflow example
43
---name: my-awesome-examplenetworks: - name: network-web subnet: 10.2.0.0/24instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run
● Simple example.● Each environment has an unique name.● Action: Create a network on our provider.
Workflow example
44
---name: my-awesome-examplenetworks: - name: network-web subnet: 10.2.0.0/24instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run
● Simple example.● Each environment has an unique name.● Action: Create a network on our provider.● Action: Create an instance type for our app.
Workflow example
45
---name: my-awesome-examplenetworks: - name: network-web subnet: 10.2.0.0/24instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run
● Simple example.● Each environment has an unique name.● Action: Create a network on our provider.● Action: Create an instance type for our app.● Count determines the number of servers.
Workflow example
46
---name: my-awesome-examplenetworks: - name: network-web subnet: 10.2.0.0/24instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run
● Simple example.● Each environment has an unique name.● Action: Create a network on our provider.● Action: Create an instance type for our app.● Count determines the number of servers.● Action: Deploy your application and run it.
Ernest + NATS● NATS is the central communication system for Ernest.● A workflow composed by multiple actions is executed by the user.● A finite state machine handles the state for the platform.
47
● Ernest core is a series of microservices talking each other through NATS.● Users posts a workflow written in a YAML file to be processed.● A list of actions are executed, one per microservice.
Ernest + NATS● NATS is the central communication system for Ernest.● A workflow composed by multiple actions is executed by the user.● A finite state machine handles the state for the platform.● Publish Subscribe is the main pattern used.
48
● Ernest core is a series of microservices talking each other through NATS.● Users posts a workflow written in a YAML file to be processed.● A list of actions are executed, one per microservice.● Actions are published to NATS ( by a topic ) and microservices are
subscribed to execute them.
Ernest : Numbers● Less than 5 developers.
49
● A developer “owns” a component / microservice.
Ernest : Numbers● Less than 5 developers.● Started as a prototype in Ruby, moving almost everything to Go.
50
● A developer “owns” a component / microservice.● Fast to develop, less errors, less dependencies.
Ernest : Numbers● Less than 5 developers.● Started as a prototype in Ruby, moving almost everything to Go.● Almost 50 microservices.
51
● A developer “owns” a component / microservice.● Fast to develop, less errors, less dependencies.● Unix philosophy, do one thing and do it well done.
Ernest : Numbers● Less than 5 developers.● Started as a prototype in Ruby, moving almost everything to Go.● Almost 50 microservices.● Everything specified. Everything tested.
52
● A developer “owns” a component / microservice.● Fast to develop, less errors, less dependencies.● Unix philosophy, do one thing and do it well done.● Using TDD on each part of the platform.
Ernest : Numbers● Less than 5 developers.● Started as a prototype in Ruby, moving almost everything to Go.● Almost 50 microservices.● Everything specified. Everything tested.
53
● A developer “owns” a component / microservice.● Fast to develop, less errors, less dependencies.● Unix philosophy, do one thing and do it well done.● Using TDD on each part of the platform.● Documentation on APIs, documentation on payloads.
Resources● http://microservices.io● https://en.wikipedia.org/wiki/Cloud_computing ● http://nats.io
54
● http://carrenza.com● http://apcera.com
Questions?
55
Thank you!
56