Upload
jon-benson
View
94
Download
2
Embed Size (px)
Citation preview
HASHICORP
Advantages of a Scheduler
Higher Resource Utilization
Decouple Work from Resources
Better Quality of Service
HASHICORP
Advantages of a Scheduler
Bin Packing
Over-Subscription
Job Queueing
Higher Resource Utilization
Decouple Work from Resources
Better Quality of Service
HASHICORP
Advantages of a Scheduler
Abstraction
API Contracts
Standardization
Higher Resource Utilization
Decouple Work from Resources
Better Quality of Service
HASHICORP
Advantages of a Scheduler
Priorities
Resource Isolation
Pre-emption
Higher Resource Utilization
Decouple Work from Resources
Better Quality of Service
HASHICORP
example.nomad
# Define our simple redis jobjob "redis" {
# Run only in us-east-1 datacenters = ["us-east-1"]
# Define the single redis task using Docker task "redis" { driver = "docker"
config { image = "redis:latest" }
resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10 dynamic_ports = ["redis"] } } }}
NomadHASHICORP
Multi-Datacenter
Multi-Region
Flexible Workloads
Job Priorities
Bin Packing
Large Scale
Operationally Simple
HASHICORP
• Multi-Datacenter
• Raft Consensus
• Large Scale
• Production Hardened
• Coordination (Locking)
• Central Servers + Distributed Clients
• Network Tomography
• Prepared Queries
HASHICORP
Prepared Queries
• Multiple instances of a given service exist in multiple
datacenters
• Clients can talk to any of them, and always prefer the
instances with lowest latency
• Policies can change, desire to not have the clients know
the details of how to locate a healthy service
HASHICORP
Prepared Queries
• New query namespace, similar to services
• Register queries to answer for parts of this namespace
• Clients use APIs, or “.query.consul” DNS lookups to run
queries
NomadHASHICORP
Inspired by Google Omega
Optimistic Concurrency
Internal State and Coordination
Service and Batch workloads
Pluggable Architecture
HASHICORP
Single Region Architecture
SERVER SERVER SERVER
CLIENT CLIENT CLIENTDC1 DC2 DC3
FOLLOWER LEADER FOLLOWER
REPLICATIONFORWARDING
REPLICATIONFORWARDING
RPC RPC RPC
HASHICORP
Multi Region Architecture
SERVER SERVER SERVERFOLLOWER LEADER FOLLOWER
REPLICATIONFORWARDING
REPLICATION
REGION B GOSSIP
REPLICATION REPLICATIONFORWARDING
REGION FORWARDING
REGION A
SERVERFOLLOWER
SERVER SERVERLEADER FOLLOWER
NomadHASHICORP
Region is Isolation Domain
1-N Datacenters Per Region
Flexibility to do 1:1 (Consul)
Scheduling Boundary
HASHICORP
External Event
Evalua?on Crea?on
Evalua?on Queuing
Evalua?on Processing
Op?mis?c Coordina?on
State Updates
HASHICORP
Server Architecture
Omega Class Scheduler
Pluggable Logic
Internal Coordination and State
Multi-Region / Multi-Datacenter
HASHICORP
Fingerprinting
Operating System
Hardware
Applications
Environment
Type Examples
Kernel, OS, Versions
CPU, Memory, Disk
Java, Docker, Consul
AWS, GCE
HASHICORP
Fingerprinting
“Task Requires Linux, Docker, and PCI-Compliant Hardware” expressed as Constraints
HASHICORP
Containerized
Virtualized
Standalone
Docker
Rocket
Windows Server Containers
Qemu / KVM
Hyper-V
Xen
Java Jar
Static Binaries
C#
NomadHASHICORP
• Million Container Challenge
• hashicorp.com/c1m.html
• github.com/hashicorp/c1m
• Nomad 0.4
• Volume support across drivers
• Advanced networking