Upload
ruslan-meshenberg
View
104.832
Download
0
Embed Size (px)
Citation preview
Lightning talksSeason 1, Episode 2
March 13, 2013
● Lightning talks: 7:00 - 7:20
● NeflixOSS Roadmap: 7:20 - 7:30
● Announcement: 7:30 - 7:45
● Demo Stations and Q+A: 8:00 - 9:30
Overview
● The nucleus of a Composable Web Service.
● Container Agnostic with extensions for frameworks/containers.
Karyon https://github.com/Netflix/karyon
● Bootstrapping○ Dependency & Lifecycle management via
Governator.○ Service registry as Eureka.○ Property management via Archaius.
Karyon https://github.com/Netflix/karyon
● Administration (Auto-Discovered Embedded console)○ Environment○ Eureka○ JMX
Karyon https://github.com/Netflix/karyon
Runtime (Coming Soon)● SLA● Throttling● Request tracing and more ....
Karyon https://github.com/Netflix/karyon
● Sample Service using Karyon available as "Hello-netflix-oss" on github
● Recipes ...
Karyon https://github.com/Netflix/karyon
Netflix OSS Recipe: RSS ReaderFirst sample application that demonstrates how to build scalable edge and middle tier application using the Netflix OSS components.
Accelerate your learning curve !
Karyon
Hystrix
Servo Turbine
Eureka
Ribbon
Archaius
First sample application that demonstrates how to build scalable edge and middle tier application using the Netflix OSS components.
Accelerate your learning curve !
Netflix OSS Recipe: RSS Reader
https://github.com/Netflix/recipes-rss
Netflix OSS Recipe: RSS Reader
● Ephemeral Volatile Cache
○ Based on memcached/spymemcached
● Highly Scalable ○ Over 2 MM RPS at peak within Netflix○ Over 100 K RPS / instance
● Fast○ Average latency ~ 1 ms
● Elastic○ Dynamically add/remove clusters, zones and/or instances
● Heavily Used @Netflix○ Fronts services & stores (Cassandra, S3, SimpleDB, RDS)
● AWS Zone Aware○ Clusters by zone○ Replication by zones ○ Eliminate cross zone traffic
● Connection Pool○ Separate pools for Read and Write Operations
● Zone fall through
● Resilient to zone outage
● Upcoming OSS○ EVCache Server, Admin and Monitoring
https://github.com/Netflix/EVCache
Denominator
● Portable Control of DNS clouds○ DynECT, Route53, UltraDNS and mock providers
● Java Api and CLI○ idempotent commands against record sets
● GSLB Functionality in progress○ Load Balanced and Directional (GEO) record sets
github.com/Netflix/denominator
Aminator
Aminator
● Evolved from Bakery
● AMI creation tool○ CLI○ EBS AMIs○ Redhat/Debian OSs○ Python 2.6 & 2.7○ boto
● What● Why● How● When● Who
Aminator
● Experiment with other distros at scale, like Ubuntu
● Remove embedded customizations
● You asked for it!
● What● Why● How● When● Who
Aminator
● Customize existing AMI○ Create volume from base AMI
snapshot○ Attach volume○ provision package○ snapshot volume○ register snapshot
● What● Why● How● When● Who
Aminator
● Coming soon, really.● What● Why● How● When● Who
github.com/Netflix/aminator
Aminator
● Brian Moyles○ @bmoyles
● Karate Vick○ @kvick
● Michael Tripoli○ @mtripoli
● What● Why● How● When● Who
NetflixGraph
● Compact in-memory data structure used to represent generic data sets.
● Reduce the size of your application's memory footprint○ Potentially by an order of magnitude
● Remove I/O bottlenecks by holding data in RAM
● Simple, easy to use API● Built-in serialization helps to transmit your
data sets across your infrastructure
NetflixGraph - Memory Reduction
● Drastically reduce the size of your data sets in memory○ Eliminate Access Latency - Pull more into RAM○ Reduce Cost - Use Less Hardware○ Stabilize your application
NetflixGraph - Memory/CPU tradeoff
● Easily select and test your memory-CPU tradeoff for different parts of your data sets.
● Choose between tighter memory or faster access times on a per-type basis.○ Mix and match, or just use the defaults!
NetflixGraph - Memory/CPU tradeoff
● Tighter memory requirements? Use compact sets.○ Variable-byte integers encode connections between
nodes
Wikipedia: Variable-Length Quantity
NetflixGraph - Memory/CPU tradeoff
● Faster access times required? Use hashed sets!○ Still RAM-efficient, provides O(1) access for contains() operations.
NetflixGraph - Built-in Serialization
● Transmit data sets across your infrastructure● Build the data set on one server, then distribute.
http://netflix.github.com/netflix-graph
Netflix OSS Continuous Integration
● Make current state of code visible to all● Build & test on public Jenkins, hosted by our
friends at CloudBees ● Each push or merge to master is built and
status shown on the GitHub project page
Netflix OSS Continuous Integration
Netflix OSS Continuous Integration
● Build & test all pull requests● Isolated environment to protect against
malicious code● Jenkins adds a comment to the pull request
Netflix OSS Continuous Integration● One-click creation of build jobs for new
projects
https://netflixoss.ci.cloudbees.comhttp://bit.ly/jobdsl
…