Embed Size (px)
Citation preview
Ironfan: Principle-DrivenScalable Ops Tools
http://github.com/infochimps-labs/ironfan
Ironfan
• Open-source core (Apache) with enterprise features
• http://github.com/infochimps-labs/ironfan
• Based on Chef (http://www.opscode.com/chef)
Principles
• Progressive - match what you’d whiteboard
• Declaratory - show all (and only) what’s interesting
• Coherent - full state is versioned as a whole
• Decoupled - minimize shared knowledge
• Helpful - common things are simple
• Inevitable - declarations imply actions
Ironfan:your system diagram
made real
Declaratory:simple stories, simply told
• HBase Master• HBase RegionServers
Simple, right?
HBase Cluster
• one-ish:• HBase master• HBase secondary master• Hadoop Namenode• Hadoop 2NN• Hadoop Jobtracker• Zookeeper Quorum x3
• many:• HBase RegionServer• HBase Stargate• HBase Thrift• Hadoop Datanode• Hadoop Tasktracker
HBase Cluster
Powerful Black Boxes
Beautiful Glue
Decouple
• HBase
• ElasticSearch
• Cassandra
• TokyoTyrant
• SimpleDB
• MongoDB
• MySQL
• Redis
• sqlite
• whisper (graphite)
• file system
• S3
Data Stores in Production
• Emacs
• Keynote
• Preview
• Chrome
• ruby (pry)
• Skitch
• finder
• flickr.com
• google image search
• ssh
Programs Used for This Talk
A Database is just a Bucket
Service Discovery:game recognize game
gibbon-nnjt-0
gibbon-worker-0
gibbon-hadoop-namenodegibbon-hadoop-jobtracker
¿dónde está mi namenode?
gibbon-nnjt-0
ack ack ack
gibbon-nnjt-0
gibbon-worker-0
¿dónde está mi namenode?
gibbon-nnjt-0
¿dónde está mi jobtracker?
gibbon-nnjt-0
gibbon-hadoop-namenodegibbon-hadoop-jobtracker
ack ack ack
gibbon-nn-0
gibbon-worker-0
¿dónde está mi namenode?
gibbon-nn-0
¿dónde está mi jobtracker?
gibbon-jt-0
ack ack ack
gibbon-jt-0
gibbon-hadoop-namenode
gibbon-hadoop-jobtracker
declaratory + decoupled =Recapitulation
ex: DB Migrationsthe Ironfan Way
• HBase Master• HBase RegionServers
alice
alice
bob
alice
bob
bob
Aspects:Cross-cutting Concerns
Amenities
• Logs
• Ports
• Daemons
• Dashboards
• Libs / Executables
Amenities:It’s good to be king
Amenities
• Logs: rotation, archiving
• Ports: remote monitoring (on and off)
• Daemons: process monitoring
• Dashboards: Dashboard dashboard
• Libs / Executables: Decoupled imports
• Versions: Shims, Discovery
Inevitability:born this way
Ironfan:
Chef:
10 lines
10 lines, 5 flaws
can you spot all 5?
hard coded
fails on bsd
not DRY
SECURITY HOLEdaemon-writeable
conf dir
10 lines, 5 flaws
‘0755’ not 755
Ironfan
Inevitabilitymust do this:
to get a log dir
gibbon-hadoop-namenode
so when you announce,
AWESOMENESS IS INEVITABLE
Contract:word is bond yo
Resource Sharing:can’t we all just get along?
• Region Server (dominate)• TaskTracker + Children (wussy)• DataNode (moderate)
• DataNode (moderate)• TaskTracker + Children (dominate)
HBase
Hadoop
Rails Apps > 5k LOCconsidered Harmful
one big app many apps tax
Resource Sharing:can’t we all just get along?
Monolithic Software
means Meetings
Meetings
are Death
Decentralize. Decouple.
Fast
Resilient
Simple
It’s not about Performance
We’re Hiring
github.com/ infochimps-labs
...
• System Diagram made real• Decoupled Components connect• Components Announce their capabilities• Announcements enable Service Discovery• Components announce cross-cutting Aspects• Aspects enable zero-conf Amenities• Announcement as Contract• Contracts enable zero-conf specification testing• Specs + monitoring enable zero-conf integration
testing• Systems Bind to provisioned resources• Resource Sharing
