Upload
aater-suleman
View
132
Download
1
Tags:
Embed Size (px)
DESCRIPTION
This the talk I gave at Docker Docker Docker Austin Cloud Users meetup in Austin on January 21st, 2014. The talk was about the use of Docker at Flux7 Labs (flux7.com). I chose to deep dive into one of the most interesting Internet of Things use cases: Implement multi-tenancy on Solar Panel Monitoring Solution using Docker.
Citation preview
DOCKER @ FLUX7 LABS
DOCKER AUSTIN
FOR MORE DISCUSSIONS ON DOCKER!
Who am I?
● Aater Suleman○ Part-time UT Professor ○ Geek, Architect, Developer, Ops, DevOps …
● Co-founder & CEO Flux7 Labs ○ DevOps Solutions
■ Deployments■ Cost/performance optimized large scale
website (Ruby on rails, node.js, Django) and Hadoop deployments
VyScale Dev Flow
Docker's impact on performance (whitepaper WIP)
Multi-tenancy
Live process migration using CRIU (criu.org)
Four projects:
Application: Single Service Provider
Receive Sensor Data
Report Generation
based on data
Report sent to End User
Internet of Things -- Solar Panel Monitoring
XML Data over TCP
Big Data Analytics
Single Provider System
Provider Span
Location1 Location2Location3
Gateway1 Gateway2
Gateway3sensors
A provider has Mifi routers installed at multiple Locations which collect data from sensors and sends it to a remote TCP server via the internet.
TCP serverPort 6000
Cassandra port 9160
Flask AppPort 80
BrowserUses the Flask app at port 80Internet
COMPONENTS
1. Cassandra for data persistence which we later use for generating reports for each gateway.
2. A Twisted TCP server listening at PORT 6000, for data ingestion from multiple gateways owned by the provider.
3. A Flask app serving at PORT 80 as the admin panel for setting customizations and viewing reports.
GG G
G
Customer NCustomer 2 …Customer 1 Each customer can have multiple gateways
commissioned to them.
Remote Twister TCP Server (Non–Blocking I/O)
Cassandra NoSQL data store(High Volume High Velocity Write which scales Linearly across the
cluster )
Power consumption status on website and
mails
*G - Gateway
Web App
Mailer
SINGLE PROVIDER LAUNCH
For launching the single provider version, the following was done:
1. nohup python tcp_server.py & # For firing up the TCP server.
2. nohup python flask_app.py & # For firing up the admin panel
Both these code bases houses hard-coded Cassandra KEYSPACE
Success!
Application: Multiple Service Providers
…
Provider 1 sends data to port 6001
and accesses flask app at port 8081
Provider 2 sends data to port 6002 and accesses flask app at
port 8082
Flask container-runs flask app at port 80. Exposes port 80 and
published it to port 8081 for provider 1
Flask container-runs flask app at port 80. Exposes
port 80 and published it to port 8082 for provider 2
TCP server container-runs at port 6000.
Exposes port 6000 and published it to port 6001
for provider 1
TCP server container-runs at port 6000. Exposes
port 6000 and published it to port 6002 for provider 2
Cassandra
Internet
KNEE-JERK APPROACH
Sprinkle Tenant ID everywhere in the code and DB
Time consuming
Expensive
Poor isolation
Security
Maintenance
Rigidity
An alternate solution is to use Virtual Machine (VM)
Hosts are expensive ($)VMs are expensive (high overhead)
MULTIPLE HOST/VMS
AND THE SOLUTION
How: Isolated environments for running multiple instances of the app
WHY DOCKER?
Docker containers provide isolation that is
Fast
Inexpensive
Create a docker container for the new version of the app
PLAN
Setup environments/dependencies correctly
Start a Cassandra container.
# start a docker container for consuming gateway data at gateway_portstart_command = 'python software/remote_server.py ' + provider_idremote_server = docker_client.create_container('flux7/labs', # docker imagecommand=start_command, # start command contains the keyspace parameter, keyspace is the provider_idname='remote_server_' + provider_id, # name the container, name is provider_idports=[(6000, 'tcp'),]) # open port for binding, remote_server.py listens at 6000docker_client.start(remote_server, port_bindings={6000: ('0.0.0.0', gateway_port)}, links={'db': 'cassandra'})
AUTOMATION
# start a docker container for serving admin panel at admin_portstart_command = 'python software/flask_app.py ' + provider_idremote_server = docker_client.create_container('flux7/labs', # docker imagecommand=start_command, # start command contains the keyspace parameter, keyspace is the provider_idname='admin_panel_' + provider_id, # name the container, name is provider_idports=[(80, 'tcp'),]) # open port for binding, remote_server.py listens at 6000docker_client.start(remote_server, port_bindings={80: ('0.0.0.0',admin_port)}, links={'db': 'cassandra'})
An automation was the next foreseeable step, and for that we found Docker-py extremely useful. We used something like:
# Yes. We love Python!def start_provider(provider_id, gateway_port, admin_port ):docker_client = docker.Client(base_url='unix://var/run/docker.sock', version='1.6', timeout=100)
For now, a locally running container serving at PORT 9160 using the command
similar to this:
docker run -d -p 9160:9160 -name db flux7/cassandra
OUR SOLUTION- EXPLAINED
▪ Create a keyspace ‘provider1’ using pycassaShell.
We fired up our two code bases on two separate containers like this:
OUR SOLUTION- EXPLAINED
docker run -name remote_server_1 -link db:cassandra -
p 6000:6000 flux7/labs python software/remote_server.
py provider1
docker run -name flask_app_1 -link db:cassandra -p
6000:6000 flux7/labs python software/flask_app.py
provider1
DOCKER ISSUES DISCOVERED
Docker does not support multiple instances of Cassandra running on the
same machine.
Hosting multiple database instances on a single machine can quickly
cause resource shortages
❑ Followed the traditional solution to make an application multi-tenant
OUR SOLUTION
Code Changes •• To data ingestion server and web server by adding the keyspace parameter to the DB accesses.
Cassandra KEYSPACE /
provider ID •• Passed to each instance of the app on the
command line.
❑ Each provider in the data store gets a separate namespace without making
any changes to the column family schema.
Use of KEYSPACE as the namespace for each provider in the data store
LESSONS WE LEARNED
1. Docker is an extremely fast and elegant isolation framework: easy to port, cheap to run, easy to orchestrate
2. Multi-tenancy != changing the app to support multiple tenants
3. Docker orchestration frameworks are not at par with Docker today. What we have written is yet another one but for multi-tenancy.
4. Dockerfiles still need work -- we used shell scripts in some places
5. We can run multiple commands/container
DOCKER AUSTIN
FOR MORE DISCUSSIONS ON DOCKER!