Embed Size (px)
Citation preview
How to deploy Machine Learning models in the Cloud
Alex Casalboni
PYCON SETTE @ Firenze
clda.co/pycon7-alex
About Me
PYCON SETTE @ FIRENZE
@alex_casalboni
clda.co/pycon7-alex
Bachelor in Computer Science
Master in Sound & Music Engineering
6+ years of Web Development
3+ years of Python
Have you ever met a Data Scien@st?
PYCON SETTE @ FIRENZE
Bob
Bob leads a skilled team of data scienCsts
Bob has built a great ML model
Bob loves Python
Bob likes user-‐centered ML
clda.co/pycon7-alex
How did Bob build his model in Python?
PYCON SETTE @ FIRENZE
clda.co/pycon7-code
clda.co/pycon7-alex
Bob
PYCON SETTE @ FIRENZE
GPU availability
distributed compuCng
infrastructure-‐aware code
1. Bob needs Elas@city
clda.co/pycon7-alex
And why is deploying a challenge?
Bob
PYCON SETTE @ FIRENZE
MulC-‐model architecture
A/B tesCng & versioning
RESTful API
2. Bob needs plenty of Features
Auth layer
clda.co/pycon7-alex
And why is deploying a challenge?
Bob
PYCON SETTE @ FIRENZE
infrastructure maintenance
conCnuous delivery & deployments
automaCon, performance & monitoring
3. Data Scien@sts ain’t DevOps
≠
clda.co/pycon7-alex
And why is deploying a challenge?
Data ScienCst DevOps
PYCON SETTE @ FIRENZE
model design, training & tesCng
code reviews & debugging
data exploraCon
4. DevOps ain’t Data Scien@sts
≠
clda.co/pycon7-alex
And why is deploying a challenge?
Data ScienCstDevOps
Deployment Strategies
PYCON SETTE @ FIRENZE
1. Web-‐app controller
clda.co/pycon7-alex
how will Bob update his model(s)?
same website auth layer?
shared (limited) uWSGI processes?
It’s disarmingly simple, but…
Bob
Deployment Strategies
PYCON SETTE @ FIRENZE
2. Fleet of servers
clda.co/pycon7-alex
same problems as before
Bob has many more machines to maintain
sCll no elasCcity (over-‐provisioning)
Bigger capacity and no code changes, but…
Bob Bob2 Bob3
Load Balancing
Deployment Strategies
PYCON SETTE @ FIRENZE
3. Auto Scaling
clda.co/pycon7-alex
<Bob>
sCll shared resources? containers?
even bigger lack of ownership
what about caching, versioning and auth?
Bob achieved elasCcity and availability , but…
AWS ELB + Auto Scaling (or maybe ElasCc Beanstalk?)
Deployment Strategies
PYCON SETTE @ FIRENZE
4. Serverless
clda.co/pycon7-alex
+Amazon API Gateway
AWS Lambda
Happy Bob
Infinite scalability, global CDN and caching
Transparent elasCcity (and PAYG)
RESTful & auth layer + versioning (for free!)
Bob doesn’t need to be a DevOps anymore!
The Serverless Cloud
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
There is always a server somewhere, you just don't have to worry about it :)
Does Bob’s code change a lot?
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
Basically, no change!
Bob
How does Bob keep the model updated?
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
Bob
Hint: use gzip!
PYCON SETTE @ FIRENZE
clda.co/pycon7-lambda
clda.co/pycon7-alex
Serverless Cloud Domina@on
PYCON SETTE @ FIRENZE
Amazon Web Services (Lambda)
Microso` Azure (FuncCons)
Google Cloud Plaborm (FuncCons)
IBM Bluemix (OpenWhisk)
hook.io
Cloud PlaSorms Hybrid & Microservices
stackhut.com
webtask.io
iron.io
clda.co/pycon7-alex
