10
USING DOCKER FOR DATA SCIENCE

Using python and docker for data science

Embed Size (px)

DESCRIPTION

PyData London meetup group lightning talk slides on getting an ipython notebook with the scipy stack and custom packages running in a notebook server in 5 minutes.

Citation preview

Page 1: Using python and docker for data science

USING DOCKER FOR DATASCIENCE

Page 2: Using python and docker for data science

WHY DOCKERPortable environmentIsolated between projectsStatelessFast local file accessHetrogenous

Page 3: Using python and docker for data science

GET DOCKERboot2docker .dmg or .exeapt-get install docker.io ...

https://docs.docker.com/installation/

Page 4: Using python and docker for data science

RUN SCIPYSERVER$ docker run -d -e "PASSWORD=YourPassword?" ipython/scipyserver

$ docker run \ -d \ -e "PASSWORD=YourPassword?" \ --name dev_notebook \ -p 443:8888 \ ipython/scipyserver

https://localhost:443https://{boot2docker ip}:443

Page 5: Using python and docker for data science

CREATE DATA-ONLY CONTAINERS$ docker run \ -d \ -v ~/notebooks:/notebooks \ --name notebooks_container \ ubuntu echo notebooks

$ docker run -d -v ~/data:/data --name data_container ubuntu echo data

Page 6: Using python and docker for data science

MOUNT DATA-ONLY CONTAINERS$ docker stop dev_notebook$ docker rm dev_notebook

$ docker run \ -d \ -e "PASSWORD=YourPassword?" \ --name dev_notebook \ -p 443:8888 \ --volumes-from data_container \ --volumes-from notebooks_container \ ipython/scipyserver

Page 7: Using python and docker for data science

CREATE A DOCKERFILEFROM ipython/scipyserverMAINTAINER Calvin Giles <[email protected]>

COPY requirements.txt /requirements.txtRUN pip2 install -r /requirements.txtRUN pip3 install -r /requirements.txt

$ docker build \ -t calvingiles/ds-notebook \ .

$ docker run \ -d \ -e "PASSWORD=YourPassword?" \ --name dev_notebook \ -p 443:8888 \ --volumes-from data_container \ --volumes-from notebooks_container \ calvingiles/ds-notebook

Page 8: Using python and docker for data science

NEXT TIMEConnecting to local database containersTweaking the boot2docker vm memory from 2GB to 8 (ormore...)Linking to private git repositoriesAutomated builds with github linking

Page 9: Using python and docker for data science

MORE?Docker User Guide:

Docker Reference:

ipython docker images:

my docker image:

http://docs.docker.com/userguide/

http://docs.docker.com/reference/commandline/cli/

https://registry.hub.docker.com/repos/ipython/

https://github.com/calvingiles/ds-notebookhttps://registry.hub.docker.com/u/calvingiles/ds-notebook/

Page 10: Using python and docker for data science

ABOUT MECalvin GilesData Scientist at AdthenaPyData Meetup [email protected]@calvingiles on twitter, github, docker hub (and many more)