Using python and docker for data science

Preview:

DESCRIPTION

PyData London meetup group lightning talk slides on getting an ipython notebook with the scipy stack and custom packages running in a notebook server in 5 minutes.

Citation preview

USING DOCKER FOR DATASCIENCE

WHY DOCKERPortable environmentIsolated between projectsStatelessFast local file accessHetrogenous

GET DOCKERboot2docker .dmg or .exeapt-get install docker.io ...

https://docs.docker.com/installation/

RUN SCIPYSERVER$ docker run -d -e "PASSWORD=YourPassword?" ipython/scipyserver

$ docker run \ -d \ -e "PASSWORD=YourPassword?" \ --name dev_notebook \ -p 443:8888 \ ipython/scipyserver

https://localhost:443https://{boot2docker ip}:443

CREATE DATA-ONLY CONTAINERS$ docker run \ -d \ -v ~/notebooks:/notebooks \ --name notebooks_container \ ubuntu echo notebooks

$ docker run -d -v ~/data:/data --name data_container ubuntu echo data

MOUNT DATA-ONLY CONTAINERS$ docker stop dev_notebook$ docker rm dev_notebook

$ docker run \ -d \ -e "PASSWORD=YourPassword?" \ --name dev_notebook \ -p 443:8888 \ --volumes-from data_container \ --volumes-from notebooks_container \ ipython/scipyserver

CREATE A DOCKERFILEFROM ipython/scipyserverMAINTAINER Calvin Giles <calvin.giles@gmail.com>

COPY requirements.txt /requirements.txtRUN pip2 install -r /requirements.txtRUN pip3 install -r /requirements.txt

$ docker build \ -t calvingiles/ds-notebook \ .

$ docker run \ -d \ -e "PASSWORD=YourPassword?" \ --name dev_notebook \ -p 443:8888 \ --volumes-from data_container \ --volumes-from notebooks_container \ calvingiles/ds-notebook

NEXT TIMEConnecting to local database containersTweaking the boot2docker vm memory from 2GB to 8 (ormore...)Linking to private git repositoriesAutomated builds with github linking

MORE?Docker User Guide:

Docker Reference:

ipython docker images:

my docker image:

http://docs.docker.com/userguide/

http://docs.docker.com/reference/commandline/cli/

https://registry.hub.docker.com/repos/ipython/

https://github.com/calvingiles/ds-notebookhttps://registry.hub.docker.com/u/calvingiles/ds-notebook/

ABOUT MECalvin GilesData Scientist at AdthenaPyData Meetup Organiseruntangleconsulting.iocalvin.giles@gmail.com@calvingiles on twitter, github, docker hub (and many more)

Recommended