Upping your NiFi Game with Docker

  • View

  • Download

Embed Size (px)



+Upping your NiFi game with Docker29 October 2015Aldrin Piri@aldrinpiri

What is Docker?A line of khakis, pants and casual clothing

Its an obscure open source project

What is Docker?Docker is an open platform for building, shipping and running distributed applications. It gives programmers, development teams and operations engineers the common toolbox they need to take advantage of the distributed and networked nature of modern applications.

Docker is an ecosystemDocker Hub https://hub.docker.com/A registry of available images

ToolingLinux provides a native experienceWindows & OS X make use of a VM and associated tools

People and Organizations

Glossary for the talkImageBinary representation of a filesystemContainerInstance of an imageCopy on WriteImages are untouched until changes are needed, at which point a new copy is createdVolumesBridge the external host world into the container

EnvironmentOS X 10.10Docker Toolboxhttps://www.docker.com/docker-toolbox

Docker, Docker Machine, & Docker Compose

DOCKER-MACHINEProvisions Docker environmentsLocally: Native Docker daemon, VirtualBox

Cloud: Amazon EC2, Azure, DigitalOcean

Establishes command line for communicating with instances

DockerRun images to create containersThe lifespan of a container is tied to the process that originates itDefaults specified via ENTRYPOINT or CMD

Specify arguments on run inclusive of providingVolumesLinks to other running containers (unidirectional network mappings via an alias)Environment variablesExposed PortsAlternate ENTRYPOINT/CMD

If an image is present, it will attempt to be pulled from the Docker Hub and locally cached

Our base Docker imagealdrin/apache-nifiDocker Hubhttps://hub.docker.com/r/aldrin/apache-nifi/Sourcehttps://github.com/apiri/dockerfile-apache-nifi

Where does it all come from?Dockerfile is our source code

Where does it all come from?An image is our compiled binary

Docker for DevelopingVolumes allow mapping libraries directly into the environment

Links allow interacting with services at defined aliases


Docker NetworkDockers libnetwork makes great strides in increasing the networking modelService DiscoveryPluggable drivers

Unfortunately, it has not been releasedComing soon: ~1.9.0

There are a few ways to handle this with the current release inclusive of running a DNS server/service discovery, but we are going a simple route for ease.

Enter AmbassadorsProvides service/container portability as links are hardcoded


Introducing Docker ComposeOriginally a project called fig

Provides the definition and running of multi-container applications

YAML descriptor files

docker-compose drives controlling the application


ResourcesDocker https://www.docker.com/

Visualizing Docker Containers and Imageshttp://merrigrove.blogspot.com/2015/10/visualizing-docker-containers-and-images.html

Jess Frazelle - Dockerize everythingContainer Hacks and Fun Images Presentationhttps://www.youtube.com/watch?v=1qlLUf7KtAwhttps://github.com/jfrazelle

Source Code and Scripthttps://github.com/apiri/docker-with-nifi