Modern Continuous Delivery with Docker and Liferay

Modern Continuous Delivery Liferay meets Docker

Manuel de la Peña, Software Engineer

@mdelapenya #LRDEVCON

PRODUCTION SYSTEMSTHIS TALK IS NOT ABOUT…

http://cdn.wallpapersafari.com/41/27/KhsIdT.jpg


THIS TALK IS ABOUT…

DEVELOPMENT PROCESSEShttp://www.itechscope.gr/img/banner-image-process.jpg


CONTINUOUS DELIVERY @ LIFERAY

▸Continuous Delivery

▸Challenges

▸Approach to our Development Process


CONTINUOUS DELIVERY

http://www.jsg.utexas.edu/lacp/files/PGC_Pipeline_Data.jpg


CONTINUOUSLY DELIVER VALUE TO CUSTOMERS


CONTINUOUS DELIVERY▸Helps in building the product with the most possible

quality

▸Repeat hard tasks (release, deploy) more times

▸Find bugs the soonest, when it’s still cheap

▸Reduce feedback loop from customers


CONTINUOUSLY DELIVER VALUE TO INTERNAL

CUSTOMERS


INTERNAL CUSTOMERS▸Engineering teams:

▸Developers, Support engineers, Testers, Consultants, Designers, Product Managers

▸Other nontechnical teams:

▸Sales, Marketing, …


CD FOR OUR TEAM MEANS…▸Very briefly…

▸Automate Build + Dependency Management

▸Automate Testing (Pull Request Tester)

▸Automate internal deployments


PULL-REQUEST TESTER▸JSP Compile, Semantic Versioning, Portlet

Spec TCK, Plugins compile…

▸JBoss 7, Tomcat 8, Weblogic 12, Wildfly 10…

▸Hypersonic, MySQL 5.6, MariaDB10, Oracle12, PostgreSQL 9.4, Sybase16, DB2 10.5…

http://www.vwand.com/images/usecases/delivery.jpg


CHALLENGES

http://www.greenbookblog.org/wp-content/uploads/2015/06/challenges.jpg


CHALLENGES▸Old-School IT procedures

▸Divergent environments

▸Bug reproducibility

1. OLD-SCHOOL IT PROCEDURES


OLD-SCHOOL IT PROCEDURES▸New environments request

▸Physic Servers && Virtual Machines

▸Virtualization (Proxmox)

▸Operate the environment


OLD-SCHOOL IT PROCEDURES▸Manual or semi-automated deployments:

▸Jenkins jobs

▸Customised Shell scripts

▸rsync

▸Specific application configuration


OLD-SCHOOL IT PROCEDURES▸Documentation

▸Wikis

▸ Intranets

▸Google Docs

▸SysAdmin’s head?

2.DIVERGENT ENVIRONMENTS


DIVERGENT ENVIRONMENTS▸Manual configuration produces divergency

▸DEV servers are not a copy of PROD servers

▸O.S., CPUs, RAM, disk, libraries, configuration

▸Same problem with DEV laptops

PETS OR CATTLE?

@mdelapenya #LRDEVCON https://www.reconnix.com/blog/wp-content/uploads/2015/02/Cow.jpg


3. BUGS REPRODUCIBILITY


INCREASED COMPLEXITY

▸Different source code versions

▸Different O.S.

▸Different application servers

▸Different databases


DB2 10.1

MariaDB 10

MySQL 5.6

Oracle 12cR1

PostgreSQL 9.3

SQL Server 2012

Sybase ASE 16

JBoss EAP 6.4 X X X X X X X

Tomcat 8.0 X X X X X X X

Weblogic 12c X X X X X X X

Websphere 8.5 X X X X X X X

Wildfly 10.0 X X X X X X X


💔❤▸Not changing context,

stay in the GIT workspace

▸Using an already-built environment, many times

▸Using automated procedures

▸Changing context: stash or WIP commit

▸Building in a specific commit, many times

▸Using manual procedures (DRY)

DEVELOPERS: LOVE IT OR LIST IT


THE SYSTEM WE LOVE… ❤▸Modern and easy to use

▸Builds environments in an automated way, with everything (application plus dependencies)

▸Stores those environments for reusability

▸ Increases teams’ autonomy

▸ Increases collaboration with I.T.▸Does it sound as DEVOPS culture?

@mdelapenya #LRDEVCONREPRODUCIBILITY

DIVERGENCYENVIRONMENT

BUG

ENVIRONMENT

REPRODUCIBILITY


JENKINS + DOCKER CONTAINERS


JENKINS▸Automation server

▸Jobs that trigger builds

▸Builds compound by steps

▸Plugins ecosystem


DOCKER CONTAINERS▸Docker Inc. is a company supporting

containers (Linux and Windows!)

▸Container = lighter virtualisation than VM


DOCKER CONTAINERS▸Standar De-Facto

▸Other container technologies: OpenVZ, LXC, Rocket, etc.

▸Image model based in Layers

▸CLI and REST API

▸BIG ecosystem around


ROLES▸Docker Registry

▸Docker Engine

▸Docker Builder

▸WOMC! (Works On My Computer)


ROLE: DOCKER PRIVATE REGISTRY▸Docker Hub: public IT infrastructure hub

▸Private Registry: on-premise

▸VM with massive storage for images

▸docker-compose

▸Containers: Apache + Registry

▸Security: OpenSSL

▸ https://docs.docker.com/registry/recipes/apache

https://docs.docker.com/registry/recipes/apache


ROLE: DOCKER ENGINE▸Docker daemon > O.S. package

▸Application infrastructure and runtime dependencies standardised

▸Same runtime environment from Docker Engine to Docker Engine

▸Docker Engine = Docker Host


ROLE: DOCKER ENGINE▸VM per team as team’s lab

▸VM provisioned Ansible role

▸Runs Liferay images from private registry

▸docker run or docker-compose

▸Liferay + Database

▸Containers and images clean-up


WHY VMS?▸They are truly helpful for us

▸Moving a whole docker host between machines

▸Backup of a whole docker host, including its containers

▸Security: isolated teams (potential shutdowns, resources-draining by other teams)


ROLE: DOCKER BUILDER▸Jenkins slaves

▸Fetches portal source code from master branch, reading last commit ——> HASH commit

▸Generates a Liferay Docker image every night

▸Uploads those images to the private docker registry

▸Deploys that MASTER to a controlled environment

http://www.vwand.com/images/usecases/delivery.jpg


NAME CONVENTION FOR IMAGES

PRIVATE_REGISTRY/GIT_REPO :

GIT_BRANCH-HASHTIMESTAMP


NAME CONVENTION FOR IMAGES (2)

LOCALHOST:5000/LIFERAY-PORTAL :

20161115123000MASTER-ABCDEF1234567890



PRIVATE_REGISTRY/GIT_USER/GIT_REPO :

GIT_BRANCH-HASH



LOCALHOST:5000/MDELAPENYA/LIFERAY-PORTAL :

FEATURE-1-ABCDEF1234567890


DOCKER ENGINE

CT.1 CT.2 CT.N

docker-engine-team3

DOCKER ENGINE

CT.1 CT.2 CT.N

docker-engine-team1 DOCKER ENGINE

CT.1 CT.2 CT.N

docker-engine-team2

CT.1 CT.2 CT.N

DOCKER ENGINE

PRIVATE REGISTRY

docker-engine-core

Virtual Machine

Container

Physical Machine

docker-builder-i

push

run

run

pull

run


CT.1 CT.2 CT.N

DOCKER ENGINE

PRIVATE REGISTRY

docker-engine-core

pull & push

DOCKER ENGINE

CT.1 CT.2 CT.N

docker-engine-team-i

run

run

run

DOCKER ENGINE

CT.1 CT.2 CT.N

local-machine-j

run

Virtual Machine

Container

Physical Machine


DIRECT BENEFITS▸Less dependency on IT

▸More autonomous teams

▸Both Developers and nontechnical Users can build environments more quickly (demos and showcases!!)

▸With training, Developers can reproduce new environments starting from already created ones


SUMMARISING…▸Define your CD pipeline: min(Build + Test + Deploy)

▸Build Docker image within the pipeline

▸Publish images for teams

▸More autonomous teams

▸Increase of development speed

THANK YOU! QUESTIONS?

LIKED IT?, PLEASE VOTE!

My Docker images

https://hub.docker.com/r/mdelapenya/liferay-portal https://github.com/mdelapenya/docker-liferay-portal

Technology

Modern Continuous Delivery with Docker and Liferay