Redesigning a pipeline flow with Docker Compose@AlmBrand
Loke Norlin Johannessen
Senior System Specialist
Commenting out the legacy code instead of deleting it
Old setup
Jenkins
• Single instance
• Pinged by Gitlab pipeline
Gitlab
• Build pipeline
Docker Enterprise
• Called by custom script
Old deploy method
- custom script -
Old “deploy” pipeline
Jenkins pipeline
New setup
Base systems
Ubuntu
• Minimal/Cloud
• Small iso – under 300mb
vSphere
• On-Prem virtualizasion provider
Gitlab
• CI/CD system
Docker Enterprise
• UCP – Access control
• DTR – Image repository
• Engine – Container engine
Ansible
• Configuration management
• Used during template generation
• Used during deploy of infrastructure
Packer
• VMware iso/template generation
Terraform
• Infrastructure provisioner
Automation tools
Gitlab repo
Dockerfile
• TIP: Comment the different stages
• TIP: Specify base image tag ( Do not use latest )
Basic layout
Basic layout
Single job pipeline
Compose
Use variables
• TIP: Gitlab CI/CD has a LOT of environmental variables
Lint a Dockerfile
• TIP: Hadolint is an excellent linting tool
Hadolint
ADD
TIP: Use COPY instead of ADD
COPY
TIP: Environment variables can be defined in a file
• TIP: Define multiple compose files in .env
• TIP: Order of precedence matters
• TIP: Use docker-compose config > combined_config.yml
Tip: Set defaults Tip: Mandatory variables
TIP: Set defaults if ENVs are not definedTIP: Allows you to build/deploy on workstationTIP: Requires docker-engine 18.09
• TIP: When installing packages, ensure you specify the version
• TIP: WORKDIR is recommended to use if you need to CD
• TIP: scratch is a perfect image to use with golang ( “zero” attack surface )
• TIP: Use “COPY --from” to import ca-certs/trusted-certs into scratch images
TIP: Use commit-SHA as tags
• TIP: Add environmentals in compose-files
• TIP: Environmentals are great in a multistage pipeline ( Dev -> Test -> Prod )
• TIP: Pushing to a Gitlab or DTR is easy ( We impersonate users and do the push on their behalf with ucp-bundles )
• TIP: Use dive to rate quality of image ( size, wasted space )
Tip:
Include pipelines
• TIP: Include/Use centralized gitlab configs for pipelines
Tip:
local storage driver
• TIP: The local storage driver is powerful – NFS mount without any plugins