Continuous Integration

Collaborative development issues

• Checkout of a shared version of software (“mainline”)• Creation of personal working copies of developers

• Software development: modification of personal working copies• Working copies diverge from mainline

• Different parallel changes by different developers

• Integration of a working copy into mainline• Infrequent integration: likely conflict with changes from other developers

• Time-consuming and error-prone process (“integration hell”)

• Worst-case scenario: discard previous work and redo work on top of current mainline

Continuous Integration (CI)

• Frequently integrate developer work into shared mainline• Minimize divergence of working copy from mainline

• Small integration effort

• CI server• Builds software from source code

• Runs tests on built software

• Reports test failures to developers• Errors can be corrected immediately

• Runs periodically or after integration events

Workflow

Principles

• Revision control system• Minimize divergence of branches

• Automated build process

• Integration into mainline every day• Small conflicts are easy to resolve and facilitate communication between

developers

• Automation of software quality control

• Availability of latest build• Facilitates further testing

Pros and cons

• Pros• Immediate testing of software changes

• Immediate feedback to developers on impact of their changes

• Easily revert changes that introduced a bug

• Avoid chaos when release date approaches

• Latest build always available for testing, demos, or releases

• Frequent check-in encourages writing modular code

• Cons• Initial setup time

• Test suite development

Continuous delivery

• Also called continuous deployment

• Extends continuous integration up to software releases

• Frequent automated or semi-automated releases

• Reduced manual overhead

• Minimizes risks when introducing new release

• Every developer commit may result in a new release• Encourages quality work

Atlassian Bamboo

• CI Server from Atlassian

• https://www.atlassian.com/software/bamboo/

• Web interface

• Supports multiple revision control systems• Git, Mercurial, CVS, Subversion, Perforce

• Can be used with any programming language

• Native support for multiple build tools

• Supports sequential and parallel task execution

• Can be integrated with JIRA issue tracker

Bamboo workflow

Bamboo plan

• Belongs to a Bamboo project

• Plan build: execution of a plan

• Single stage by default, but multiple stages possible

• Specifies a default source code repository

• Processes one or more stages, run sequentially

• Specifies how the build is triggered

• Specifies triggering dependencies from other plans in the project

• Notification of build results

Plan creation (1/3)

Plan creation (2/3)

Plan creation (3/3)

• Plan trigger options:• Poll repository for changes

• Repository triggers build when changes are committed

• Scheduled

• Single daily build

• Manual

Tasks

• Source control• Source code checkout, VCS branching, VCS tagging

• Builder• Ant, Maven, Grails, Nant, MSBuild, Visual Studio, Command, Script

• Tests• JUnit, TestNG, PHPUnit, MBUnit, NUnit, MSTest

• Deployment• Tomcat, Heroku, SCP, SSH

Plan summary

Agents

• Service running Bamboo builds

• Number of agents determines number of jobs executable in parallel

• Local agent• Runs in the same process as Bamboo server

• Remote agent• Runs outside Bamboo server

• Agent capabilities• Determine what jobs can be run

• Version control system client, executable, JDK, custom capability

Stages

• Executed sequentially in a plan• Example: compilation stage, test stage, deployment stage

• One default stage in a new plan

• Can process its jobs in parallel• If agents are available

• Must successfully complete all its jobs before the next stage in the plan can be processed

• Manual stages:• Build execution paused until user manually runs stage

Jobs

• Executed (potentially) in parallel within a stage

• Job requirements matched against agent capabilities

• Process one or more tasks sequentially on the same agent

• Final tasks

• Job artifacts: files created during job• Available as plan build results

• Defined as part of job definition

• Artifact sharing: artifact created in a job can be used outside the job• Artifact sharing between plans

Build results

• Plan build or job build

• Build status: successful, failed, incomplete

• Responsible users• Automatically or manually assigned

• Notified of build failures

• Test results

• Code changes

• Artifacts

• Logs

Quarantined tests

• Tests in a build whose result is disconnected from build results

• Still run during builds• Test results available in build results

• Use cases• Tests known to fail, but artifacts still good

• In test-driven development, tests for unimplemented functionalities

• Tests with unpredictable results

• Removing a test from a plan without affecting other plans

Plan branches

• Associated to branches in source repository

• Inherit configuration from containing plan• Containing plan associated to branch indicated in default repository

• Manual branching• Define plan branches manually

• Automatic branching• Create plan branch when new branch is detected in repository

• Remove plan branch if no commits are detected on repository branch

Plan branches: automatic merging

• 2 merging models

Plan build dependencies

• A successful plan build triggers another plan• Parent plan -> child plan

• Works in addition to child plan triggers

• Checkout of same version in source repository

• Works with plan branches• If branch names in parent and child plan branches match

• Dependency blocking• Block child build if parent builds are queued or in progress

• Block child build if parent plans have unbuilt changes

Deployment projects (1/3)

• Provide separation between build artifacts and deployed releases

• Identify deployment environments• E.g. QA, staging and production environments

• Motivations:• Deployed builds are difficult to find

• Difficult to find what changes were made between deployments

• Difficult to know what was deployed, and when and where it was deployed

• Lack of insight into the QA process

• Poor control over who can deploy

Deployment projects (2/3)

Deployment projects (3/3)

Deployment project configuration

• Build plan

• Shared artifacts

• Release naming scheme

• Deployment environment• Tasks

• Source control, builder, deployment

• Triggers

• Agents• Can be dedicated to deployment projects

• Permissions

Branch deployment

Notifications

• Plan-level and job-level events

• Recipients: Bamboo users, groups, responsible users, committers, watchers

• Bamboo wallboard• All plans, favorite plans, filtered plans

• E-mail

• Instant messaging

• RSS feeds

• System-level notifications

Wallboard

Usage patterns: Fail Fast

Usage patterns: artifact sharing

Usage patterns: manually triggered stages