Igor Stoppa & Eduard Bartosh & JF Ding - Tizen Wiki · PDF file• Services packages: Jenkins-jobs, Jenkins-plugins, Jenkins-scripts, ... • For other projects it’s up to the project

OTC Tools

Development and Release process

Igor Stoppa & Eduard Bartosh & JF Ding V0.12 23 May 2013

INTEL CONFIDENTIAL 2 SSG System Software Division

Purpose of the document

• This document describes development and release workflow for the following projects: • Tools releases: bmap-tools, depanneur, gbs, git-buildpackage and mic • QA Tools: CATS, QA Reports. These are released independently. • Services packages: Jenkins-jobs, Jenkins-plugins, Jenkins-scripts, and all

the packages for BOSS. • For the above mentioned projects it’s strongly recommended to follow the

workflow and rules described in this document.

• For other projects it’s up to the project maintainers to decide to use the workflow and rules described in this document, use them partly or not to use at all.

• In the following of this document, all the references of “Tools” names are for the convenience of description, which can be replaced by other different component name, e.g. “Services”


1) developers submit patches for review

2) preliminary automated verification: test packages are built in a sandbox and smoke tested

3) the submission is reviewed – when accepted it requires rebase and resubmit for other changes queued for review

4) upload to OBS of the new version of the package(s)

5) the packages are published to the appropriate location

Development & Release Process

(Jenkins) pre-review

build & test

devel

patch A

patch B

(Gerrit) human review

& acceptance

gerrit

devel

branch

OBS build in

sandbox

(Jenkins) upload

submit for

review

(OBS) integration into <project>:Devel

build test

OBS <project>:Devel

(Jenkins) publish

Repos

tizen.org

tzdev.org

intranet Branching

& version

management

1 2 3 4 5

OBS <project>:Devel

<project>:Pre-Release <project>

rebase

required


Branching process 1. Developers submit changes to devel branch

2. When the devel branch meets the criteria for rolling out a release, it is branched into a release-<rnumber> branch

3. The release-<rnumber> undergoes a stabilization phase during which all changes submitted to release-<number> branch have to be submitted also to devel.

4. Release candidates should be tagged as v<rnumber>rc<num>

5. Eventually the branch is tagged as v<rnumber> and a release is made.

6. Master Branch is reset to the release tag and doesn’t receive any direct commit.

7. Bugfix branch is made for current release

8. Master Branch is reset once more to the latest bugfix release.

devel

branch

release-<rnumber>

branch

tag & release patch

2

3 4

Master

branch

5 1

6

7

8


Versioning

• Recommended versioning scheme for releases is major.minor[.bugfix] • Release candidates should add -rc<num> suffix to release version, for example: 1.2-rc1 or

1.2.3-rc1

• In order to make package versions reflect versioning scheme and look similar for rpm and deb they should follow these recommendations:

• rpm .spec files: • Release candidate: Version: 1.2, Release: 0.rc1.<CI_CNT>.<B_CNT> • Final release: Version: 1.2, Release: 1.<CI_CNT>.<B_CNT>

• debian/changelog • Release candidate: 1.2-0.rc1 • Final release: 1.2-1

• After creating release- branch package version in devel branch must be increased to ensure

that version in <project>:Devel repository is always higher than in <project>Pre-release and <project>


Advice of the Versioning for Services Packages

• Because, we need not to release the services packages to end users

currently, not like developer tools. And we need only to deploy them to

the product infrastructure servers instead. So the versioning of these

packages can be simpler:

• No rc versions

• No bugfixing versions

• Using <Major>.<Minor> versioning schema only, for every release, just

increase the minor number. And for some important milestone release,

increase the major number.


Submitting changes

• Changes are submitted for review using git push <origin> <commit id>:refs/for/<target branch>

• Every submission triggers Jenkins job OTC-Tools-Tester-<project name>. This job does the following: • Submits sources to OBS for the build to a project home:tester:Tools-<project

name>-<gerrit change number>.<gerrit patchset number>. OBS buld targets for the builds are taken from base project. For Tools releases it is Tools:Devel. Result packages uploaded to build repository at http://download.otctools.jf.intel.com/home%3a/tester%3a/Tools-<package name>-<gerrit change number>.<gerrit patchset>/

• Performs package installations in clean virtual environments for all target distributions.

• Optionally runs pylint and nosetests for python projects and produces violation, code coverage and test reports

http://download.otctools.jf.intel.com/home:/tester:/Tools-<package




Accepting changes • When change is accepted to the target branch(devel or release-<number>) in

Gerrit OTC-Tools-Tester-<project name> is also triggered. For merged changes it does the same steps as on change submission, but upload sources to base OBS project. • For devel branch it uploads sources to Tools:Devel • For release-<number> branches it uploads sources to Tools:Pre-release This way system maintains consistency between code in devel and release

branches and packages in correspondent OBS projects.

• It’s recommended to configure Gerrit to allow only fast-forward merges. In this case changes have to be rebased on top of target branch before merging and pushed to Gerrit again. This will trigger OTC-Tools-Tester-<project name> job again and change will be tested again. It will reduce chances to get broken code to the target branches.


Criteria for rejection of submissions

• the submission claims to fix a bug, but the bug is still reproducible with roughly the same frequency (if the frequency is significantly reduced we probably still want to accept it)

• the submission claims to implement a feature, but the feature verification fails so much that the feature is considered to be unusable (in practice the submission is not adding any value)

• the submission degrades the overall code quality (fails automated code analysis & verification / has problems with the implementation)

• the submission introduces major regressions (more on regression testing in separate foil)


Regression testing

• Regression testing must be performed at least before releasing a tool.

• It should be done also periodically, during the development phase,

preferably synchronised with integration of major changes.

• Specific test cases depend on the nature of each tool.

• Generic steps:

• take previous execution of the stable (production) release of the tool

• use the same data with the development release of the tool

• verify that the output is consistent and the only differences are due to bug-

fixes and planned features.


More about Services Regression Testing

• Because it’s very critical for Tizen development process if we release

and deploy the services packages in product servers. We need three

level testings:

• Unittest by Jenkins tester jobs for each commit

• For the packages in pre-release, we need to deploy them to staging

infrastructure servers to confirm there’s no regression

• After deployed in the product servers, we need a checklist for operations to

check all the outputs are what we expected and there’s no regression

against the previous release.


Releasing

• Release stabilization is done in release-<rnumber> branches

• Jenkins job OTC-Tools-Tester-<project name> is taking care of synchronizing

code in release-<rnumber> branches and packages in Tools:Pre-release

• Pre-release testers should install packages from

http://download.otctools.jf.intel.com/home%3a/tester%3a/Tools-<package

name>-<gerrit change number>.<gerrit patchset>/ repos to verify changes

under review and from http://download.otctools.jf.intel.com/Tools%3a/Pre-

release/ to test accepted changes.

• When release is ready it’s tagged with v<rnumber> tag and master branch is

resetted to it. Packages from Tools:Pre-release are copied to Tools




http://download.otctools.jf.intel.com/Tools:/Pre-release/




Bugfix releases (do we need to update it?)

• Bugfix releases should be done in bugfix-<rnumber> branches. Release-<rnumber> branches can’t be used for bugfix releases as they’re mapped to Tools:Pre-release OBS project, which may contain packages scheduled for the next release.

• Jenkins job OTC-Tools-Tester-<project name> is taking care of synchronizing

code in bugfix-<rnumber> branches and packages in Tools:Bugfix

• Bugfix testers install packages from http://download.otctools.jf.intel.com/home%3a/tester%3a/Tools-<package name>-<gerrit change number>.<gerrit patchset>/ repos to verify changes under review and from http://download.otctools.jf.intel.com/Tools%3a/Bugfix/ to test accepted changes.

• When release is ready it’s tagged with v<rnumber> tag and master branch is reset to it. Packages from Tools:Bugifx are copied to Tools






Publishing releases and pre-releases

• Tools and Tools:Prerelease repositories are published to the http://download.tizen.org/tools/ . Job is run by release managers manually. Job publishes repos to public location, extracts release notes from packages and updates .repo files with the proper urls.

• Similarly, Services and Services:prerelease repos will be published to http://download.tizen.org/tools/ci/.

• Package maintainers should put RELEASE_NOTES file into packages using %doc RELEASE_NOTES in spec file. Release notes will be extracted by the publishing job and uploaded to the public site as RELEASE_NOTES.<package name>

• Structure of the public download site contains archive of all releases, pre-release and link to

the latest release.

http://download.tizen.org/tools/


Development and release flow (Tools):

‘Devel’ branch

‘release-*’ branch

‘master’ branch

patch

Git branch

peer

review

pre-review

testing

Jenkins

pass

failed

Tools:Devel

Tools:pre-release

Tools

accept download.otctools.jf.intel.com/Tools:/Devel/

reject

peer

review

accept hot fix download.tizen.org/tools/pre-release/

download.tizen.org/tools/archive/ download.tizen.org/tools/latest-release/

OBS download repo

Functional testing

Full testing

Sanity testing

testing

handled by Jenkins automatically

branch

merge Deploy request

Release announce


Development and release flow (Services):

‘Devel’ branch

‘release-*’ branch

‘master’ branch

patch

Git branch

peer

review

pre-review

testing

Jenkins

pass

failed

Services:Devel

Services:pre-release

Services

accept download.otctools.jf.intel.com/Services:/Devel/

reject

peer

review

accept hot fix download.tizen.org/tools/ci/pre-release/

download.tizen.org/tools/ci/archive/ download.tizen.org/tools/ci/latest-release/

OBS download repo

Functional testing

Full testing

Sanity testing

testing

handled by Jenkins automatically

branch

merge Staging Deploy request

Deploy request

Announcement for the changes which need developers to know

Documents

Igor Stoppa & Eduard Bartosh & JF Ding - Tizen Wiki · PDF file• Services packages: Jenkins-jobs, Jenkins-plugins, Jenkins-scripts, ... • For other projects it’s up to the project