Effective Packaging (2010). Petr Uzel

7/31/2019 Effective Packaging (2010). Petr Uzel

1/40

Effective packaging

Petr UzelL3/[email protected]

ptr uzl @ IRC


2/40

About this talk

collection of tips and tricks that make (some) package maintenancetasks more efficient

no rocket science, no cool technologies

based on experience

there are definitely better ways

2/1 cNovell Inc. All Right Reserved


3/40

Which package is broken?

given a bug (e.g. /path/to/bin crashes), sometimes it is notobvious which package should be fixed

bugzilla does not track package info

rpm -qf /path/to/file gives name of the RPM

we need source package, not RPM

$ rpm -qf /lib64/libblkid.so.1

libblkid1-2.18-56.1.x86_64

$ osc search -s --package blkid

No matches found for blkid in packages



4/40

Source package name

lets ask RPM:

$ rpm -q --queryformat %{SOURCERPM}\n \

libblkid1-2.18-56.1.x86_64

util-linux-2.18-56.1.src.rpm

Everything combined together (file source package)

$ rpm -q --queryformat %{SOURCERPM}\n \

$(rpm -qf /lib64/libblkid.so.1) | \

sed -e s/$^.*$-.*-.*src.rpm/\1/

util-linux

Where did I install the package from?

$ rpm -q --queryformat %{DISTURL}\n util-linux

obs://build.opensuse.org/home:puzel:staging/openSUSE_11.3/6a[CUT]



5/40

Get package source from OBS

With branching:

slow: osc branch openSUSE:Factory foo; osc cohome:*:branches:PRJ

faster: osc branch --checkout openSUSE:Factory foo

fastest: osc getpac foo or osc bco foo

hyperspeed: osc getpac + osc bash completion

Without branching:

function show_devel {

osc meta pkg openSUSE:Factory ${1} | fgrep devel

}

go directly to devel project



6/40

Building package

osc build in package working copy usually does the right thing

more control: osc build [OPTS] REPOSITORY ARCH SPECFILE

REPOSITORY, ARCH and SPECFILE detected automatically if notspecified

list REPO-ARCH combinations: osc repos [PRJ]



7/40

osc build options

build debuginfo and debugsource packages:osc build --debuginfo/-d

skip post build checks:

osc build --nochecksdelete old build root before build:osc build --clean

use alternative project for build:osc --alternative-project=openSUSE:Factory

use ccache to speed up build:osc build --ccache



8/40

Use custom RPMs in buildroot

foo depends on bar, which is not in buildservice yet

but we have bar.rpm

osc build -p/--prefer-pkgs DIR: prefer packages from DIRwhen installing buildroot

$ cd foo

$ osc build -p /path/to_dir_with_bar.rpm foo.spec

osc build -x/--extra-pkgs PKG: add this package to buildroot



9/40

Buildroot tips

by default, buildroot is /var/tmp/build-root

OSC BUILD ROOT overrides the default

$ export OSC_BUILD_ROOT=/var/tmp/build-root-temp

$ osc build

use when parallelly building multiple packages

buildroot on tmpfs5% speedup of parted build

$ grep build-root /etc/fstab

tmpfs /var/tmp/build-root tmpfs size=5G 0 0



10/40

Avoid password for building

by default, osc build asks for root password; to avoid itset su-wrapper = sudo in ~/.oscrc

add yourlogin ALL = (root) NOPASSWD: /usr/bin/buildto sudoers file



11/40

Build log

build log is stored in $OSC BUILD ROOT/.build.log

slow: less /var/tmp/build-root/.build.logfaster: osc lbl/localbuildlog

respects $OSC BUILD ROOTfrom working copy

osc rbl/remotebuildlog PRJ PKG REPO ARCH - get remote buildlog



12/40

Version update workflow

1 get package source

2 download tarball

3 verify tarball

4 repack tarball5 check differences

6 adjust specfile

7 refresh patches

8 run test build9 test the package

10 write changelog entry

11 commit



13/40

Downloading tarball with ncftp

many projects publish tarballs on FTP sites

ncftp is a command-line FTP client with bookmarking support

useful to remember locations where upstream publishes tarballs

supported by bash-completion

$ ncftp ut -> ncftp util-linux

ncftp... ls

ncftp... get fil -> filename

ncftpbookmarks: bookmark editor



14/40

Repacking tarball

packaging policy requires the tarball to be bzip2-ed (or xz-ed)

sometimes upstream provides only .tar.gz

$ bznew parted-2.3.tar.gz

$ ls *.bz2

parted-2.3.tar.bz2



15/40

Whats new in tarball?

Compare outputs from configure --help

$ git clone git://gitorious.org/opensuse/pack-tools.git

$ diffconf gnupg-2.0.10.tar.bz2 gnupg-2.0.16.tar.bz2



16/40

Change specfile

usually just open specfile and increase the version number

small helper:

$ alias vs=vim -p *.spec

in vim (normal mode), press Ctrl+A/Ctrl+X to increase/decreasenumbers



17/40

Packaging with quilt

about quilt

Quilt allows you to easily manage large numbers of patches by

keeping track of the changes each patch makes. Patches can be

applied, un-applied, refreshed, and more.



18/40

Packaging with quiltquilt setup

initialize a source tree from RPM spec file1 unpack tarball2 apply patches3

un-apply patches (but remembers the series)IOW: %prep phase

$ quilt setup -v foo.spec

$ cd foo-1.0.0

shortcut:

$ alias qs=quilt setup -v *.spec



19/40

Packaging with quiltworking with series of patches

now we are in the directory created by quilt setup (unpackedtarball)

none of the patches is applied (but quilt knows about them)quilt push applies one patch (-a: all patches)

quilt pop un-applies one patch (-a: all patches)

quilt top prints name of the last applied patch

quilt refresh refreshes last patch (so that it applies cleanly,without fuzz and offsets)

quilt series prints the names of all patches



20/40

Packaging with quiltinspecting patches

$ quilt diff

$ quilt diff --color=auto or quilt diff | colordiff

$ quilt files or quilt diff | lsdiff

$ quilt diff | diffstat

$ quilt diff -U num

view differences in vimdiff (side by side):

$ alias qdv=quilt diff -p0 -C 9999 --color=never --diff=vimdiff



21/40

Packaging with quiltcreating new patch

1 quilt push -a

2 quilt new

foo-fix-bug.patch

3 quilt add src/file.c

4 vim src/file.c

5 hack, hack

6 quilt refresh

steps 3 and 4 can be

combined toquilt edit src/file.c

the patch appears inparent directory (after

quilt refresh)you have to add it tospecfile



22/40

Packaging with quiltimporting patch

goal: import patch (e.g. taken from upstream) to quilt

1 quilt push -a

2 quilt import /path/to/bugfix.patch

3 quilt push

4 quilt refresh

bugfix.patch appears in the parent directory

-p num controls how many directory levels to strip (as withpatch(1))

-P name overrides name of the patch

multiple patches can be imported at once



23/40

Packaging with quiltrejected patches

when quilt push fails, use quilt push -f

applies as much as possible, leaves rejects (file.c + file.c.rej)$ quilt pushApplying patch gnupg-files-are-digests.patchpatching file g10/gpg.cHunk #3 succeeded at 1998 (offset -4 lines).Hunk #4 succeeded at 2487 (offset -4 lines).patching file g10/options.hHunk #1 FAILED at 194.

1 out of 1 hunk FAILED -- rejects in file g10/options.hpatching file g10/sign.cPatch gnupg-files-are-digests.patch does not apply (enforce with -f)



24/40

Packaging with quiltrejected patches - continued

failed hunks in *.c.rej have to be applied manually (vim *.c)

then quilt refreshhelper: vim quilt plugin (zypper install vim-plugin-quilt)(Contrib)

vim foo.c - if there is also foo.c.rej - opens vertically split view

easier to copy code from rejected hunks to original filethe plugin can do more: :help quilt



25/40

Packaging with quiltmake quilt more friendly

$ alias q=quilt

$ alias qd=quilt diff --color=always

$ alias qdv=quilt diff -p0 -C 9999 --color=never --diff=vimdiff

$ alias qp=quilt push --color=auto$ alias qs=quilt setup -v *.spec

$ cat ~/.quiltrc

# show C function name in hunk header

QUILT_DIFF_OPTS=-p



26/40

osc status, osc addremove

we have to tell osc about new/deleted files

example: update foo-1.0 to foo-2.0

$ ls foo*bz2

foo-1.0.tar.bz2 foo-2.0.tar.bz2

$ osc status? foo-2.0.tar.bz2

$ rm foo-1.0.tar.bz2

$ osc ar # osc addremove

D foo-1.0.tar.bz2

A foo-2.0.tar.bz2

$ osc st # osc status

D foo-1.0.tar.bz2

A foo-2.0.tar.bz2

osc add + osc remove = osc ar



27/40

Committing changes

osc diff displays changes in the package

more readable:

$ alias osccd=osc diff | colordiff | less

osc vc opens foo.changes file (prepares new entry header, unless-e option is given)

vim can open compressed tarballs:

:tabnew foo-2.0.tar.bz2search for Changelog, NEWS, . . . ; opencopy, paste and edit changelog entry

osc commit

osc sr



28/40

Vim and changelogs

to make changelog editing with vim more convenient, add to .vimrc:

set nocompatible

filetype plugin on

syntax on

autocmd BufRead,BufNewFile *.changes.* set filetype=changes

autocmd FileType changes set spelllang=en_us spell textwidth=67



29/40

osc bash completion

supports completing:

osc subcommand names (checkout, commit, . . . )global long osc options (e.g. --quiet, --debug, . . . )long subcommand options

project namespackage names (in given project)file names (in given project/package)repository names (openSUSE Factory, . . . ). . .

http://gitorious.org/opensuse/osc-bash-completiongit clone

git://gitorious.org/opensuse/osc-bash-completion.git

see README



30/40

osc bash completionexamples

osc com => osc commit

osc build --a => osc build --alternative-project

osc ls Ar => osc ls Archiving

osc ls Base:System pa => osc ls Base:System parted

osc meta a => osc meta attribute

osc request => osc request

accept approvenew co decline checkoutlist log revoke show wipe

...



31/40

Browsing source code

packager often needs to understand code written by somebody else

application crashespatch does not apply to new versionbuild fails

there are tools that significantly help:

ctags

cscope

both work with C/C++, ctags supports many other languageszypper install ctags cscope



32/40

Browsing source codectags

ctags creates database of symbols in the source code tree

editor (or other tools) use this database to quickly search for symbolsand jump to the definition

to create the database (tags file):

$ cd foo-1.0.0

$ ctags -R .

vim automatically uses ctags database if present

Ctrl+] / Ctrl+L click jump to definition of symbol under cursorCtrl+t / Ctrl+R click jump back:tag symbol jump to definition of symbolvim -t symbol open vim and jump directly to definition of symbol

:help ctags



33/40

Browsing source codevim + ctags + bash completion

sometimes you forget exact name of the symbol

bash completion becomes handy:

$ ctags -R .

$ vim -t sym => vim -t symbol_name_full

http://vim.wikia.com/wiki/Using bash completion with ctags and Vim

put it to ~/.bash completion

zypper install bash completion



34/40

Browsing source codecscope

ctags can only jump to symbol definition; cscope can do more

whith cscope, it is possible to quickly lookup e.g.

where is the symbol defined (same as ctags)where is the symbol usedwhat functions call this functionwhat functions are called by this functionwhich files include this file

. . .support in vim

http://cscope.sourceforge.net/cscope vim tutorial.html



35/40

Browsing source codetaglist

source code browser plugin for vim

provides overview of the structure of source code files

zypper install vim-plugin-taglist (Contrib)

set up taglist binding in .vimrc:

nnoremap :Tlist



36/40

Browsing source codetaglist screenshot



37/40

Patches from Fedora

do not reinvent wheel - before writing patch, make sure it does notalready exist

Fedora uses git now:

# Credits: Pavol Rusnak

function fedora_getpkg()

{

git clone git://pkgs.fedoraproject.org/${1}.git}



38/40

Patches from git...with tig

tig is ncurses interface for git (zypper install tig)

run tig from the cloned repository, press h for help

to find and extract patch form git repository, add the following to/.tigrc:

bind generic P !git format-patch %(commit)^..%(commit)

run tig (e.g. tig v2.2..)

select the patch you want to extract

hit P - the patch will be saved as sth like 0001-fix-name.patch



39/40

RPM macros

sometimes not obvious what rpm %macro doesrpmbuild --showrc lists all defined RPM macrosrpm -E %macro prints expanded macrosupported by bash completion:

$ rpm -E %la => rpm -E %lang_package

%package lang

Summary: Languages for package %{name}

Group: System/Localization

Requires: %{name} = %{version}

Provides: %{name}-lang-all = %{version}

Supplements: packageand(bundle-lang-other:%{name})

BuildArch: noarch

%description lang

Provides translations to the package %{name}



40/40

EOF

Questions?


Documents

Effective Packaging (2010). Petr Uzel