BASIS IntroductionBuild system And Software Implementation Standard
by Andreas Schuh and Andrew Hundt
Copyright © 2011 University of Pennsylvania. Copyright © 2013 Carnegie Mellon University.
2
BASIS Introduction Outline Introduction
Project Template
CMake Modules
Utilities
Milestones
Tutorials11/22/2013
4
BASIS Goals
The lifetime of your software should be longer than the time you or your team spends on it. Therefore BASIS aims to: establish a unified software structure. ease package integration. streamline maintenance. Make software more robust and reliable. Increase the impact of your work!
11/22/2013
5
Approach
Specify standards/conventions for the software development process.
Standardize project organization and software build including testing and packaging.
Provide basic utility functions in order to promote compliance with lab conventions and to reduce common development burden.
11/22/2013
6
Vision
11/22/2013
BASIS
…
Toolkit
ODVBA
COMPAREPREDICT
GLISTR
MICO
Integrated Distribution Package
Projects• COMPARE• PREDICT• …
Toolkit
BASIS
7
Ah, another Toolkit…
BASIS is not simply a toolkit. It consists of:
Standardized project specifications on the website
A preset project template
Utility functions for integration between multiple supported programming languages.
However, the distinction may be fuzzy at times…11/22/2013
8
BASIS Overview
11/22/2013
Standard Specifications
Template
File
syst
em
H
iera
rchy
Tem
pla
te F
iles
CMake Modules
Basi
sPro
ject
.cm
ake
(CM
ake
)
Basi
sTest
.cm
ake
(CTe
st)
Basi
sPack
.cm
ake
(CPa
ck)
Tools
Pro
ject
Cre
ati
on
Auto
mate
d
Test
ing
Docu
menta
tion
Utilities
Com
mand
Exe
cuti
on
Com
mand-l
ine
Pars
ing
Test
ing
Fram
ew
ork
s
BASIS IntroductionCopyright (c) 2011 University of Pennsylvania. All rights reserved. 9
Project TemplateMad-libs style project creation
11/22/2013
10
basisproject
basisproject is a command line program that: Automatically generates a package source
directory Populates it with starter files using mad-lib
style substitution to customize it for you in seconds.
11/22/2013
11
Template
A directory with all mad-libs style template files, i.e., the project files before substitution
Used by basisproject Follows the BASIS project layout
standard For an example, see the
data/templates subdirectory of the BASIS source package.
11/22/2013
BASIS IntroductionCopyright (c) 2011 University of Pennsylvania. All rights reserved. 12
<source> is the root directory of the source tree.
<project> is the project name in lowercase only.
Source Code Filesystem Hierarchy
<source>/ config/
data/
doc/exampl
e/include/ <projec
t>/src/
test/
11/22/2013
Installation Filesystem Hierarchy
11/22/2013
BASIS Introduction Copyright (c) 2011 University of Pennsylvania. All rights reserved. 13
<prefix> is the installation directory, e.g., /usr/local/ on Unix.
<sinfix> is a project- and possibly version-specific string, e.g., <project>
<prefix>/
bin/ <sinfix>/
include/ sbia/ <project>/
lib/
cmake/ <project>/
<sinfix>/
perl5/ 5.x.x/ <project>/
pythonx.x/ site-packages/ <project>/
share/ <sinfix>/
doc/
example/
…
14
Filesystem Hierarchy
Main executables go in <prefix>/bin/<sinfix>/
Auxiliary executables go in <prefix>/lib/<sinfix>/.
Symbolic links created upon installation by default From
▪ <prefix>/bin/ To (selected) executables in
▪ <prefix>/bin/<sinfix>/.
More details in the specification.11/22/2013
16
BASIS extends and replaces most CMake commands. add_executable() basis_add_executable()
Supports multiple languages, not just C++: Java (not yet), Python, Perl, BASH, and MATLAB.
Implements the BASIS standard, adhering guidelines Filesystem hierarchy Software documentation Packaging Testing
CMake Modules
11/22/2013
17
CMake Modules
The main modules are: BasisProject.cmake▪ Must be part of every project. Defines meta-data.
BasisTest.cmake – Extends CTest.cmake.▪ Used for software tests.▪ Included by basis_project_initialize().
BasisPack.cmake – Extends CPack.cmake.▪ Used for packaging.▪ Included by basis_project_finalize().
11/22/2013
18
CMake Modules
The most important functions to know are:
basis_find_package() basis_add_executable() basis_add_library() basis_add_test() basis_add_doc() basis_include_directories() basis_target_link_libraries() basis_set_target_properties() basis_get_target_property() basis_install()
11/22/2013
19
CMake Example
<source>/config/Depends.cmake# find NIfTI-1 tools for MATLABbasis_find_package (MatlabNiftiTools REQUIRED)
basis_add_include_directories (“${MatlabNiftiTools_INCLUDE_DIRS}”)
<source>/src/CMakeLists.txt# MEX-file build from C++basis_add_library (look_for_gold_instead MEX look_for_gold.cxx)
# executable build from MATLAB scriptsbasis_add_executable (miner neurominer.m)basis_target_link_libraries (miner look_for_gold_instead)
11/22/2013
20
CMake Modules
See the API documentation of BASIS for details on the CMake variables, functions, and macros.
Documentation can be automated so it is generated automatically every night (less frequently in later stages) from the in-source code comments using Doxygen.
11/22/2013
22
basisproject
Create a new BASIS project
Modify an existing project.
Upgrade a project to a newer version of BASIS.
3-way file merge combines your changes with the changes in the newer BASIS template.
Uses TEMPLATE argument of basis_project to determine currently used template version.
11/22/2013
23
basistest
Wrapper for command-line tools implemented for the automated execution of software tests:
The basistest_cron is executed, e.g., every 5 minutes. It in turn just calls basistest_master with the
arguments specific to the given system. A configuration file tells the basistest_master what
branches of which projects to test at which time interval,…
The actual tests are executed by instances of basistest_slave which are submitted to SGE in our case and am using CTest.11/22/201
3
24
basistest
basistest can create CDash dashboard of test results for two testing regimens: Continuous▪ Executed after a change was committed to the
repository.
Nightly▪ Run once every certain time interval, e.g., every
night.
11/22/2013
25
Documentation Standard
BASIS automates the process of setting up documentation
Doxygen is used for general code documentation Natively, Doxygen knows how to parse C/C++, Java, and Python
(not satisfactory, though)
Filters can be used to translate other languages into something Doxygen understands best: C++. BASIS includes such filters for the missing languages, i.e.(, Python), Perl, BASH, and MATLAB.
This is transparent to the BASIS developer.
A simple basis_add_doc(api GENERATOR Doxygen) in the build CMakeLists.txt is sufficient.
11/22/2013
26
Documentation Standard
See the API documentation of the BASIS utilities for examples of documentation generated fromin-source comments only.
11/22/2013
28
Command Execution Standard
Standardizes application startup and conventions for programs to interact.
Main commands commonly execute several utility commands in *nix and research environments.
Also need to ensure the right commands are executed and the correct libraries are loaded.
11/22/2013
29
Command Execution Standard Calling Conventions aim to ensure
executables are referenced by build target name rather than the filename to avoid ambiguity and errors.
BASIS provides utility functions for each supported language to do just that in a safe and reusable way.
11/22/2013
30
Command-line Parsing
Any command-line tool needs to parse command-line arguments.
BASIS requires each executable to output a help screen on --help (-h)
a shorter usage information on ––helpshort and its version and copyright on --version.
11/22/2013
31
Command-line Parsing
BASIS provides command-line parsing libraries
Implemented for each supported language.
These libraries create a help screen from brief descriptions provided for each option. No need to format the help screen manually. Able to output a man page or XML description. Such help is more likely to be up-to-date rather
then being forgotten when an option was renamed, removed, or added…
11/22/2013
32
Command-line Parsing
C++ TCLAP Gflags Boost::program_options
Java args4j
Python argh (wrapper for argparse)
Perl Getopt::Tabular Getopt::Declare
BASH shflags (extended shFlags)
MATLAB none available yet11/22/201
3
See Command-line Parsing documentation on the BASIS website for more details.
33
Testing Frameworks
C++ gtest gmock libtap++ Boost.Test
Java JUnit JTap
Python unittest PyTap
Perl Test::Simple Test::More Test::MockObject
BASH shtap (slightly modified tap-functions library)
MATLAB MATLAB xUnit Test Framework
11/22/2013
Already part of Perl installation.
Test Anything Protocol (TAP) Producers
Common use and standardized output which can be processed, for example,by Test::Harness written in Perl.
Built into Python language.
35
Releases 0.1 through 2.1.4
See ChangeLog of BASIS distributed by SBIA
11/22/2013
36
Future Release: 3.0
BASIS Moves to GitHub and expands beyond the Section of Biomedical Image Analysis (SBIA) at the University of Pennsylvania.
3.0 strips of some SBIA specific defaults to allow more customization required by other organizations.
Everything organization specific shall be separated from the main functionality and be configurable.
Default templates will be available that should provide a good starting point for anyone.
11/22/2013
37
Check GitHub for the latest developments
Read More▪ https://github.com/schuhschuh/cmake-basis
Get the Source▪ git clone https://github.com/schuhschuh/cmake-basis.git
11/22/2013
Beyond the Horizon
38
Feedback
Remember, your participation and feedback is required to improve BASIS!
What do you like? What concerns you? What hinders you? What is missing?
Need tutorial-like examples to go more into details regarding certain aspects with hands-on examples
Post questions and ideas on GitHub Issue tracker▪ https://github.com/schuhschuh/cmake-basis/issues
Wiki▪ https://github.com/schuhschuh/cmake-basis/wiki 11/22/201
3