The Makefile utility

Motivation

Small programs single file

“Not so small” programs :

– Many lines of code– Multiple components– More than one programmer

Mozilla

Mozilla Directory Structure

One very large directory tree26,000 files2,500 subdirectories240 file types

One very large directory tree26,000 files2,500 subdirectories240 file types

Mozilla Directory References

1,521 directory referencesAvg. fan-in/fan-out 12Max fan-in=100Max fan-out=42Median fan-in=5Median fan-out=12

1,521 directory referencesAvg. fan-in/fan-out 12Max fan-in=100Max fan-out=42Median fan-in=5Median fan-out=12

Build-levelFile (atomic entity)

–Source–Documentation

Directory tree (container) = source tree

Build process–To build/install software–Driven by make/ANT/…

Configuration process–To control build process–Driven by configure/configuration files/…

Build-level interfaces–Build interface–Configuration interface

Mozilla Build Level

1,350 Makefiles40,000 LOC build instructions16,000 LOC configurationCyclic dependenciesTwo-phase build processCentralized build/configuration knowledgeCode duplicationComponent implementations scattered

1,350 Makefiles40,000 LOC build instructions16,000 LOC configurationCyclic dependenciesTwo-phase build processCentralized build/configuration knowledgeCode duplicationComponent implementations scattered

Motivation – continued

Problems:

– Long files are harder to manage

(for both programmers and machines)– Every change requires long compilation– Many programmers can not modify the

same file simultaneously– Division to components is desired

Motivation – continued

Solution : divide project to multiple files Targets:

– Good division to components– Minimum compilation when something is

changed– Easy maintenance of project structure,

dependencies and creation

Project maintenance

Done in Unix by the Makefile mechanism A makefile is a file (script) containing :

– Project structure (files, dependencies)– Instructions for files creation

The make command reads a makefile, understands the project structure and makes up the executable

Note that the Makefile mechanism is not limited to C programs

Project structure

Project structure and dependencies can be represented as a DAG (= Directed Acyclic Graph)

Example :– Program contains 3 files– main.c., sum.c, sum.h– sum.h included in both .c files– Executable should be the file sum

sum (exe)

sum.omain.o

sum.csum.h sum.hmain.c

Make: Header Dependencies What if …

– Sensor.cc changes?

– Sensor.h changes?

– Robot.h changes?

Pattern rule ignores header dependencies!

Requires unnecessary “make clean; make”

Let gcc figure out the header dependencies for us!

The –MM option produces make rules in a .d file which we can then include in the Makefile

Sensor.h Sensor.cc

Robot.h Robot.cc

robotest.cc

makefile

sum: main.o sum.o

gcc –o sum main.o sum.o

main.o: main.c sum.h

gcc –c main.c

sum.o: sum.c sum.h

gcc –c sum.c

Rule syntax

main.o: main.c sum.h

gcc –c main.c

tab

dependency action

Rule

Equivalent makefiles

.o depends (by default) on corresponding .c file. Therefore, equivalent makefile is:

sum: main.o sum.ogcc –o sum main.o sum.o

main.o: sum.hgcc –c main.c

sum.o: sum.hgcc –c sum.c

Equivalent makefiles - continued

We can compress identical dependencies and use built-in macros to get another (shorter) equivalent makefile :

sum: main.o sum.o

gcc –o $@ main.o sum.o

main.o sum.o: sum.h

gcc –c $*.c

make operation

Project dependencies tree is constructed Target of first rule should be created We go down the tree to see if there is a target

that should be recreated. This is the case when the target file is older than one of its dependencies

In this case we recreate the target file according to the action specified, on our way up the tree. Consequently, more files may need to be recreated

If something is changed, linking is usually necessary

make operation - continued

make operation ensures minimum compilation, when the project structure is written properly

Do not write something like:

prog: main.c sum1.c sum2.c

gcc –o prog main.c sum1.c sum2.c

which requires compilation of all project when something is changed

Make operation - example

File Last Modified

sum 10:03

main.o 09:56

sum.o 09:35

main.c 10:45

sum.c 09:14

sum.h 08:39

Make operation - example

Operations performed:

gcc –c main.c

gcc –o sum main.o sum.o

main.o should be recompiled (main.c is newer).

Consequently, main.o is newer than sum and therefore sum should be recreated (by re-linking).

Another makefile example# Makefile to compare sorting routines BASE = /home/blufox/baseCC = gccCFLAGS = -O –WallEFILE = $(BASE)/bin/compare_sortsINCLS = -I$(LOC)/includeLIBS = $(LOC)/lib/g_lib.a \ $(LOC)/lib/h_lib.aLOC = /usr/local

OBJS = main.o another_qsort.o chk_order.o \ compare.o quicksort.o

$(EFILE): $(OBJS)@echo “linking …”@$(CC) $(CFLAGS) –o $@ $(OBJS) $(LIBS)

$(OBJS): compare_sorts.h$(CC) $(CFLAGS) $(INCLS) –c $*.c

# Clean intermediate filesclean:

rm *~ $(OBJS)

Example - continued

We can define multiple targets in a makefile Target clean – has an empty set of

dependencies. Used to clean intermediate files.

make– Will create the compare_sorts executable

make clean– Will remove intermediate files

Passing parameters to makefile

We can pass parameters to a makefile by specifying them along with their values in the command line.

For example: make PAR1=1 PAR2=soft1

will call the makefile with 2 parameters: PAR1 is assigned the value “1” and PAR2 is assigned the value “soft1”. The same names should be used within the makefile to access these variables (using the usual “$(VAR_NAME)” syntax)

Passing parameters - continued

Note that assigning a value to a variable within the makefile overrides any value passed from the command line.

For example: command line : make PAR=1

in the makefile:PAR = 2

PAR value within the makefile will be 2, overriding the value sent from the command line

Conditional statements

Simple conditional statements can be included in a makefile.

Usual syntax is:

ifeq (value1, value2)

body of if

else

body of else

endif

Conditional statements - example

sum: main.o sum.ogcc –o sum main.o sum.o

main.o: main.c sum.hgcc –c main.c

#deciding which file to compile to create sum.oifeq ($(USE_SUM), 1)

sum.o: sum1.c sum.hgcc –c sum1.c –o $@

elsesum.o: sum2.c sum.h

gcc –c sum2.c –o $@

endif

Make: Advanced Options Text manipulation functions

– $(patsubst pattern,replacement,text)– $(patsubst %.o,%.cc,<list of objfiles>)

Pattern rules– Uses a pattern in the target with % as wildcard– Matched % can be used in dependencies as well– Simple Example:

%.o : %.cc <tab>command …

Pattern rules with automatic variables– $@ full target name– $< first dependency– $* string which matched % wildcard– Advance Example:

%.o : %.cc <tab>$(CC) $(CCFLAGS) –c $< $(INCPATHS)

Make: A Simple ExampleCC=g++ # Compiler to useFLAGS=-g # Compile flagsMASLAB_ROOT=maslab-software # Maslab software root directoryLIB_DIR=$(MASLAB_ROOT)/liborc # orc-related library directoryINC_DIR=$(MASLAB_ROOT)/liborc # orc-related include directoryLIBS=-lm –lpthread -lorc # Library files

all : helloworld

helloworld.o : helloworld.cc$(CC) $(FLAGS) –c $*.cc –o $@

helloworld: helloworld.o $(CC) -o helloworld helloworld.o $(LIBS)

clean: rm -f *.o helloworld

Make: Example MakefileMASLABROOT = /mnt/maslab/software/maslab-software-current

#This is the list of places to look for include filesINCPATHS = -I$(MASLABROOT)/libs/liborc \

-I$(MASLABROOT)/libs/libim \ -I$(MASLABROOT)/libs/libbotserver \ -I/opt/intel/ipp/include

#This is the list of places to look for librariesLIBPATHS = -L$(MASLABROOT)/libs/liborc \

-L$(MASLABROOT)/libs/libim \ -L$(MASLABROOT)/libs/libbotserver \ -L/opt/intel/ipp/sharedlib

#This is the names of the librariesLIBS = -lippipx -lippcvpx -lim -lorc -lbotserver -lm -lpthread -ljpeg -lpng

#This is the compiler to useCC = g++

# This is your c++ file extension (usually cc or cpp)CCEXT = cc

Make: Example Makefile (cont)CCFLAGS = -g -Wall

# This rule builds everything. You should put the names of all your

# programs in this list.

all : robotest

# This rule says how to turn any .cpp file into a .o file.

%.o : %.$(CCEXT)

$(CC) $(CCFLAGS) -c $< $(INCPATHS)

#----------------------------------------------------------------------

# robotest

PROGRAM_NAME = robotest

PROGRAM_OBJS = robotest.o BumpSensor.o Robot.o

GLOBAL_OBJS += $(PROGRAM_OBJS)

JUNK += $(PROGRAM_NAME)

$(PROGRAM_NAME): $(PROGRAM_OBJS)

$(CC) $(PROGRAM_OBJS) -o $(PROGRAM_NAME) $(LIBS) $(LIBPATHS)

chmod a+x $(PROGRAM_NAME)

Extending Example Makefile

#----------------------------------------------------------------------

# Sensor Incremental Test

PROGRAM_NAME = sensor-test

PROGRAM_OBJS = sensor.t.o BumpSensor.o

GLOBAL_OBJS += $(PROGRAM_OBJS)

JUNK += $(PROGRAM_NAME)

$(PROGRAM_NAME): $(PROGRAM_OBJS)

$(CC) $(PROGRAM_OBJS) -o $(PROGRAM_NAME) $(LIBS) $(LIBPATHS)

chmod a+x $(PROGRAM_NAME)

So to add a new executable program, simply cut and paste the robotest section and enter your own PROGRAM_NAME and PROGRAM_OBJS

Extending example MakefileDEPENDS += $(patsubst %.o, %.d, $(GLOBAL_OBJS))

JUNK += $(DEPENDS)

include $(DEPENDS)

depend : $(DEPENDS)

depend.$(CCEXT) = set -e; $(CC) $(CCFLAGS) -MM $(DEFS) $(INCPATHS)

depend.filt = sed 's/\($*\)\.o[ :]*/\1.o $@ : /g' > $@; [ -s $@ ] || rm -f $@

%.d: %.$(CCEXT)

$(depend.cc) $< | $(depend.filt)

#======================================================================

# Miscellaneous Rules

#----------------------------------------------------------------------

# This rule cleans your directory. You could also add commands here

# to remove program files and any other files that should be cleaned

clean:

rm -f $(JUNK) *~ *.o core.[0-9]* core

ANT:Another Nice Tool

What is Ant?

Java-based Build tool from Apache De facto standard for building, packaging, and

installing Java applications Accomplishes same objectives that make does on

Unix based systems Files are written in XML

Why Ant?

Many claim.. That – Unlike makefiles, Ant files work cross platform

- No need for multiple, complex makefiles depending on the operating system.- Tasks declared in platform independent way; Ant engine translates to OS specific commands.

But still need to know (potentially) complex path/install data. So not as general as they claim

Easy to create own Ant “tasks”, in addition to core tasks

Installing Ant

Download Ant binary distribution from:

http://ant.apache.org/bindownload.cgi Set ANT_HOME to where you installed Ant Include $ANT_HOME/bin in PATH Make sure JAVA_HOME is set to point to JDK

Running Ant

Type “ant” at the command line Automatically looks for build.xml file in current

directory to run Type “ant –buildfile buildfile.xml” to specify

another build file to run.

Ant Output

Sample build.xml<project name="MyProject" default="dist" basedir="."> <property name="src" location="src"/> <property name="build" location="build"/> <property name="dist" location="dist"/>

<target name="init"> <tstamp/> <mkdir dir="${build}"/> </target>

<target name="compile" depends="init" > <javac srcdir="${src}" destdir="${build}"/> </target>

<target name="dist" depends="compile" > <mkdir dir="${dist}/lib"/> <jar jarfile="${dist}/lib/MyProject-${DSTAMP}.jar" basedir="${build}"/> </target>

<target name="clean" > <delete dir="${build}"/> <delete dir="${dist}"/> </target></project>

Ant Overview: Project

Each build file contains exactly one project and at least one target

Project tags specify the basic project attributes and have 3 properties:

- name

- default target

- basedir Example: <project name=“MyProject” default=“build” basedir=“.”>

Ant Overview: Targets

Target is a build module in Ant Each target contains task(s) for Ant to do One must be a project default Overall structure of targets:

<target name="A"/>

<target name="B" depends="A"/>

<target name="C" depends="B"/>

<target name="D" depends="C,B,A"/>

Ant Overview: Tasks

Each target comprises one or more tasks Task is a piece of executable Java code (e.g.

javac, jar, etc) Tasks do the actual “build” work in Ant Ant has core (built in) tasks and the ability to

create own tasks

Ant Overview: Tasks

Example: <target name="prepare" depends="init“ >

<mkdir dir="${build}" /> </target>

<target name="build" depends="copy" >

<javac srcdir="src" destdir="${build}">

<include name="**/*.java" />

</javac> </target>

Ant Overview: Core Tasks

javac – Runs the Java Compiler java – Runs the Java Virtual Machine jar (and war) – Create JAR files mkdir – Makes a directory copy – Copies files to specified location delete – Deletes specified files cvs – Invokes CVS commands from Ant

Ant Overview: Writing Own Task

Create a Java class that extends org.apache.tools.ant.Task For each attribute, write a setter method that is

public void and takes a single argument Write a public void execute() method, with no

arguments, that throws a BuildException -- this method implements the task itself

Ant Overview: Properties

Special task for setting up build file properties: Example:

<property name=“src” value=“/home/src”/> Can use ${src} anywhere in build file to denote

/home/src Ant provides access to all system properties as if

defined by the <property> task

Ant Overview: Path Structures

Ant provides means to set various environment variables like PATH and CLASSPATH.

Example of setting CLASSPATH:

<classpath>

<pathelement path="${classpath}"/>

<pathelement location="lib/helper.jar"/>

</classpath>

Command Line Arguments

-buildfile buildfile – specify build file to use targetname – specify target to run (instead of

running default) -verbose, -quiet, -debug – Allows control over

the logging information Ant outputs -logger classname – Allows user to specify their

own classes for logging Ant events

IDE Integration

Eclipse, NetBeans, JBuilder, VisualAge, and almost any other Java IDE has Ant integration built-in to the system

Refer to each IDE’s documentation for how to use Ant with that IDE

Web Development with Ant

Tomcat comes with special Ant tasks to ease Web application development and deployment

Copy $TOMCAT_HOME/server/lib/catalina-ant.jar to $ANT_HOME/lib

Ant tasks for Tomcat:

- install

- reload

- deploy

- remove

Documentation/References

Good tutorial for makefiles http://www.gnu.org/manual/make-3.80/make.html

Good tutorial for cmakef

http://www.cmake.org/HTML/Documentation.html ANT User Manual:

http://ant.apache.org/manual/index.html

Sun’s Web development tutorial (Ant and JSPs):http://java.sun.com/webservices/docs/1.2/tutorial/doc/GettingStarted3.html

Standardized Build Interface

Different build systems exist, e.g., –make, ANT, shell scripts, IDE

Different software systems require different build actions, e.g., –make

–make bootstrap, make, make install

make cleanmake allmake checkmake (un)installmake dist

make cleanmake allmake checkmake (un)installmake dist

The real Problem

How do we handle platform specific issues?– Providing a different Makefile for each architecture– Using Autoconf, Automake and Libtool

The installer needs only– Bourne shell– C compilers– Make program

Standardized Configuration InterfaceDifferent mechanisms exist to control software

construction, e.g.,–configuration files, configuration tools, Makefile

editing

Standardized configuration interface to enable uniform compile-time configuration:

configure --helpconfigure --prefix=/usrconfigure --with-aterm=/usr/libconfigure --with-optimization=trueconfigure --with-debug=false

configure --helpconfigure --prefix=/usrconfigure --with-aterm=/usr/libconfigure --with-optimization=trueconfigure --with-debug=false

Explicit Context Dependencies

Dependencies on build-level components are declared in configuration interfaces

> configure --help … --with-aterm=DIR use ATerm Library at DIR --with-sglr=DIR use SGLR Parser at DIR …

> configure --help … --with-aterm=DIR use ATerm Library at DIR --with-sglr=DIR use SGLR Parser at DIR …

Independent Deployment

Build-level components are deployed as packages

A package is a versioned release of a build-level component, e.g.,

firefox-1.0.tar.gz

Packages are published on web/ftp sites

Third-party Composition

A configuration interface enables late-binding of dependencies

Compositions are thus not predefined

and can be defined by a third party

> configure --with-aterm=/usr/local/aterm --with-sglr=/usr/local/sglr --with-…=

> configure --with-aterm=/usr/local/aterm --with-sglr=/usr/local/sglr --with-…=

Some advantages when using GNU autotools

The installation of a program is straightforward:

./configure; make; make install

This procedure checks for system parameters, libraries, location of programs, availability of functions and writes a Makefile

./configure supports many options to overwrite defaults settings

GNU autoconf

autoscan

configure.scan

configure.ac(configure.in)

autoconf

configure

Source Code

aclocal

GNU automake

Makefile.am

automake

Makefile.in

configure

Makefile

configure.ac dnl Comment … …

AC_INIT(main.c)

AM_INIT_AUTOMAKE(project_name, 1.2.8)

AM_PROG_LIBTOOL it supports libtool and shared libraries

AC_PROG_CC (or AC_PROG_CXX) it locates the C (C++) compiler

AC_HEADER_STDC it checks for standard headers

AC_CHECK_HEADERS(sys/time.h /header.h) it checks for headers availability

AC_CHECK_LIB(crypto SSLeay_version) it checks for libraries availability

AC_CHECK_FUNCS(ctime) it checks for functions availability

AC_PROG_INSTALL it checks for BSD compatible install utility

AC_OUTPUT([Makefile])

Makefile.am

bin_PROGRAMS = foo/configure --prefix=… (default /usr/local)

foo_PROGRAMS=foo.c foo.h

noist_PROGRAMS=test(make compiles, make install does nothing)

EXTRA_DIST=disclaimer.txt

Example foo.c :

#include <stdio.h>main(){ printf(“Cum grano salis\n");}

Makefile.am :bin_PROGRAMS = foofoo_SOURCES = foo.c

configure.ac :AC_INIT(foo.c)AM_INIT_AUTOMAKE(latin_words, 0.9)AC_PROG_CCAC_HEADER_STDCAC_PROG_INSTALLAC_OUTPUT([Makefile])

Summary

Source Code, configure.ac, Makefile.am autoscan; aclocal; autoconf Create NEWS README AUTHORS ChangeLog

automake –add-missing ./configure; make; make dist Result: project_name-2.10.tar.gz

aclocal.m4 autom4te-2.53.cache ChangeLog config.status configure.in COPYING install-sh Makefile.am missing NEWS README AUTHORS autoscan.log config.log configure configure.scan INSTALL Makefile.in mkinstalldirs code.c

References GNU Autoconf, Automake, and Libtool http://sources.redhat.com/autobook/autobook/autobook_toc

.html

GNU Autoconf Manualhttp://www.gnu.org/manual/autoconf

GNU Automake Manualhttp://www.gnu.org/manual/automake

GNU Libtool Manualhttp://www.gnu.org/manual/libtool

Learning the GNU development toolshttp://autotoolset.sourceforge.net/tutorial.html

The GNU configure and build systemhttp://www.airs.com/ian/configure/configure_toc.html

GNU macro processor (GNU m4)http://www.gnu.org/manual/m4-1.4/m4.html