Compiling Software that Uses the GNU Build System (autoconf, automake, etc.)

It used to be that almost all open-source software for Linux used the GNU build system. Anytime you saw that ubiquitous 'configure' script in your source code directory, you knew what you were in for. Recently, though, it seems that use of autoconf/automake/etc. has declined, and I'm seeing more use of cmake, ninja-build, and other newer tools.

Now, that doesn't mean that these tools have completely disappeared, like imake seems to have. And, because of this, it seems worthwhile to write up a quick guide on what to do, as a builder-of-software, when you see that 'configure.ac' file, in a source code repository. I'm not going to go into depth, here, so if you would like to read more, you can check out Gentoo Linux's excellent overview.

I'm going to use the diskfit tool as an example, since I ran across it, and went through the process, earlier today. When you clone the repository from GitHub, you see this collection of files:

$ git clone https://github.com/velnias75/diskfit.git
Cloning into 'diskfit'...
remote: Enumerating objects: 1647, done.
remote: Total 1647 (delta 0), reused 0 (delta 0), pack-reused 1647 (from 1)
Receiving objects: 100% (1647/1647), 340.88 KiB | 4.94 MiB/s, done.
Resolving deltas: 100% (1201/1201), done.
$ cd diskfit
$ ls -l
total 56
-rw-r--r--. 1 ajacocks ajacocks  3456 Oct 14 14:22 configure.ac
-rw-r--r--. 1 ajacocks ajacocks 35147 Oct 14 14:22 COPYING
-rw-r--r--. 1 ajacocks ajacocks   255 Oct 14 14:22 diskfit.bc.in
-rw-r--r--. 1 ajacocks ajacocks   140 Oct 14 14:22 diskfitrc
-rw-r--r--. 1 ajacocks ajacocks   484 Oct 14 14:22 Makefile.am
-rw-r--r--. 1 ajacocks ajacocks   826 Oct 14 14:22 README.md
drwxr-xr-x. 4 ajacocks ajacocks   103 Oct 14 14:22 src

You'll notice that there is a 'configure.ac' file, but not a 'configure' script. This means that we need to build it ourselves. The process is actually very easy, but it was a real pain to find a good explanation. I did finally find one, on stackoverflow, which explains the process very well. Just in case this disappears, I'll repeat it, here:

In an autoconf/automake/libtool project you need to run:

• libtoolize: this copies/links a few support scripts, including ltmain.sh (which is the main component of libtool).
• aclocal: this looks up all m4 macros that your configure script will need, and make a local copy for easier access.
• autoheader: optional, if you want to use config.h/AC_CONFIG_HEADERS, otherwise all the test result macros will be inlined when you call the compiler.
• autoconf: to expand all the macros used by configure.ac into the configure script.
• automake: to convert all the Makefile.am into Makefile.in templates. You probably want to invoke this with --add-missing so additional support scripts can be linked/copied to your project (such as compile, missing, depcomp, test-driver, etc).

Don't worry about running each tool. Just invoke autoreconf -i and it'll run the tools that are needed. Add -v if you want to see what tools is being executed. To avoid mistakes, just put a script like this at the root of your project:

#!/bin/bash -x
mkdir -p m4
exec autoreconf --install "$@"

Users that checkout/clone the project directly from the source repository will need to run this ./bootstrap script at least once. This is not needed if the user got a tarball distribution.

Automake can take fairly good care of itself; it'll re-invoke the above tools when needed, when you run make. But if you generate a broken Makefile, you'll need to invoke ./bootstrap and ./configure again to generate new Makefiles.

This is a far better explanation than I could have given. Thanks, DanielKO!

To continue our build process, we just need to do what DanielKO said in his snippet:

$ mkdir -p m4
$ autoreconf --install
libtoolize: putting auxiliary files in '.'.
libtoolize: copying file './ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
configure.ac:19: warning: The macro `AC_PROG_CC_C99' is obsolete.
configure.ac:19: You should run autoupdate.
./lib/autoconf/c.m4:1659: AC_PROG_CC_C99 is expanded from...
configure.ac:19: the top level
configure.ac:103: warning: AC_OUTPUT should be used without arguments.
configure.ac:103: You should run autoupdate.
configure.ac:9: installing './compile'
configure.ac:9: installing './config.guess'
configure.ac:9: installing './config.sub'
configure.ac:10: installing './install-sh'
configure.ac:10: installing './missing'
src/Makefile.am: installing './depcomp'

You will now see that 'autoreconf' has created a number of new files, in particular the 'configure' script that we will need to build the program: 

$ ls -l
total 1132
-rw-r--r--. 1 ajacocks ajacocks  70520 Oct 14 14:37 aclocal.m4
drwxr-xr-x. 2 ajacocks ajacocks    150 Oct 14 14:37 autom4te.cache
-rwxr-xr-x. 1 ajacocks ajacocks   7400 Oct 14 14:37 compile
-rwxr-xr-x. 1 ajacocks ajacocks  49939 Oct 14 14:37 config.guess
-rw-r--r--. 1 ajacocks ajacocks   4070 Oct 14 14:37 config.h.in
-rwxr-xr-x. 1 ajacocks ajacocks  35796 Oct 14 14:37 config.sub
-rwxr-xr-x. 1 ajacocks ajacocks 506842 Oct 14 14:37 configure
-rw-r--r--. 1 ajacocks ajacocks   3456 Oct 14 14:22 configure.ac
-rw-r--r--. 1 ajacocks ajacocks  35147 Oct 14 14:22 COPYING
-rwxr-xr-x. 1 ajacocks ajacocks  23568 Oct 14 14:37 depcomp
-rw-r--r--. 1 ajacocks ajacocks    255 Oct 14 14:22 diskfit.bc.in
-rw-r--r--. 1 ajacocks ajacocks    140 Oct 14 14:22 diskfitrc
-rwxr-xr-x. 1 ajacocks ajacocks  15358 Oct 14 14:37 install-sh
-rw-r--r--. 1 ajacocks ajacocks 332808 Oct 14 14:37 ltmain.sh
drwxr-xr-x. 2 ajacocks ajacocks    104 Oct 14 14:37 m4
-rw-r--r--. 1 ajacocks ajacocks    484 Oct 14 14:22 Makefile.am
-rw-r--r--. 1 ajacocks ajacocks  31261 Oct 14 14:37 Makefile.in
-rwxr-xr-x. 1 ajacocks ajacocks   6878 Oct 14 14:37 missing
-rw-r--r--. 1 ajacocks ajacocks    826 Oct 14 14:22 README.md
drwxr-xr-x. 4 ajacocks ajacocks    122 Oct 14 14:37 src

Now that we have what we need, we can execute 'configure' to build the Makefiles needed to successfully compile 'diskfit':

$ ./configure --prefix=/opt/diskfit
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking how to print strings... printf
checking for gcc... gcc

(trimmed for brevity)

config.status: creating src/python/qdiskfit/util/Makefile
config.status: creating config.h
config.status: executing libtool commands
config.status: executing depfiles commands

And now, we compile:

$ make
make  all-recursive
make[1]: Entering directory '/home/ajacocks/src/diskfit'
Making all in src
make[2]: Entering directory '/home/ajacocks/src/diskfit/src'
Making all in lib
make[3]: Entering directory '/home/ajacocks/src/diskfit/src/lib'
/bin/sh ../../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I../..  -DHAVE_INLINE -DGSL_C99_INLINE  -fvisibility=hidden -ffast-math -fstrict-aliasing  -finline-functions -g -O2 -MT libdiskfit_la-libdiskfit.lo -MD -MP -MF .deps/libdiskfit_la-libdiskfit.Tpo -c -o libdiskfit_la-libdiskfit.lo `test -f 'libdiskfit.c' || echo './'`libdiskfit.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../.. -DHAVE_INLINE -DGSL_C99_INLINE -fvisibility=hidden -ffast-math -fstrict-aliasing -finline-functions -g -O2 -MT libdiskfit_la-libdiskfit.lo -MD -MP -MF .deps/libdiskfit_la-libdiskfit.Tpo -c libdiskfit.c  -fPIC -DPIC -o .libs/libdiskfit_la-libdiskfit.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../.. -DHAVE_INLINE -DGSL_C99_INLINE -fvisibility=hidden -ffast-math -fstrict-aliasing -finline-functions -g -O2 -MT libdiskfit_la-libdiskfit.lo -MD -MP -MF .deps/libdiskfit_la-libdiskfit.Tpo -c libdiskfit.c -o libdiskfit_la-libdiskfit.o >/dev/null 2>&1

(trimmed for brevity)

/usr/bin/sed -e 's|@bindir[@]|/opt/diskfit/bin|g' < diskfit.bc.in > diskfit
make[2]: Leaving directory '/home/ajacocks/src/diskfit'
make[1]: Leaving directory '/home/ajacocks/src/diskfit'

And now, we install:

$ sudo make install
Making install in src
make[1]: Entering directory '/home/ajacocks/src/diskfit/src'
Making install in lib
make[2]: Entering directory '/home/ajacocks/src/diskfit/src/lib'
make[3]: Entering directory '/home/ajacocks/src/diskfit/src/lib'
 /usr/bin/mkdir -p '/opt/diskfit/lib'
 /bin/sh ../../libtool   --mode=install /usr/bin/install -c   libdiskfit.la '/opt/diskfit/lib'

(trimmed for brevity)

 /usr/bin/install -c -m 644 README.md '/opt/diskfit/share/doc/diskfit'
make[2]: Leaving directory '/home/ajacocks/src/diskfit'
make[1]: Leaving directory '/home/ajacocks/src/diskfit'

The results:

$ tree /opt/diskfit
/opt/diskfit
├── bin
│   └── diskfit
├── etc
│   └── diskfitrc
├── include
│   └── diskfit
│       └── diskfit.h
├── lib
│   ├── libdiskfit.a
│   ├── libdiskfit.la
│   ├── libdiskfit.so -> libdiskfit.so.1.0.2
│   ├── libdiskfit.so.1 -> libdiskfit.so.1.0.2
│   └── libdiskfit.so.1.0.2
└── share
    ├── doc
    │   └── diskfit
    │       └── README.md
    └── man
        └── man1
            └── diskfit.1

11 directories, 10 files

I hope that this helps someone, or in all honestly, me. This is something that I've had to research every time that I have done it, since I don't do it often, and my memory isn't all that good.