Make, tmpfs, and [amc]time granularity

March 8th, 2009 by lucas

As you might have guessed (or not), I’ve been playing with the idea of improving my Debian rebuild setup by building on tmpfs: disk I/O take a signifiant amount of time, that could easily be reduced by building (mostly) in memory.

Most packages build fine on tmpfs: only 8 packages fail only on tmpfs. Apparently, most of the failures are caused by old versions of autoconf: this version used /tmp to store temporary files ; for example, when generating Makefile, it would create it on /tmp, then move it to the current directory. The problem arises when /tmp is an ext3 filesystem, while the current directory is on tmpfs: when moving the file back to the current directory, the mtime of the file is truncated.

This can lead make to think that a given target needs to be rebuilt. A nice example of that is the smuxi package, that just loops when building.

Demo:

# cat Makefile
f2: f1
        echo "Need to rebuild f2!"

First case: f1 and f2 are stored in the current directory. Make agrees that
f2 was created after f1:

# rm f1 f2 ; touch f1 ; touch f2 ; make
make: `f2' is up to date.

Second case: f2 is created on /tmp, then moved to the current directory. Make thinks that f2 was created before f1 (and is right, according to stat).

# rm f1 f2 ; touch f1 ; touch /tmp/f2 ; mv /tmp/f2 f2 ; make
echo "Need to rebuild f2!"
Need to rebuild f2!
# stat f1 f2
  File: `f1'
[..]
Modify: 2009-03-08 18:21:08.900237000 +0100
  File: `f2'
[..]
Modify: 2009-03-08 18:21:08.000000000 +0100

Posted in PlanetDebian, PlanetUbuntu 4 Comments »

4 Responses to “Make, tmpfs, and [amc]time granularity”

  1. Marcus wrote on 03/8/09 at 11:16 pm :

    What’s the speedup using tempfs compared to normal ext3?

  2. Lucas wrote on 03/8/09 at 11:24 pm :

    13%, over the total build time, with big variations between packages.

  3. Ralf Wildenhues wrote on 03/10/09 at 7:23 pm :

    Which autoconf versions did you see this with, and if you still see it with current, could you please report it? Thanks.

  4. Lucas wrote on 03/10/09 at 7:47 pm :

    @Ralf: it was with ancient versions. An example is the dnstracer package, which ships a configure script generated by Autoconf 2.53. With newer versions, the temporary files are created in the current directory instead of /tmp, which solves the problem.