Anomos: anonymous BitTorrent

The following information is based on Anomos’ website. Funnily, when you go to that website, the default page is a blog that doesn’t answer the “What’s anomos?” question. The answer to that question is available on the About page, on this blog post, and those slides.

So, Anomos seems like a good answer to the “it’s to easy to determine who is donwloading a file with BitTorrent” problem. It:

  • uses a mix network (like Tor) to transfer all the content.
  • encrypts the file being transferred.

    It is based on BitTorrent for performance, however, performance might be seriously affected by the mix network (which probably don’t resist too well to heavy load since most internet connections have asymmetric upload/download bandwidths).

    A strange point is that it uses a central tracker. I’m not sure why the central tracker couldn’t be dropped, to simply use a DHT instead. (And traffic to the DHT could go through the mix network as well).

  • Make, tmpfs, and [amc]time granularity

    As you might have guessed (or not), I’ve been playing with the idea of improving my Debian rebuild setup by building on tmpfs: disk I/O take a signifiant amount of time, that could easily be reduced by building (mostly) in memory.

    Most packages build fine on tmpfs: only 8 packages fail only on tmpfs. Apparently, most of the failures are caused by old versions of autoconf: this version used /tmp to store temporary files ; for example, when generating Makefile, it would create it on /tmp, then move it to the current directory. The problem arises when /tmp is an ext3 filesystem, while the current directory is on tmpfs: when moving the file back to the current directory, the mtime of the file is truncated.

    This can lead make to think that a given target needs to be rebuilt. A nice example of that is the smuxi package, that just loops when building.

    Demo:

    # cat Makefile 
    f2: f1
            echo "Need to rebuild f2!"
    

    First case: f1 and f2 are stored in the current directory. Make agrees that
    f2 was created after f1:

    # rm f1 f2 ; touch f1 ; touch f2 ; make
    make: `f2' is up to date.
    

    Second case: f2 is created on /tmp, then moved to the current directory. Make thinks that f2 was created before f1 (and is right, according to stat).

    # rm f1 f2 ; touch f1 ; touch /tmp/f2 ; mv /tmp/f2 f2 ; make
    echo "Need to rebuild f2!"
    Need to rebuild f2!
    # stat f1 f2     
      File: `f1'
    [..]
    Modify: 2009-03-08 18:21:08.900237000 +0100
      File: `f2'
    [..]
    Modify: 2009-03-08 18:21:08.000000000 +0100
    

    Wysiwyg tool to create simple websites?

    Hi,

    I’m looking for a tool to create simple websites (5-10 pages), only static content (no PHP). Think of the typical small business or sports club website: text + pictures about what they did/do, nothing really complex.

    Requirements:

  • usable by a non-programmer who doesn’t want to know about gory technical details (HTML, CSS, etc)
  • doesn’t make it too hard to create a website with a few pages (i.e it should do a bit more than just allowing to create one page)
  • built-in support for (S)FTP uploading of the website, usable by a non-programmer after someone computer-litterate will have set it up. (that’s not an absolute requirement, but would still be nice to have. also, what are the nice graphical tools to do that, currently?)

    I’m asking for two different persons with different needs. So please don’t limit your replies to free software (although it would be nice): cheap proprietary applications might be adequate as well. Regarding platform, Linux, Mac OS X or Windows would all be suitable.

    I don’t think that a CMS is the right tool here: the focus should be put on the content (which won’t be often modified anyway) and its presentation, not on the ease of modification. For example, I’d like to be able to position pictures in a precise way. However, I might be interested in a CMS that is:

  • easy to setup (for a reasonably computer-literate person)
  • extremely easy to learn (a couple hours at most, for a non-programmer)
  • easy to transform into something that looks good

    In your comments, it would be great if you could include examples of what you did with those tools (yes, links are welcomed!)

    Thank you.

  • PhD !

    This blog post is long overdue, but like Lenny, it was finally released. Yes, I defended my PhD thesis on December 4th. So I’m officially a Doctor from the Universit√© Joseph Fourier Grenoble.

    The general context of my thesis is distributed systems, like peer-to-peer systems and applications running on HPC (High Performance Computing) cluster and grids. More specifically, I worked on methods and tools to study such systems, trying to answer the following questions: take a given application that will run on a distributed system. What can we learn or verify about its performance or its characteristics? Will it scale well, even with 1000 or 10000 nodes? Does this app perform better than this another app? Under which conditions? (that’s probably the hardest question, because you need to be able to run both apps under the same conditions if you want a fair comparison!)

    My work involved studying basic building blocks that can be used to study distributed systems, like network emulators (Dummynet, Linux’s traffic control subsystem, NISTNet). I also designed P2PLab, which allows to emulate network-centric peer-to-peer systems using a cluster, targetting experiments with thousands of emulated instances of the application. For example, it allowed to run experiments with ~15000 emulated BitTorrent nodes using 160 real machines.

    So what’s next? Well, my first-priority plan is to make use of my PhD to apply to an academic position (maitre de conf√©rences, i.e assistant professor). If you know an open position where I would be a good fit, don’t hesitate to tell me! Of course, this plan might very well fail (I will know around end of June). In that case, I’ll probably be looking for a job, likely Free-Software-related. Again, if you hear of something for me, ping me!

    Creating a large file without zeroing it: update

    Given the large number of comments I got (26!), I feel obliged to post a summary of what was said.

    First, the problem:
    I want to create a large file (let’s say 10 GB) to use as swap space. This file can’t be a sparse file (a file with holes, see wikipedia if you don’t know about sparse files).
    Since I’m going to mkswap it, I don’t care about the data that is actually in that file after creating it. The stupid way (but only solution on ext3) to create it is to fill it with zeroes, with is very inefficient.

    Theodore Tso provided more information in a comment, which I’m copying here:

    Yes, it will work on ext4. A convenient which makes this easy to use can be found here at http://sandeen.fedorapeople.org/utilities/fallocate.c. It was written by Eric Sandeen, a former XFS developer who now works for Red Hat, who has been a big help making sure ext4 will be ready for Fedora and Red Hat Enterprise Linux. (Well, I guess I shouldn’t call him a former XFS developer since he still contributes patches to XFS now and then, but he’s spending rather more time on ext4 these days.)

    One warning about the program; it calls the fallocate system call directly, and it doesn’t quite have the right magic architecture-specific magic for certain architectures which have various restrictions on how arguments need to be passed to system calls. In particular, IIRC, I believe there will be issues on the s390 and powerpc architectures. The real right answer is to get fallocate into glibc; folks with pull into making glibc do the right thing, please talk to me.

    Glibc does have posix_fallocate(), which implements the POSIX interface. posix_fallocate() is wired to use the fallocate system call, for sufficiently modern versions of glibc.

    However, posix_fallocate() is probablematic for some applications; the problem is that for filesystems that don’t support fallocate(), posix_fallocate() will simulate it by writing all zeros to the file. However, this is not necessarily the right thing to do; there are some applications that want fallocate() for speed reasons, but if the filesystem doesn’t support it, they want to receive the ENOSPC error message, so they can try some other fallback — which might or might not involve writing all zero’s to the file.

    The other shortcoming with posix_fallocate() is that it doesn’t support the FALLOC_FL_KEEP_SIZE flag. What this flag allows you to do is to allocate disk blocks to the file, but not to modify the i_size parameter. This allows you to allocate space for files such as log files and mail spool files so they will be contiguous on disk, but since i_size is not modified, programs that append to file won’t get confused, and tail -f will continue to work. For example, if you know that your log files are normally approximately 10 megs a day, you can fallocate 10 megabytes, and then the log file will be contiguous on disk, and the space is guaranteed to be there (since it is already allocated). When you compress the log file at the end of the day, if the log file ended up being slightly smaller than 10 megs, the extra blocks will be discarded when you compress the file, or if you like, you can explicitly trim away the excess using ftruncate().

    fallocate works fine: creating a 20 GB file is almost immediate. Also, syncing or umounting the filesystem is also immediate, and reading the file returns only zeros. I’m not sure how it is implemented, but it looks nice :-). However, it still doesn’t solve my initial problem: mkswap works, but not swapon:

    :/tmp# touch tmp
    :/tmp# /root/fallocate -l 10g tmp
    :/tmp# ls -lh tmp 
    -rw-r--r-- 1 root root 10G Mar  3 11:01 tmp
    :/tmp# du tmp 
    10485764	tmp
    :/tmp# mkswap tmp 
    Setting up swapspace version 1, size = 10737414 kB
    no label, UUID=a316ce8e-cf33-412b-8dc0-e10d9f2ebdbb
    :/tmp# strace swapon tmp
    [...]
    swapon("/tmp/tmp")                      = -1 EINVAL (Invalid argument)
    write(2, "swapon: tmp: Invalid argument\n"..., 30swapon: tmp: Invalid argument
    ) = 30
    exit_group(-1)
    

    (swapon works fine if the file is created normally — without using fallocate()).

    Any other ideas?