Help us get rid of dash build failures in Debian!

Following my rebuild of all packages using /bin/dash as /bin/sh, we now have a nice list of bugs.

Those bugs are cool:

  • The relaxed NMU rules apply to them, since they are part of the dash release goal. Which means that they can all be fixed using 0-day NMUs.
  • Most of them are really easy to fix. (for many of them, a patch is already provided by Ubuntu)

So, if you want to get involved in Debian development, try to submit NMU patches for those bugs. If you want me to sponsor the upload, please Cc me when you submit the patch.

And even if you don’t care about Debian, but only about Ubuntu (which is fondamentally wrong, as everybody knows that Ubuntu is based on Debian), fixing these bugs also helps Ubuntu a lot: all those bugs hurt Ubuntu, since Ubuntu uses dash as /bin/sh by default, and, even if the packages in Ubuntu have been patched, it’s still causing additional work every time a new package is uploaded to Debian, and has to imported in Ubuntu.

If you have questions with the process, please ask them using the comments below.

Notes:
I use the following process on those bugs, and it’s probably a good idea that you do the same if you intent to submit patches:

  • build the package with a chroot using bash
  • build the package with a chroot using dash (you can just use a different pbuilder tarball for that). Check that the problem can be reproduced.
  • fix the package
  • build the new package with a chroot using bash
  • build the new package with a chroot using dash
  • compare the built packages using debdiff for old_package_using_bash, new_package_using_bash, and new_package_using_dash. If the content doesn’t match, we have a problem.

If you submit NMU patches, please also have a look at the other bugs in the package, or at the lintian output. I don’t mind sponsoring small fixes for non-cosmetic/non-wishlist stuff at the same time.

Rebuilding the archive with dash as /bin/sh

I rebuilt all Debian packages twice. The first time with bash as /bin/sh, and the second time with dash as /bin/sh.

About 120 packages built fine with bash, but failed to build with dash. I filed all the bugs that weren’t filed already.

Then I debdiff’d the resulting binary packages, and found about 40 packages that built fine with bash and dash, but produced different binary packages (mostly files missing/being added)!

All the bugs (including those that were already filed) have been usertagged. Most of those bugs are quite easy to reproduce and fix, so, if you are bored, send patches!

I was a bit surprised by the high number of problems I found, since Ubuntu has been using dash as /bin/sh for more than a year, even on their buildds. Some packages have been fixed in Ubuntu, but the Debian maintainers didn’t include the fix. But many packages were simply broken in Ubuntu.

Now I’ll try to find time to work on the Build Daemon From Hell idea, and on rebuilds inside qemu.

Ubuntu giving back to Debian: facts and numbers!

I’ve always been annoyed by the discussions about “is Ubuntu really giving back to Debian?”. Debian Developers usually don’t see a lot of “giving back”, and Ubuntu Developers complain about Debian Developers ignoring their bug reports and patches.

So, a few months ago, I proposed that Ubuntu developers use a usertag when they report bugs to the Debian BTS, so they can be tracked.

Results are available:

Comments:

  • It’s really good to see Ubuntu developers reporting bugs and contributing patches to Debian!
  • … But more bugs (and patches) would be better, of course. Let’s continue the good work!
  • Many patches are applied very fast in Debian (as usual), but in some cases, the patches are ignored (as usual, too). It would be great if Debian Developers could treat those bugs as higher priority, since it makes life easier on the Ubuntu side (less difference means less work)

If you are an Ubuntu Developer, read wiki.u.c/Bugs/Debian/Usertagging for the details on how to tag the bugs you file. Note that the submittodebian script in the ubuntu-dev-tools package already sets the usertags.

Ubuntu: the Not Universal Operating System?

Debian describes itself as the Universal Operating System, and basically tries to run in the same way everywhere, from an NSLU2 to an s390.

I find it funny to see the number of Ubuntu variants. I can understand that for marketing reasons (Debian doesn’t really know about marketing), it is good to advertise Kubuntu, Xubuntu, etc. Even if it’s the same distro and the same packages. But seriously. An Xubuntu EEE now.

I’m the happy owner of an Asus EEE PC, running Debian. Installing Debian on it is quite straightforward, and only requires you to install the module for the ethernet NIC (available as Debian package), to compile the madwifi driver (non-free, patched version needed) for Wifi, and hack a few scripts in /etc/acpi if you want to get the hotkeys (volume, etc) working. Nothing impossible, really. And for those not familiar with the command line, some detailed documentation could be written, or a Debian package or a script that does all the tuning could be provided. (I think all of this has been done already anyway)

But an new Ubuntu derivative! Just for that! Are we going to see a new Ubuntu derivative for each new laptop that is not perfectly supported yet? Hey, my desktop PC’s keyboard has some hotkeys that don’t work properly without hacking .xmodmaprc. Can I get my Ubuntu derivative? :) Also, GNOME works perfectly on the EEE, and I assume KDE does as well. What about Ubuntu EEE, and Kubuntu EEE?

Now, let’s watch the flow of blog posts from Ubuntu fans acclaiming the release of the Xubuntu EEE.

Comments closed, trackbacks open.

Re: Is it hard for new contributors to help Debian? Can we improve things a bit?

Andreas Schuldei writes about my blog post about making it easier to contribute to Debian:

It is much more gratifying for a contributor if his effords have
immediate effects.
[…]
So what can people who want to help Debian do to achive more imminent gratification? Pick projects that let you help directly with direct svn/git/whatnot access (Security team, Debian Edu, Debian-Installer…)

While I agree that Instant Gratification is very important, I don’t think that the proposed solutions solve anything. It’s easy to have things sleep in an SVN repository instead of the BTS (lots of teams do that, sometimes for good reasons, like the lack of sponsors – hint: Games team). It doesn’t make things any better. What we need is tasks whose results will be available without too much wait in Debian unstable, for everybody.

Also, I must admit that I know very little about the inner workings of the Security Team. But I have the idea that it involves quite a lot of procedures (helping with bugs tagged security probably doesn’t, but going further than that probably does). Contributing to Debian already involves A LOT of procedures. A new contributor, even very good technically, can easily get lost in all the different ways to package stuff and solve common problems. So we should identify tasks that don’t have huge requirements. Ubuntu has the concept of bite-size bugs.

I started working on a page explaining the different ways someone can contribute to Debian. The goal is to provide a good entry point (not something that replaces existing documentation) and keep it at a manageable size. Its focus is mainly to replace the emails we write to people asking “I’d like to contribute, but I don’t know what I can do or where I should start!”. The page is available on wiki.d.o/HelpDebian/Start. Don’t hesitate to improve it (see also wiki.d.o/HelpDebian for a TODO list). When it will be good enough, I plan to move it out of the wiki (the content shouldn’t change much anyway) to eg http://help.debian.net/, so we can improve the design (I really like the idea of adding pictures of past Debconfs like Christian Perrier did in his FOSS.in talk).

Is it hard for new contributors to help Debian? Can we improve things a bit?

Debian could use more manpower, but is it actually easy for new contributors who would like to help to do useful things, and get the impression that they actually improved Debian?

There are several problems here:

  • Find bugs that are not too hard to fix. I don’t know about your packages, but in mine, there are some bugs that are easy to fix, and it’s just a matter of finding the time. If someone provided a patch, I would be very happy and apply it or provide feedback as fast as possible.
  • Find packages with active maintainers. It is no secret that if you choose the wrong package, your patch could very well sleep on the BTS for years.
  • Find bugs that actually matter. It’s not strictly a requirement. But fixing bugs in packages that should probably be removed from Debian instead is not really rewarding.

I think that other projects are better than us at providing directions to newcomers. Gnome has the whole Gnome Love thing, for example.

I’m thinking about building lists of “interesting” bugs. That would include RC bugs without patches, of course, but could also include a list of bugs based on an usertag that would mean “This bug is suitable for new contributors (it’s not too hard, and you could learn things fixing it). If you want to work on this bug, I’ll provide help and feedback and integrate your patch ASAP.

(This is very different from the “help” tag, which usually means “I couldn’t fix this, it’s too hard, I need help.“. It’s a bad idea to direct newcomers to bugs that are too hard for the package’s maintainer!)

Additionally, a nice feature would be to be able to filter the list of bugs with the packages that the user has installed.

  • What do you think?
  • As someone willing to contribute to Debian, do you often find it hard to find stuff you could do?
  • As a maintainer, do you have a lot of bugs that could use such a tag? Would you use it?

Comments are open, please us them ;)

Re: Two years of Python

Andrew writes about Python, Perl and Ruby:

Ruby has some nice things (like Perlish regular expression handling), but it brings back all that punctuation noise again.

He gives an example of punctuation noise in Perl:

I go back to Perl and my eyes bleed after trying to dereference a reference to a scalar, or something like that. It’s just ugly in Perl.

I don’t think that’s fair to compare Perl’s ponctuation noise with Ruby. Ruby has a feature that could qualify as punctuation noise, but it’s really a feature: variables are prefixed with their scope. For example, class variables start with @@ (@@var), instance variables start with @ (@var), global variables start with $ ($var), and constants are in CAPS.

I don’t know if Ruby “invented” that, or if it comes from another language (it’s probably the case: Ruby takes a lot of good ideas everywhere).

Also, Andrew, joining elements of an array in ruby is written myarray.join(” “), and Ruby has =~. So you should definitely give Ruby a try ;) No, seriously, Ruby is a really interesting language. It’s really a shame that it’s still so japanese-centric (most development decisions are taken on a japanese mailing list !!), since it’s not really help a wide adoption.

Where is the NM bottleneck?

The current status of the NM process, with 1 NM awaiting FD approval, 7 awaiting DAM approval, and 30 waiting for their accounts to be created, leads people to thinking that the big bottleneck is the account creation stage. However, when you look at what happened since december 2005, it’s not that simple.

NM queues status

See the graph linked above, which shows where people are waiting. While it seems that the FD stage hasn’t been blocking people for nearly a year, it’s the DAM stage which has been the biggest blocker. Indeed, on average, 8.3 NMs have been blocked by FD, 16.2 by DAM, and 9.0 by account creation.

I understand that processing NM reports in batch mode makes it more efficient. However, I’m wondering if processing them 20 per 20, is really that more efficient than processing them 5 per 5, which would justify such long queues. The NEW queue, which used to be a problem, is now being processed on a very regular basis, and hasn’t grown widely recently, except after Debconf when the ftpmaster processing NEW was on VAC. Couldn’t we have the same thing for DAM, account creations, and while we are at it, removals from unstable? Having clearly defined human-crontab-jobs would certainly make working on Debian less frustrating.

SC07, MIT Fab labs, and empowering people

This morning’s SuperComputing’07 keynote was a talk by Neil Gershenfeld, director of the Center for Bits and Atoms at the MIT. He mentioned the MIT Fab Labs, a project to bring “Personal Fabrication” to people around the around.

At our own level, that’s something I find very exciting with Free Software: it empowers people to do things with their computers that they couldn’t do if they were using proprietary software, simply because proprietary software is built to fit most people’s needs, not very specific goals some people could have. (and this is similar to the Long Tail stuff, in some ways)

Also, if you are at SC07 and reading this, feel free to ping me! I haven’t seen a lot of Debian/Ubuntuers here, even if the Free/Proprietary software ratio is very high.