Feedback on Gandi hosting?

November 16th, 2010 by lucas

For the last few years, I’ve been a mostly happy customer of Sivit (now bought by Nerim). But recently, technical problems have become more frequent, and today, for the first time, a crash made me lose some data which was rather painful to restore from backups. Also, they are still stuck on with an ancient 2.6.16 kernel.

So, I’m looking into alternatives, and a promising one is Gandi hosting, based on virtual machines like Sivit.

  • Would you recommend it for a mission-critical server?
  • What’s your uptime? When was the last time your server had an unscheduled outage?
  • When was the last time you lost data because of Gandi?
  • What are the limits due to the Xen-based hosting? Can you run your own kernel (like the squeeze one)? Can you build additional modules (like tun)?
  • What else should I know?


Triaging X Strike Force bugs with UDD Bugs Search

November 3rd, 2010 by lucas

Thanks to the work of Julien Viard de Galbert, UDD Bugs Search just gained the ability to help the triaging of X Strike Force bugs.

He worked on the patch by installing his own copy of bugs.cgi on alioth. Don’t hesitate to do the same with your team!

Mini DebConf Paris 2010

October 31st, 2010 by lucas

So, I’m (almost) back from the first edition of MiniDebConf Paris, which was a success thanks to the great organizing skills in panic mode of Carl Chenet and Mehdi Dogguy. I think that everybody is already looking forward to the next edition.

I met lots of people, and it was great to be able to finally put faces on names from the french free software community. I gave two talk there, the first one on Debian and Ubuntu (slides), the second one on Debian Quality Assurance (slides), which both went well (I think).

I also hacked a bit:

  • I added tags and “claimed by” to UDD Bugs Search.
  • I worked on my piuparts replacement (which really needs a sexy name ; I’m open to suggestions), processing the results, making some improvements to the script, and filing 16 RC bugs.

If you want to help me, here are some easy tasks that need takers (it’s unlikely that I’ll be able to tackle them anytime soon):

  • One missing thing in UDD Bugs Search is information about the deferred/delayed queue. Unfortunately, the ftp-masters’ status information is not available in a machine-parseable format.
    • Improve dak code to also export the data in a machine-parseable format (JSON, YAML). I was told the changes are to be done in dak/
    • Write the corresponding UDD importer (actually, that’s a 30 mins job for me, so I could easily find time to do it if the data was available).

    Update: done!

  • Another missing thing is comments (as on I don’t really have good ideas about how to implement them, so if you want to tackle that with your own idea, go ahead. (It’s probably easy to setup a copy of bugs.cgi on alioth if you are not a DD). The CGI is in Ruby.
  • Help me process the logs of my piuparts replacement, and file the bugs. I don’t have any good scripts to automate bug filings yet. You need a good understanding of packages dependencies and maintainers scripts, but it’s a good way to learn, too. (I would need to regenerate the full results first, so talk to me if you are interested)

One of the other things I’d like to do is discuss ways to improve the synchronization of UDD (currently, it’s synchronized at arbitrary times, while we could sync Sources+Packages just after dinstall, for example).

Debian becoming upstream for ubuntu-dev-tools?

October 30th, 2010 by lucas

Launchpad bug #643691: Please sync ubuntu-dev-tools 0.103 (universe) from Debian experimental (main)

Did development of ubuntu-dev-tools move from Ubuntu to Debian? :-)

Reboot on kernel panic

October 19th, 2010 by lucas

Nice tip, I didn’t know about it.

How to demotivate people (Re: Making Debian Responsible For Its Actions)

September 29th, 2010 by lucas

In my Ruby packaging in Debian and Ubuntu: Mythbusting and FAQ, I wrote The Ruby developers (or expert users) community is generally very hostile towards Debian. Many harsh words and insults are used in discussions mentioning Debian in the ruby-core@ mailing list, and there are frequent recommendations to avoid the Debian packages and install from sources, which is quite demotivating (Actually, what prompted that blog post is someone calling Debian-specific changes unforgivable).

Zed A. Shaw just provided a very good example of that. Quoting the most juicy parts:

It’s simply a tactic to make sure that you are stuck on Debian.

If we look at this problem from an open source perspective, Debian is at a minimum being a horrible citizen.

What is happening is control is not aligned with responsibility. Debian has taken control of the software package and recrafted it for their very narrow set of system administrator users. However, they refuse to take responsibility for the defect and quality problems they create. Instead I end up being responsible for their mistakes, which is wrong.

If Debian is in control, then Debian should be made responsible. That’s the only way to get them to change it. If they don’t want to be responsible, then they should let me be in control.

I’m imagining a mass petition, maybe some funny ad campaigns, in-person confrontations meant to embarrass package maintainers, SEO tricks to get people off Debian, promotion of any alternative to Debian, anything to make them pay, apologize, and listen.

If you write software and you’re sick of Debian screwing up your gear, then shoot me ideas for how we can make this happen. I think that if enough of us band together to make Debian responsible for its actions we can actually get them to start asking us before they package our software to find out how we think it should be packaged.

On the technical part of the post:

The problem that Zed is experiencing is unknown to me, and also to the Debian and Ubuntu bug trackers. To my knowledge, rubygems doesn’t not require openssl or net/scp to work, and a simple grep through the sources kind-of confirms that. But of course, that blog post was written as a rant, not as a proper bug report.

My best guess is that the gem that Zed is installing requires openssl and net/scp. But of course, rubygems can’t just depend on all the libraries that applications or libraries installed via rubygems could possibly require. Regarding openssl, we had a problem in Ruby because of the licensing situation of Ruby (dual licensed GPLv2+ and Ruby license, linked with openssl, which might raise some eyebrows), so it was split off the main standard library package until recently (and, apparently, in the version Zed is using). For net/scp, it is not part of the standard library, and not a dependency of rubygems, so I don’t see why Zed expects it to be installed.

On the rest of the post:

Following my last post, several people volunteered to help with Ruby on Debian (and Ubuntu). In particular, Laurent Arnoud has been doing some fantastic work fixing the pkg-ruby-extras team’s RC bugs. Also, all the messages of support both in the comments of my last post were very much appreciated.

But I’m really wondering why the Ruby community generates so many poisonous people. All my work on Debian is done on a volunteer basis, and it’s really hard to get so much shit from it.

UDD bugs search news

September 18th, 2010 by lucas

Many changes to the UDD-backed Debian Bugs Search:

  • moved to
  • new filters:
    • Bugs affecting stable
    • Packages with outdated binaries in squeeze or sid
    • Packages with different versions in squeeze and sid (that need to migrate)
    • Packages which are newer in Ubuntu than in sid (good candidates for patches)
  • Much nicer layout
  • Some SQL optimization


Helping improve Ruby on Debian and Ubuntu

September 18th, 2010 by lucas

After my last blog post on Ruby in Debian and Ubuntu, I got a few offers for help, which is excellent.

So I’ve decided to update this wiki page with pointers to the various team resources, and some examples of tasks that newcomers can do.

I’d like to stress that many pending tasks are really doable by someone not very qualified in Debian packaging. They are not done just because nobody has the time to do them. Some good examples are the libinotify-ruby, libimage-size-ruby, libffi-ruby RC bugs.

Debian Bugs Search using UDD

September 14th, 2010 by lucas

After a few hours of hacking, I came up with Debian Bugs Search. This Ultimate Debian Database tool enables to query different kinds of Debian bugs (especially RC bugs using a multi-criteria search. It is aimed at being a replacement for the unofficial RC Bugs tracker, which has some bugs and fails to display the correct status for some bugs (Debian Bugs Search sees 450 RC bugs affecting squeeze, vs 278 for the unofficial RC bugs tracker).

While Debian Bugs Search still lacks some features that the unofficial RC bugs tracker has, it also brings some new features. The main advantage is that it makes it easy to find some classes of “interesting” bugs, for example:

RC bugs affecting squeeze and sid, tagged patch, sorted by time of last change. Why weren’t they fixed using the patch?

RC bugs affecting squeeze and sid, tagged pending, sorted by time of last change. Why weren’t they fixed by an upload if they were tagged pending a long time ago?

Bugs affecting squeeze but not sid, and marked as done. Why didn’t the fixed packages migrate to testing?

I’m open to suggestions of improvements, bug reports, or better, patches. Since it is UDD-based, it is quite easy to add more information. My own TODO list is:
- add popcon information (also enable sorting by popcon)
- add a way to display/hide additional columns (popcon, package’s versions)

Ruby packaging in Debian and Ubuntu: Mythbusting and FAQ

September 12th, 2010 by lucas

A lot is being said on how Ruby is packaged in Debian and Ubuntu. In this post, I’m trying to go through the most common myths and give my position as a Debian Ruby maintainer. Note that those are my views, not necessarily those of the other Debian Ruby maintainers.

Myth: Ruby is completely outdated in Debian and Ubuntu

There’s a culture in the Ruby community of always using the latest bleeding-edge version. There’s also a culture (though this one has recessed a bit, I think) that it’s perfectly fine to change APIs in incompatible ways if it makes the software slightly better.

In Debian (and Ubuntu) stable releases, we try to provide a rock-solid version of Ruby that won’t be allowed to change API during the support lifetime of a release. Bugfixes for important bugs can be applied, but that’s basically all. So it’s not surprising that Ruby in released versions of Debian and Ubuntu is behind the latest upstream version.

In development branches, we are following the latest upstream versions quite closely. Debian 6.0 ‘squeeze’, to be released by the end of the year, includes Ruby 1.8.7-p302, and will include 1.9.2-p0 (the currently of Ruby 1.9.2 in squeeze is currenty a SVN snapshot from beginning of august).

There are people interested in installing a packaged Ruby that just works, and other people interested in installing their own version of Ruby (through RVM and such). People in the Ruby community tend to forget about the first category, and consider that, because they install Ruby from source or with RVM, that should be the solution for everybody.

We need the Ruby community to respect that different people have different ways of using Ruby, and to stop recommending that everyone using Debian and Ubuntu should install Ruby from source.

We also need to leverage the backports services for both Debian and Ubuntu to provide a way to install the latest upstream version on stable releases. The only reason why we are not doing it yet is lack of time.

Myth: Ruby is split into a myriad of packages in Debian

In the past (pre-2005), a decision was made to split some libraries from the standard library into separate packages. This was motivated by:

  • the size of some libraries, like REXML
  • the fact that some libraries, like OpenSSL, readline, gdbm and tk require other library packages to be installed
  • licensing concerns about shipping some code (a .so) linked with OpenSSL, and some other code (another .so) linked with readline, in the same package.

Most of the split was undone a very long time ago, and the readline and openssl splits were undone this year. The current package in Debian squeeze has:

  • ruby1.8: binaries (ruby itself, irb, rdoc)
  • libruby1.8: the standard library, except ruby-tk
  • libtcltk-ruby1.8: ruby-tk (merging it with libruby1.8 would add several megabytes)
  • ri1.8: contains both the ri binary and documentation
  • ruby1.8-dev: development headers (to build applications or gems linked against the Ruby C API), and statically built version of the library
  • ruby1.8-elisp: Ruby emacs mode
  • ruby1.8-examples: Examples
  • libruby1.8-dbg: debug symbols

It is always possible to merge more packages, but merging any of the other packages would significantly increase the size of what you get by default, and might discourage people to use it in some contexts (disk space is not always cheap. Think embedded systems and smartphones).

Myth: Rubygems is crippled in Debian

There are two differences in the rubygems Debian packages. First, gems are installed in /var/lib/gems, not /usr/local/lib/gems. That is because rubygems are seen as “state” for the Rubygems package, not independant software installed separately. A change for this is currently being discussed, and should be done after the squeeze release. (I don’t think that it’s reasonable to push that change now). A consequence of installing to /var/lib/gems is that gems’ binary are outside of $PATH. After the change, gems’ binary would also be installed in /usr/local/bin.

Second, gem update --sytem is disabled in the Debian package, because that would replace code from the Rubygems Debian package with code downloaded from the Internet, and can break your system in subtle ways. We have changed this (in SVN, not in any package in the archive yet) to allow gem update --system if a specific environment variable is set, which gives us the opportunity to display a warning message.

Myth: Ruby is slow on Debian

Ruby is built with the pthread threading code, because not using it breaks many native Ruby libraries (Ruby-gnome2, for example). And other Linux distributions are doing the same.

There are some very old benchmark results floating around in some blogs that show that the pthread code has a performance impact of about 20%. There are been patches for that for a long time (applied in Ruby Enterprise Edition), but they were never properly sent back. Actually, all the discussions about this issue were rumors and blog posts, and there wasn’t even a bug report about it!

After the bug was reported properly, it was fixed in a timely manner. The fix is not yet included in the Ruby 1.8.7, but is being applied as a patch to the Debian package since (So the Debian package is actually newer and faster than upstream Ruby in that regard).

What makes Ruby so difficult to maintain in Debian?

The main problems we have with Ruby are portability issues. The only officially supported platform for Ruby is IA32 (i386 in Debian), and everything else is best effort. It means that it is not even supported officially on x86_64/amd64! Debian supports a dozen of architectures, and the Ruby package has to work on each of them. Ruby 1.8 is mostly OK nowadays thanks to the help of Debian porters, but we still have serious problems with 1.9.2, since the test suite fails on sparc, hppa, ia64 and Debian GNU/kFreeBSD.

Why doesn’t Ruby use the alternatives system on Debian?

The alternatives system provides a way to switch between different software providing the same functionality. Currently, if you want to use Ruby 1.9.2, you need to use the ruby1.9.1 executable (1.9.1 = ruby compatibility version here). The alternatives system would allow to switch ruby to ruby1.9.1 on a system-wide basis.

However, the situation of applications and libraries working only with Ruby 1.8 should be clarified: those should use /usr/bin/ruby1.8 instead of /usr/bin/ruby. Another problem is that the way to specify packages dependencies still need to be determined. There’s nothing impossible to do about that, it just needs someone to do the work.

Where do we go from here?

The Ruby developers (or expert users) community is generally very hostile towards Debian. Many harsh words and insults are used in discussions mentioning Debian in the ruby-core@ mailing list, and there are frequent recommendations to avoid the Debian packages and install from sources, which is quite demotivating (Actually, what prompted that blog post is someone calling Debian-specific changes unforgivable).

This atmosphere makes it hard to recruit people, and the Debian Ruby teams are completely understaffed, which is clearly the major blocker to improving the situation further. We are 3 co-maintainers for the interpreter packages, and I’m the only Debian Developer that is really active on a regular basis in the pkg-ruby-extras team (that does libraries and applications packaging). We desperately need help, but at the same time I have no time to improve our documentation and make it easier to join the team.

While I haven’t reached a final decision yet, I am also likely to stop doing Ruby work in Debian after squeeze is released, as I am getting increasingly tired and demotivated.