Cool stats about Debian bugs

Now that bug #500000 has been reported, let’s have a look at all our other bugs, using UDD.

Number of archived bugs:

select count(*) from archived_bugs;
 count  
--------
 402826

Number of unarchived bugs marked done:

select count(*) from bugs where status = 'done';
 count 
-------
  8267

Status of unarchived bugs (“pending” doesn’t mean “tagged pending” here):

select status, count(*) from bugs group by status;
    status     | count 
---------------+-------
 pending       | 53587
 pending-fixed |  1195
 forwarded     |  6778
 done          |  8267
 fixed         |   167

The sum isn’t even close to 500000. That’s because quite a lot of bugs disappeared:

select id from bugs union select id from archived_bugs order by id limit 10;
 id  
-----
 563
 660
 710
 725
 740
 773
 775
 783
 817
 819

Now, let’s look at our open bugs.
Oldest open bugs:

select id, package, title, arrival from bugs where status != 'done' order by id limit 10;
  id  |    package     |                                   title                                    |       arrival       
------+----------------+----------------------------------------------------------------------------+---------------------
  825 | trn            | trn warning messages corrupt thread selector display                       | 1995-04-22 18:33:01
 1555 | dselect        | dselect per-screen-half focus request                                      | 1995-10-06 15:48:04
 2297 | xterm          | xterm: xterm sometimes gets mouse-paste and RETURN keypress in wrong order | 1996-02-07 21:33:01
 2298 | trn            | trn bug with shell escaping                                                | 1996-02-07 21:48:01
 3175 | xonix          | xonix colors bad for colorblind                                            | 1996-05-31 23:18:04
 3180 | linuxdoc-tools | linuxdoc-sgml semantics and formatting problems                            | 1996-06-02 05:18:03
 3251 | acct           | accounting file corruption                                                 | 1996-06-12 17:44:10
 3773 | xless          | xless default window too thin and won't go away when asked nicely          | 1996-07-14 00:03:09
 4073 | make           | make pattern rules delete intermediate files                               | 1996-08-08 20:18:01
 4448 | dselect        | [PERF] dselect performance gripe (disk method doing dpkg -iGROEB)          | 1996-09-09 03:33:05

Breakdown by severity:

select severity, count(*) from bugs where status != 'done' group by severity;
 severity  | count 
-----------+-------
 normal    | 27680
 important |  7606
 minor     |  6921
 wishlist  | 18898
 critical  |    29
 grave     |   209
 serious   |   384

Top 10 submitters for open bugs:

select submitter, count(*) from bugs where status != 'done' group by submitter order by count desc limit 10;
submitter                      | count 
----------------------------------------------------+-------
 Dan Jacobson                  |  1455
 martin f krafft                |   667
 Raphael Geissert                |   422
 Joey Hess                        |   392
 Marc Haber            |   368
 Julien Danjou                     |   342
 Josh Triplett                |   331
 Vincent Lefevre                |   296
 jidanni@jidanni.org                                |   260
 Justin Pryzby  |   245

Top bugs reporters ever:

select submitter, count(*) from (select * from bugs union select * from archived_bugs) as all_bugs
group by submitter order by count desc limit 10;
                  submitter                   | count 
----------------------------------------------+-------
 Martin Michlmayr             |  4279
 Dan Jacobson            |  3652
 Daniel Schepler  |  3045
 Joey Hess                  |  2836
 Lucas Nussbaum     |  2701
 Andreas Jochens                |  2605
 Matthias Klose         |  2442
 Christian Perrier        |  2302
 James Troup                |  2198
 Matt Zimmerman               |  2027

You want more data? Connect to UDD (from master.d.o or alioth.d.o, more info here), run your own queries, and post them with the results in the comments!

Looking for cliques in the GPG signatures graph

The strongly connected set of the GPG keys graph contains a bit more than 40000 keys now (yes, that’s a lot of geeks!). I wondered what was the biggest clique (complete subgraph) in that graph, and also of course the biggest clique I was in.

It’s easy to grab the whole web of trust there. Finding the maximum clique in a graph is NP-complete, but there are algorithms that work quite well for small instances (and you don’t need to consider all 40000 keys: to be in a clique of n keys, a key must have at least n-1 signatures, so it’s easy to simplify the graph — if you find a clique with 20 keys, you can remove all keys that have less than 19 signatures).

My first googling result pointed to Ashay Dharwadker’s solver implementation (which also proves P=NP ;). Googling further allowed me to find the solver provided with the DIMACS benchmarks. It’s clearly not the state of the art, but it was enough in my case (allowed to find the result almost immediately).

The biggest clique contains 47 keys. However, it looks like someone had fun, and injected a lot of bogus keys in the keyring. See the clique. So I ignored those keys, and re-ran the solver. And guess what’s the size of the biggest “real” clique? Yes. 42. Here are the winners:

CF3401A9 Elmar Hoffmann 
AF260AB1 Florian Zumbiehl 
454C864C Moritz Lapp 
E6AB2957 Tilman Koschnick 
A0ED982D Christian Brueffer 
5A35FD42 Christoph Ulrich Scholler 
514B3E7C Florian Ernst 
AB0CB8C0 Frank Mohr 
797EBFAB Enrico Zini 
A521F8B5 Manuel Zeise 
57E19B02 Thomas Glanzmann 
3096372C Michael Fladerer 
E63CD6D6 Daniel Hess 
A244C858 Torsten Marek 
82FB4EAD Timo Weingärtner
1EEF26F4 Christoph Ulrich Scholler 
AAE6022E Karlheinz Geyer 
EA2D2C41 Mattia Dongili 
FCC5040F Stephan Beyer 
6B79D401 Giunchedi Filippo 
74B11360 Frank Mohr 
94C09C7F Peter Palfrader
2274C4DA Andreas Priesz 
3B443922 Mathias Rachor 
C54BD798 Helmut Grohne 
9DE1EEB1 Marc Brockschmidt 
41CF0322 Christoph Reeg 
218D18D7 Robert Schiele 
0DCB0431 Daniel Hess 
B84EF12A Mathias Rachor 
FD6A8D9D Andreas Madsack 
67007C30 Bernd Paysan 
9978AF86 Christoph Probst 
BD8B050D Roland Rosenfeld 
E3DB4EA7 Christian Barth 
E263FCD4 Kurt Gramlich 
0E6D09CE Mathias Rachor 
2A623F72 Christoph Probst 
E05C21AF Sebastian Inacker 
5D64F870 Martin Zobel-Helas 
248AEB73 Rene Engelhard 
9C67CD96 Torsten Veller

It’s likely that this happened thanks to a very successful key signing party somewhere in germany (looking at the email addresses). [Update: It was the LinuxTag 2005 KSP.] It might be a nice challenge to beat that clique during next Debconf ;)

And the biggest clique I’m in contains 23 keys. Not too bad.

tool to mirror a website locally?

Dear lazyweb,

I need a tool to mirror a website locally (so I can browse it offline). Requirements:
– not GUI-based (I want to run it in a script)
– support recursive retrieval and include/exclude lists (like wget)
– no output when everything is fine, but still output errors (not possible with wget, which still output “basic information” when running with –no-verbose, and doesn’t output errors when running with –quiet)
– understands timestamps, and retransfers files if timestamps or sizes don’t match
– not too over-engineered, not too badly maintained, etc…

Thank you.

New Debian Developers!

We got a lot of (>= 10) new Debian developers recently. I’m really happy to see that the bottlenecks in the New Maintainer process were (at least partially) solved. My first NM (actually my second, my first one is on hold) also became a DD today.

So, how long does it take to become a DD ? Let’s take 2 examples. Both are very active and skilled new contributors, that probably were quite close from being the faster you can be through NM:

Name Applied AM assigned Approved by AM Account created
Chris Lamb 2008-05-01 2008-06-12 2008-07-22 2008-09-16
Sandro Tosi 2008-03-24 2008-05-06 2008-06-22 2008-09-16

We have the proof: provided you have all the required skills, you can become a DD in less than 6 months!

Of course, some things are not perfect yet:

  • A lot of very good contributors are waiting for an AM, because not enough DDs volunteer to be AMs.
  • Some NMs still take too long to answer questions, using AMs that could probably mentor faster NMs. If your AM is waiting for you, feel guilty now!
  • Front Desk and DAM are still managed by a small set of very active (and very busy elsewhere) DDs. Many of the new DDs were FD-approved and DAM-approved by the same person, which is not so great if we want to keep this two-steps check.

Is Mozilla the new XFree86? Could Ubuntu actually help?

All the recent moves of Mozilla make me feel that they are really taking the XFree86 path. Reading the Launchpad bug log about the EULA shows that most of the posters agree on who is on the wrong side, and favor switching to IceWeasel or Epiphany+Webkit.

Even if Mozilla is apparently going to back off on the EULA story, it looks like the harm was done. If they want to fix that, they will have to start listening to other players in the Free Software community. Or just watch Webkit eat their market share.

Since Ubuntu leaders are apparently talking to Mozilla about that, I really hope that they are aiming for a solution that will help the Free Software community as a whole, and are not looking for a work-around that will “fix” the problem for Ubuntu.

There has been a lot of noise about the lack of “giving back” to the community by Ubuntu. Using Ubuntu user base to weight in and solve such issues in a way that benefit the whole community would probably be seen as a much more valuable contribution than another bunch of patches.

tiling terminals manager

I tried terminator (thanks go to Nicolas Valcarcel for asking me to sponsor a Debian upload, thus forcing me to try it, and Asheesh Laroia for doing a lightning talk at debconf about it), but I’m not convinced.
– More keybindings are clearly missing. You can only switch terminals using Previous/Next keybindings.
– More features would be great, like the ability to switch the position of two terminals (so you could reorganize them).
– It has some small usability problems, like the fact that the config is text-based, not using gconf, that it’s not possible to change the config without restarting it, that the title bar doesn’t display anything useful most of the time, since it prefixes the current terminal’s title with “Terminator: “, etc.

So, is there any other tiling terminals manager I should try, before filing tons of feature requests on terminator? My other requirement is that it mustn’t reinvent the wheel, but use the gnome-terminal widget.

Thank you.

Code for Debian versions comparison?

Do you know code that compare versions of debian packages (for example, that knows if 2:23.2.3~rc1-1 is lower than 2:23.2.3-2?), besides dpkg –compare-versions? If yes, please write a comment to this blog post, preferably with a link to the code.

Also, did someone already write a test suite for that? Who would be interested in such a test suite?

I’m considering writing a function in PL/SQL to compare debian versions (for the Ultimate Debian Database project). If someone already wrote that, I’m interested as well.

Of popular packages removed from testing, and the Ultimate Debian Database GSOC project

Some time ago, there was some flamewars^H^Hdebate about the Release Team’s removals of RC-buggy packages from testing. Basically, some people claimed that popular packages shouldn’t be removed, even if RC-buggy.

But, do we really miss popular packages in testing?

It’s difficult to know. You could get the popcon data, and compare it with the Packages files for testing and unstable. Or work with source packages (which removes a lot of noise), but then, you have to convert the popcon data (which uses binary packages names) to source packages. Not completely trivial.

That’s where the Ultimate Debian Database GSOC project comes to the rescue. The goal of Christian von Essen’s project is to gather data from various sources in Debian into a single SQL DB, so queries that combine all those data sources can easily be written.

For example, here is the query that lists the source packages that are in unstable, but not in testing, sorted by their popcon (using the number of insts of the most popular binary package of the source package as value for the source package):

SELECT DISTINCT unstable.package, insts
FROM (SELECT DISTINCT package FROM sources
WHERE distribution = 'debian' and release = 'sid') AS unstable, popcon_src
WHERE unstable.package NOT IN (
   SELECT package FROM sources
   WHERE distribution = 'debian' AND release = 'lenny')
AND popcon_src.source = unstable.package ORDER BY insts DESC;

And the results are available on the web!

Top packages (> 1000 insts):

lzo	64962
gnome-cups-manager	32346
db4.6	20708
ffmpeg-debian	12908
freetype1	10569
flashplugin-nonfree	7116
perlftlib	6769
nvidia-graphics-drivers	3864
wxwindows2.4	3640
dvi2tty	2239
kdebase-runtime	1725
easytag	1717
g-wrap	1582
yaird	1507
slocate	1499
youtube-dl	1390
hugin	1275
w3c-libwww	1058

Interested in UDD? Join #debian-qa or debian-qa@lists.d.o (or talk to me @DebConf!)

Exporting logs from Suunto X6HR watches on Linux

I’m the happy owner of a nice geeky toy: a Suunto X6HR watch, that includes an altimeter and an heart rate monitor, which I use mainly for moutain biking and hiking.

During outings, the watch can log the altitude and heart rate every 2, 10 or 60 seconds, and the data can be transfered to a PC using a serial interface. The problem is that Suunto only provides software for Windows. I got tired of using virtualbox to connect to the watch (qemu doesn’t work, Suunto Activity Manager apparently does strange things with the serial port), so I reverse-engineered the protocol (using skimanager and Jérome Kieffer’s work as a basis) and implemented a script to fetch the logs, and export them in a format suitable for gnuplot.

Of course, Suuntux is publicly available. I’d be happy to hear from you if it works for you too. Also, if you own a Suunto X6 (similar watch, without HRM), I’d be interested in supporting it too (if it’s not supported already).

Below is a example graph, from a short mountain bike ride just before leaving for Debconf.

example suuntux output

3G Internet access using mobile phone + laptop in France

In France, we only have 3 mobile network operators: Orange, SFR, and Bouygues Telecom. They usually discuss their rates together to make sure that one of them doesn’t break the market by creating too interesting rates (that’s why it would be so cool if Free.fr could get a 3G license, and why the governement can tell them “do what we want, or we will give Free a 3G license” – source).

I’ve long been interested in using my mobile phone with my laptop to access the Internet. With the 3 operators, you can get a contract for a 3G USB key, but that’s not really interesting: it’s expensive, and there’s a limit on volume of data, so you have to monitor your bandwidth. Also, I only need to use my mobile phone as modem a few times a month. So I’m not really interested in paying an expensive monthly fee for that.

What do french MNOs offer?

Most of them have rather interesting offers when you want to get unlimited internet access from your mobile, but that doesn’t cover the case of using your phone as a modem, to access the internet on another device (e.g laptop). I’m not sure how they can see the difference, but apparently they do. (It seems that there are hacks making use of those unlimited rates, using your phone as a router between the Wifi network (but your phone has to support Wifi) and the 3G network, using apps like WMWiFiRouter. Another hack is to use the same HTTP proxy on the laptop than on the phone, which seems to give unlimited HTTP access.)

But offers for using your phone as a modem are clearly less interesting (probably because they want to push their USB 3G cards offers):

Bouygues:
Apparently, you are billed only based on the volume (see page 46 of this document), with a progressive rate. 5 MB/month costs 9 EUR, 100 MB costs 20 EUR, and over 100 MB, you pay 1 EUR/MB (!).

SFR:
See page 40-41 of this document. You pay 0.50 EUR for a 30 minutes session, with 2 MB of data included. After those 2 MB of data, you pay 1 EUR/MB.

Those rates are really crazy. Using a mobile phone on an HSDPA network (called “3G+” in France), you can easily reach 1 Mbps. (and I did see this rate while fetching files using rsync, so I was not particularly aiming at performance!). 1 Mbps translates to 7.5 EUR/min using the 1 EUR/MB rate. Or 12.5 cts/sec!!!

Surprisingly, Orange (ex-France Telecom), which is usually not the most innovative MNO and ISP in France, saves us. With Orange, you pay 0.5 EUR for a 20 mins session, with unlimited volume (sessions called “session multimédia” by Orange). I was so surprised that I called the assistance to check. And after trying it yesterday, I was a bit anxious when checking my online bill this morning. But it works!

Important note before you try (from Orange website):
La session multimédia est une tarification valable pour les clients forfaits mobile Orange (hors forfaits Orange pour iPhone, Mobicarte et cartes prépayées). Pour les clients Classique, Intense, Pro, Click le forfait et Initial dont la souscription ou le réengagment est antérieur au 14 juin 2007, pour les clients forfaits bloqués dont la souscription ou le réengagement est antérieur au 16 août 2007 et pour les clients bénéficiant d’une offre blackberry, l’accord exprès de l’abonné à bénéficier de la session multimédia est nécessaire. Les autres clients bénéficient de la tarification au volume.