Debian on a Dell E4300

I received a new laptop, a Dell Latitude E4300. I was a bit anxious with its level of support on Linux, because it only recently started being shipping, and it’s full of recent Intel hardware. Also, with lenny being frozen for 5 months, we are currently a bit behind regarding hardware support.

I installed lenny on it, as I usually run testing+APT pinning. What I’m describing here is what is unsupported in lenny out of the box.

Wireless:
Needs a 2.6.27 kernel (iwlagn driver), Lenny only has 2.6.26. Downloaded from the kernel team’s unofficial repo. Firmware isn’t packaged in Debian (firmware-iwlwifi needs an update). Needed to download it from Intel.

Graphics:
Intel device, unsupported with lenny’s driver (but vesa works surprisingly well, oI only noticed that I was using vesa when I tried to play a video and found out I had no xv support).
Works with experimental’s Xorg and the experimental intel driver, but that Xorg needs some polishing. The evdev driver now handles all inputs, so the keymap has to be configured differently (as usual, #debian-x people were very helpful).

Suspend/Resume:
Works out of the box using pm-utils. No need for quirks.

All in all, it was easier than I expected, and support will probably be perfect for lenny+1/2 ;)

The Project membership procedures GR

So, the call for vote on the “Project membership procedures” GR was sent. Since I took part in the discussions that lead to that GR, I feel obliged to explain my point of view.

What this GR isn’t about
This GR isn’t about the merits of Joerg Jaspert’s proposal. I personally don’t think that the changes he wants to implement are what we need in Debian, but even if I fully agreed with those changes, I would still disagree with the process that led to those changes. Also, this GR isn’t about Joerg Jaspert himself. I deeply respect him and his awesome work for Debian. And I’m sure that this vote could have been triggered by decisions announced by someone else.

What this GR is about
I think that the question we are asked to vote on is:

Some people in Debian have a special role, and special privileges. Does it allow them to change important aspects of our project, without the approval of all the other developers?

Or, put differently:

Where is our limit between Doocracy and Democracy?

I’m all for listening to developers who contribute(d) a lot. I think that most people, when skimming through posts in a big thread on -devel@, will look for posts from people who did a lot of work for Debian, or who usually post interesting ideas. However, that doesn’t mean that those people should feel empowered to decide changes to important parts of our project, without first discussing those changes with all other developers, and making sure that those changes are consensual.

So, I hope that we will send a clear message that says: you are very welcome to propose changes, but please make sure that they are properly discussed within the developers before they are implemented.

Joerg’s initial mail on d-d-a, with a few s/decision/proposal/, and a final question phrased like “Any comments? Suggestions? Other ideas?”, could have been the start of an interesting discussion about how we can integrate non-developing contributors in Debian. And if too many developers disagreed, we could have had an opinion poll (it would be great to have an infrastructure for running polls inside Debian, like what Jeroen did three times a long time ago), or even a GR, like we did for Debian Maintainers.

For this actual vote, there are two similar options on the ballot (choices 1 and 2, respectively “amendment text A” and the original text on the vote page). I proposed this amendement (choice 1) because the original text is unclear, and mixes several different questions, forcing voters to answer several (secondary) questions at once, like:

  • Do we recognize that there are contributors that can’t become members of the project currently, and that should?
  • Do we want to thank DAM for bringing this up? (implicit question or not: … now that we are trying to release?)
  • Do we want DAM to work on a new proposal?

The amendment aims at going straight to the point. And yes, it is only asking the DAMs, but if the developers ask the DAMs not to do something, and the DAMs do it anyway, we clearly have a bigger problem. I believe that the way those decisions were announced was a mistake (that unfortunately none of the people involved have been willing to admit), not that the DAMs are trying to take-over Debian, so there’s no need for a stronger wording.

State of hardware support in Linux

I’m getting increasingly annoyed by the state of some aspects of hardware support in Linux.

My laptop (old Dell Latitude D610, Intel-based with ATI graphics) used to suspend/resume correctly with Linux 2.6.24 (ie, > 95% success rate). Later changes made the success rate drop significantly, and added a problem with kacpid taking 100% CPU because of an interrupt storm. And now, with 2.6.28-rc6, it’s completely broken. (partially documented in bug 11563, but I admit I gave up on this bug, because I’m going to change my laptop soon).

My desktop used to wake-on-lan correctly with 2.6.24 (but required some hacks, because it wouldn’t wake up if the NIC was DOWNed before shutdown), but changes in the r8169 driver broke it. (documented in bug 9512).

As a result, I’m forced to run old kernel versions on the two systems I have at home. I can understand that those issues aren’t considered high priority (not everybody use WoL), but the fact that in both cases, they are regressions, worries me a bit.

How many things are routinely broken during each kernel release cycle? Hardware support is difficult, of course, but are we really doing everything we could to make it suck less? Some things I really would like to see:

  • Distro packages for development kernel versions. For Debian, there’s this repository, but it doesn’t always contain the latest kernel versions. Maybe that’s something that should be moved to a kernel.org umbrella and generalized to all distributions, to provide beta-testers with easy-to-install packages. git bisect isn’t that user-friendly.
  • Funding for driver developers to buy specific hardware. Many drivers cover a wide range of chips/cards, and developers often only have on a small subset of them, making it difficult to debug issues specific to one chip.
  • Better/updated bugzilla on kernel.org. Many bug logs are totally confusing, and cover different issues. They could maybe benefit from new or specifically developed bugzilla features (and more bug triagers, of course).

-vote@ discussions on DFSG violations

There have been 470 mails during the last month in the DFSG violations threads on -vote@, but only 10 posters have contributed more than 10 mails so far:

85 Robert Millan
51 Manoj Srivastava
18 Pierre Habouzit
18 Josselin Mouette
16 Thomas Bushnell BSG
14 Stephen Gran
13 Frans Pop
13 Ean Schuessler
13 Adeodato Simo
12 Russ Allbery

Is someone working on a summary of the discussions? I would really hate it if we were asked to vote on this, with a “for details, see the -vote@ archives” footnote. (Robert Millan sounds like a perfect candidate for this task :-) )

Living in France? Not an April member? You are WRONG.

I’ve been a member of April, the french association for promotion and defense of Free Software, for a bit more than a year, and I often regret not becoming a member earlier. (I was feeling so guilty and shameful about not being a member that I actually postponed becoming a member.)

Stop feeling guilty and shameful, become an April member today!

Why Is becoming an April member so important?

  • Clearly, April doesn’t address the same problems as your local LUG. April is a country-wide organization, and it works on country-wide problems. It’s the only group able to work on such problems at this scale (I’m not sure of the situation in other countries, but I think CCC shares a similar role in Germany for example).
  • Each time I talk to people really involved in April (which I’m not), I’m amazed by how powerful they have become. They are able to talk to french or european deputies or ministers’ cabinets, and are considered important. They are doing a fantastic job spreading what matters to us to legislative and executive powers in France and Europe.

Some of the things they worked on recently (from the top of my head):

  • Lobbying on :
    • OOXML
    • General announcements about politics (Plan France Numérique 2012, aka Plan Besson).
    • European telecom package and HADOPI law (french graduated response) law, through Quadrature du net. (OK, it doesn’t have anything to do with April, but most of the people involved in Quadrature du Net are also involved in April :-)
    • vente liée : the fact that it’s not possible to buy a computer without a Windows license. It’s illegal in French law, but still the de facto situation almost everywhere.
  • Organization of a campaign where candidates to elections in France where asked questions, or asked to sign a declaration about Free Software. In 2007, 8 out of the 12 candidates of the french presidential election answered April’s questions.

So, really, become a member today. It’s only 10 EUR, and you already know they will be well used. April is trying to reach 5000 members by the end of 2008.

(Apparently, if you use that address, April will now that you came from me. No benefit for me at all.)

Developer status: problems and solutions

For a minute, I’d like to think about The Announcement in terms of problems and solutions.

Problem: We don’t have enough non-developing contributors in Debian
Solution in The Announcement: Give them special, official, statuses in the Debian community, so they are rewarded for their work.
Alternate solution: Are they really looking forward this status? Aren’t we just thinking that they are just as power-hungry as we are? Last time I checked, the Debian community wasn’t really welcoming non-developers. It isn’t a problem of official status. We have to work on ourselves to make Debian a better place to be for non-developing contributors. Also, giving those people a second-class citizenship, that won’t be widely recognized — “Debian Member” instead of “Debian Developer”, isn’t going to help. (or even worse, “Debian Contributor”, which doesn’t give any right except an email address).

Problem: We have a trust and security problem. It’s difficult to give upload rights and access to Debian machines to more than 1000 people, with some of them not being active anymore, or not having much experience with security (in the case of non-developing contributors).
Solution in The Announcement: Remove rights based on classes of developers, so the group of people having full access stays within manageable boundaries.
Alternate solution: Have fine-grained control on who can do want.

Problem: DMs can’t get access to Debian resources because their keys aren’t managed by the Debian keyring team, but by another team.
Solution in The Announcement: Keyring managers will take care of the Debian Maintainers keyring too. In the past, the keyring managers were the cause for huge delays in the NM process. It has improved a lot recently, but it doesn’t mean that it will always be like that.
Alternate solution: Make the DM keyring team, the keyring managers, and DSA, work together on a solution. it doesn’t sound impossible. If that’s necessary, merge the DM keyring team and the keyring managers.

Problem: DM was done without the blessing of most of the members of the loosely defined group of powerful DDs (sometimes described as “cabal”).
Solution in The Announcement: Drop it, replace it with something very similar, but originating from the cabal.
Alternate solution: Why not keep it?

Problem: Clueless DDs advocate clueless contributors for DMs. Then those DMs upload crap to the archive. (Note: I don’t necessarily agree with that problem, that’s just a problem that the announcement tries to solve)
Solution in The Announcement: Require answering a few questions before becoming a DM. Remove the right for every DD to decide which DM can upload which package. Give that control only to the NM committee.
Alternate solution: If we can’t trust one DD to take the good decision about advocating someone for DM, we could require that each person needs to be advocated by two different DDs. Or three, four, five. Asking the DM to answer a few questions, whose answers he can probably get by asking around on IRC, is not going to improve the average level of our DMs or DDs.


I believe that there are real problems that need to be solved, but that the decisions announced don’t solve the real underlying problem. Here is what I’d like to see:

  • Fine-grained access control for each DD or DM or …:
    This would allow people to have specific rights inside Debian, like:

    • Right to login on each debian.org host (a per-host switch)
    • Right to upload packages for which the person is Maintainer or Uploader
    • Right to upload any package
    • Right to vote

    All rights would default to “NO”. DDs would be allowed to change their rights for everything, while DMs would only be allowed to change specific rights. Of course, we need a secure way to change rights (HTTPS+confirmation using GPG, like the current “sudo password” thing?). But that would help with the security problem (you wouldn’t have access to all Debian hosts without enabling it). In fact, we already have that: many hosts require a switch to be enabled in LDAP before you can login, and we have tons of Unix groups to restrict access to specific areas of our infrastructure. I’m just proposing to extend that to basically everything, and to allow people to grant themselves some rights without going through DSA.

  • Modified NM process for non-developing contributors, that would still include many P&P and T&S questions, but would give “Debian Developer” status without any upload rights. That way, non-developing contributor would have an official status, and will still be able to write “Debian Developer” in their resume. Giving a different status to those people just because we don’t think that they should have the right to upload any package doesn’t feel right.

Is that enough? Probably not. We need to be more welcoming towards non-developing contributors, which is a social problem, not a technical one. But we are not going to solve that in a GR.

Hype

It’s really funny to see how popular things can get, and then totally disappear. Some time ago, there was a TV show on the french TV called “Un an de +”, that talked about what was in the news one year before. It was really interesting to see how quickly everybody can forget stuff that looks so important now.

For example, who remembers Second Life? Apparently, it has been on the decline for some time already, according to Google Trends:

Will Facebook be the next Second Life?

Importing bugs into UDD, faster.

Importing bugs from the Debian Bug Tracking System into UDD (the Ultimate Debian Database) in a reasonable time is challenging. The BTS uses flat files to store bug information, and importing a bug typically requires reading a ‘summary’ file, and a file containing the verisoning information, both of a few hundred bytes. That looks easy, but when you multiply it by ~70000 unarchived bugs, it takes a lot of time (about 40 minutes) to read those ~100k files, because the import process will block on every I/O. The problem is not the amount of data to read (19.8 MB for summary files, 7.4 MB for versioning files), but the number of files (69612 summary files, 17507 versioning files).

The obvious solution is to preload all the files into the page cache, so they are there when you need them. But you can’t simply do that with find /org/bugs.debian.org/versions/pkg -type f -exec cat {} \+ &>/dev/null, because that wouldn’t fix anything: you would still block on each file, and prevent the I/O scheduler to reorder the reads and optimize them (it’s called elevator for a reason). So, how do I tell the kernel “I’m going to read that in the future, please preload it?” readahead(2) is blocking, so it’s not helpful. The right solution is to use posix_fadvise(2), that allows to declare an access pattern. Using fadvise to preload all the files takes less than 5 minutes, and importing the bugs after that takes less than 8 minutes, so it’s really a big win.

Does someone know if there’s already an fadvise-based tool that allows to preload a list of files? That’s something that I could need in other contexts as well.

Cool stats about Debian bugs

Now that bug #500000 has been reported, let’s have a look at all our other bugs, using UDD.

Number of archived bugs:

select count(*) from archived_bugs;
 count  
--------
 402826

Number of unarchived bugs marked done:

select count(*) from bugs where status = 'done';
 count 
-------
  8267

Status of unarchived bugs (“pending” doesn’t mean “tagged pending” here):

select status, count(*) from bugs group by status;
    status     | count 
---------------+-------
 pending       | 53587
 pending-fixed |  1195
 forwarded     |  6778
 done          |  8267
 fixed         |   167

The sum isn’t even close to 500000. That’s because quite a lot of bugs disappeared:

select id from bugs union select id from archived_bugs order by id limit 10;
 id  
-----
 563
 660
 710
 725
 740
 773
 775
 783
 817
 819

Now, let’s look at our open bugs.
Oldest open bugs:

select id, package, title, arrival from bugs where status != 'done' order by id limit 10;
  id  |    package     |                                   title                                    |       arrival       
------+----------------+----------------------------------------------------------------------------+---------------------
  825 | trn            | trn warning messages corrupt thread selector display                       | 1995-04-22 18:33:01
 1555 | dselect        | dselect per-screen-half focus request                                      | 1995-10-06 15:48:04
 2297 | xterm          | xterm: xterm sometimes gets mouse-paste and RETURN keypress in wrong order | 1996-02-07 21:33:01
 2298 | trn            | trn bug with shell escaping                                                | 1996-02-07 21:48:01
 3175 | xonix          | xonix colors bad for colorblind                                            | 1996-05-31 23:18:04
 3180 | linuxdoc-tools | linuxdoc-sgml semantics and formatting problems                            | 1996-06-02 05:18:03
 3251 | acct           | accounting file corruption                                                 | 1996-06-12 17:44:10
 3773 | xless          | xless default window too thin and won't go away when asked nicely          | 1996-07-14 00:03:09
 4073 | make           | make pattern rules delete intermediate files                               | 1996-08-08 20:18:01
 4448 | dselect        | [PERF] dselect performance gripe (disk method doing dpkg -iGROEB)          | 1996-09-09 03:33:05

Breakdown by severity:

select severity, count(*) from bugs where status != 'done' group by severity;
 severity  | count 
-----------+-------
 normal    | 27680
 important |  7606
 minor     |  6921
 wishlist  | 18898
 critical  |    29
 grave     |   209
 serious   |   384

Top 10 submitters for open bugs:

select submitter, count(*) from bugs where status != 'done' group by submitter order by count desc limit 10;
submitter                      | count 
----------------------------------------------------+-------
 Dan Jacobson                  |  1455
 martin f krafft                |   667
 Raphael Geissert                |   422
 Joey Hess                        |   392
 Marc Haber            |   368
 Julien Danjou                     |   342
 Josh Triplett                |   331
 Vincent Lefevre                |   296
 jidanni@jidanni.org                                |   260
 Justin Pryzby  |   245

Top bugs reporters ever:

select submitter, count(*) from (select * from bugs union select * from archived_bugs) as all_bugs
group by submitter order by count desc limit 10;
                  submitter                   | count 
----------------------------------------------+-------
 Martin Michlmayr             |  4279
 Dan Jacobson            |  3652
 Daniel Schepler  |  3045
 Joey Hess                  |  2836
 Lucas Nussbaum     |  2701
 Andreas Jochens                |  2605
 Matthias Klose         |  2442
 Christian Perrier        |  2302
 James Troup                |  2198
 Matt Zimmerman               |  2027

You want more data? Connect to UDD (from master.d.o or alioth.d.o, more info here), run your own queries, and post them with the results in the comments!