Author: lucas

Text normalizer, anyone?

Posted on by lucas

When I write text documents (using LaTeX or Docbook), I like to wrap lines, as it makes them easier to edit (less things moving on the screen), and allow to have easy-to-read diffs.

However, I always hesitate before rewrapping paragraphs (using vim’s gqap): this mean that I will add noise to my git history. So I only do that from time to time, making “rewrapping-only commits”. But that sucks, since in the meantime, I sometimes make a lot of changes, and my lines grow long again. Of course, I could rewrap my paragraphs before each commit, but if I simply add a word to a paragraph, it might cause all the lines to be rewrapped.

So I what I would need is some kind of “text normalizer” that will:

  • split lines at The Right Place. After ‘.’, ‘,’, ‘:’, ‘;’, etc. So rewrapping won’t propagate changes too far away.
  • understand the basics of LaTeX, so it won’t rewrap 
    \begin{tabular}{|l|l|}\hline
x & y \\
1 & 2 \\\hline
\end{tabular}

    or 

    \begin{figure}
\centerline{\includegraphics{fig}}
\caption{Cool stuff}
\label{coolstuff}
\end{figure}

    (vim does rewrap those examples.)

  • be editor-agnostic. So other committers could use it as well.
  • support for other document formats (docbook XML) would be nice too.

I’ve looked at plasTeX: I could use it to parse a LaTeX document, and export it as LaTeX. But then it would be a LaTeX-only solution. Does anyone have a better solution?

Datamining Launchpad bugs

Posted on by lucas

One think that really is annoying with Launchpad is its lack of interfaces with the outside world. No SOAP interface (well, I think that work is being done on this), no easy way to export all bugs. The only way to get all the bug data in a machine-parseable is to first fetch this URL, and then, for each bug number listed there, to make another request for https://launchpad.net/bugs/$bug/+text. I filed a bug a few weeks ago, asking for a simpler way to get all the data.

A Launchpad dev suggested to do what I just described (fetch all the number, then fetch the data for each bug). I originally dismissed the idea because it just sounded too dirty/aggressive/whatever, but since I needed to practice python, I gave it a try. And actually, it works: I was able to get all the data in less than an hour (but that probably put some load on Launchpad ;-)).

That allows to write cool SQL queries.

Bugs with the most subscribers:

select bugs.bug, title, count(*) as subscribers
from bugs, subscribers
where bugs.bug = subscribers.bug
group by bugs.bug, title
order by subscribers desc
limit 10;
bug firefox subscribers
188540 firefox-3.0 crashed with SIGSEGV in g_slice_alloc() 291
154697 package update-manager 1:0.81 failed to install/upgrade: ErrorMessage: SystemError in cache.commit(): E:Sub-process /tmp/tmpjP6Bsx/backports/usr/bin/dpkg returned an error code (1), E:Sub-process /tmp/tmpjP6Bsx/backports/usr/bin/dpkg returned an error code (1), E:Sub-process /tmp/tmpjP6Bsx/backports/usr/bin/dpkg returned an error code (1), E:Sub-process /tmp/tmpjP6Bsx/backports/usr/bin/dpkg returned an error code (1) 278
141613 npviewer.bin crashed with SIGSEGV 262
59695 High frequency of load/unload cycles on some hard disks may shorten lifetime 182
215005 jockey-gtk crashed with AttributeError in enables_composite() 171
216043 compiz.real crashed with SIGSEGV 168
121653 [gutsy] fglrx breaks over suspend/resume 144
1 Microsoft has a majority market share 142
145360 compiz.real crashed with SIGSEGV 134
23369 firefox(-gnome-support) should get proxy from gconf 126

Bugs where someone is subscribed twice:

select bug, subscriber_login as cnt
from subscribers
group by bug, subscriber_login
having count(*) > 1;
bug subscriber
33065 mvo
48262 mvo
144628 skyguy
158126 benekal
213741 sandro-grundmann
216043 jotacayul
221630 kami911

(Yes, that forced me to change a primary key)

Packages with the most bugs:

select package, count(distinct bug) as cnt
from tasks
group by package
order by cnt desc
limit 10;
package number
ubuntu 5392
linux 1464
linux-source-2.6.20 1034
update-manager 826
linux-source-2.6.22 724
firefox 684
kdebase 673
firefox-3.0 668
ubiquity 590
openoffice.org 566

Bugs with the shortest titles:

select bug, title, length(title) as len
from bugs
order by len asc
limit 5;
bug title length
190560 – 1
160381 uh 2
224350 css 3
133621 gnus 4
138052 pbe5 4

If you want to play too, you can fetch the SQLite3 DB (5.8M, lzma-compressed), the DB creation script, and the script that fetches the bugs and import them into the DB. Comments about my code would be very appreciated (stuff like “oh, there’s a better way to do that in python!”), as I’m not very confident about my pythonic skills. :-)

Update: apparently, I’m not really fetching all the bugs. I’m getting the same results as when you just press “Search” on https://launchpad.net/ubuntu/+bugs. But if you click on “Advanced search”, then select all the bug statuses, and click search, you get a lot more bugs (154066 vs 49031). If someone know which bugs are excluded with the default search, I’m interested!

Update 2: Got it. Apparently the default search doesn’t list bugs that have all their “tasks” marked “Won’t fix”, “Fix Released”, or “Invalid”.

FOSSCAMP

Posted on by lucas

Last week-end, I was in Prague for FOSSCAMP, a Canonical-sponsor event aiming at bringing together people from various Free Software projects. Such an event is a very good idea, especially after the openssl debacle, where we saw how difficult it is to build good relationships with other Free Software projects.

The event was organized in a rather interesting way: the attendees make up the schedule as the event happens, by adding sessions to the timetable using marker pens on a whiteboard (see picture, hi Jorge!). If we have a spare BOF room at Debconf, it would be great to do the same thing: I always find it frustrating that many important discussions at Debconf happen outside BOFs/lectures, and are so easy to miss, simply because you submit talk proposals months before Debconf, so you can’t know yet what will be the hot stuff when Debconf finally arrives :-)

It was also interesting to see all the different ways people organized BOFs. It might be interesting to write a list of DOes and DON’T about BOFs (not that the sessions weren’t of good quality!). Does anyone know if such a list already exist?

And finally, as usual, it was great to meet all those people from this nice community again. If you still believe that some projects are fighting with each other, you need to attend such an event. And special thanks go to Ondřej Čertík for guiding me through a visit of Prague!

No, we don’t need a new openssl maintainer

Posted on by lucas

DSA-1571 is totally embarrassing. But I disagree with Julien BLACHE: we don’t need a new openssl maintainer. Mistakes sometimes happen to people who do things. That sucks, but it’s unavoidable, no matter how many levels of checking you add. Kurt has done a lot of excellent work on Debian in the past years, and I’m sure he will continue.

Of course, it’s a lot easier and less risky to write tons of stupid blog posts.

By the way, Julien, there’s an RFH bug for openssl open for more than 2 years. What about getting involved and helping Kurt yourself?

On a more constructive side: it’s not the first time that a Debian-specific change broke something (I can think of a recent grep change …). It might be useful to provide a simpler way to review those changes (reading the diff.gz files is not really user-friendly). We could have something like http://patches.ubuntu.com/ (or even something nicer :-).

Dear IRC user

Posted on by lucas

Please consider switching to screen+irssi, or bip+xchat, or irssi_proxy+irssi or xchat, etc. (This post and its comments has more details).

It is very annoying when you ask me something while I’m away, then disconnect, and then when I finally answer you, you are not connected anymore. If I’m away and you are going to disconnect, please send your question by mail or jabber.

Various stuff

Posted on by lucas

New QA website

I modified qa.debian.org‘s stylesheet/template, using the PTS‘s stylesheet as a basis. It looks a bit better. The content was also updated, so we should stop receiving totally outdated answers to the “What does the QA team do?” question in NM. Now, who is going to do the same thing with www.debian.org? :-)

Closing bugs in removed packages

When packages are removed from unstable and testing, their bugs are not necessarly marked as closed, so they can’t be archived. A few days ago, there was about 3300 open bugs filed against removed packages. Thanks to the work of Barry deFreese, Marco Rodrigues and Raphael Geissert, we are now down to ~2500 bugs. If you want to help, just drop in #debian-qa and ask about our scripts/process. (There are some tricky details)

DEP #1: NMUs

With Bas Wijnen, we finally announced the DEP about NMUs we have been working on. Please join the (currently very quiet) discussion!

19 new Debian Developers! \o/

Posted on by lucas

I am very happy that 19 contributors who were waiting for their accounts, sometimes for a very long time, became Debian Developers today. This is great news for them, and for the project as a whole. Many thanks to all people involved for making this possible, including Joerg Jaspert, Steve McIntyre and James Troup. And congratulations to (using their account names) kibi, plessy, gregoa, goneri, tincho, akumar, filipe, miriam and the others I haven’t had the chance to work with yet.

It also seems that the various pending issues (updating keys that expired, etc.) have been resolved, which is great news for several of our current DDs.

But this doesn’t solve the DAM problem on a permanent basis. Something interesting about today’s events is that the account manager asked the system administrators to create the accounts, which is a nice way to offload part of the process. But the keyring maintainance is still a SPOF. A tool has been developed to allow multiple people to collaboratively edit the same keyring (and it’s used to maintain the Debian Maintainers keyring), but I’ve heard that some people weren’t satisfied with it, unfortunately. Let’s hope that this is solved soon, so the next ones to go through NM won’t have to wait that long!

4 months and 10 days without any new Debian developer. Is Debian dying?

Posted on by lucas

Update (2008/04/18): 19 Debian Developers accounts were created today! See this post for details.
—–
It has now been more than 4 months since the last Debian developer account was created. 18 contributors have been through all steps, and are simply waiting for this simple administrative task to be done.

We are sending a terrible message to potential contributors. We have strong requirements on the technical level of our developers. During the new maintainer process, we ask them to answer about 80 questions about Debian. We ask them to do grunt work. We review their reports twice (New Maintainers’ Front Desk, then Debian Account Manager). But even after we are totally satisfied about what they did, even after they became more qualified than many of our current DDs, we ask them to wait for months, so that the only person allowed to create accounts can finally do his “job”.

It discourages the contributors currently in the NM process. I’ve seen several signs of frustration, or even depression. Some of them reduce their involvement in Debian, so we lose them before they even became Debian developers. Some of them consider resigning from the NM process. We should all feel guilty about that.

But it also discourages people from joining Debian. Instead, they go to other more welcoming projects, which is totally understandable. Debian isn’t the only distribution with developers from the community those days. There’s Gentoo, Ubuntu, Fedora, openSUSE. Some of those have nice programs for new contributors, like “school” sessions. Sure, Debian is the “biggest” distribution without a company behind it. But is independance worth all the trouble?

Of course, we have Debian Maintainers. DM is great for people who want to work on their packages. But, when we are trying to release lenny, we need more: people who are going to go through RC bugs, submitting patches. Who are going to do QA work. In short: people who care more about the whole distribution.

Can we afford not recruiting anybody? Can we run Debian with the current manpower? I don’t think so. There are more than 550 RC bugs in lenny, many packages are currently blocked from migrating to lenny because they are RC-buggy, and many packages are orphaned or neglected. There are also a lot of bugs which haven’t been filed yet (I asked for help with running piuparts, which would probably result in 100-200 new RC bugs, but nobody had time to help). Most of the work that needs to be done is not rocket science. We could use a lot more manpower. Currently, the same small set of developers is doing most of the grunt work. They will get tired too.

So, what can we do, as simple developers? There’s no magic solution, but we can try a few things.

  1. It seems that some people disagree that there’s a problem. Let’s prove them wrong: we could start a blog meme with “I, too, agree that the Debian accounts and keyring situation is severely hurting Debian, and that a solution needs to be found RSN.” It’s not going to solve the problem by itself, but it will at least show that we consider it very important. Pressure could help.
  2. We could start discussing solutions together. Our newly elected DPL said that he would talk with the problematic teams to determine how the situation could be improved. Unfortunately, this has been tried in the past (and failed). It might work this time, of course, but we could prepare a backup plan. So let’s find one or two good plans, and vote on them. (I liked the idea of giving accounts creation/management to DSA. After all, it’s only an sysadmin task once the report has been approved by FD and DAM.)
  3. We could push forward Josip Rodin’s proposal about infrastructure teams. It might not solve the DAM problem immediately, but would probably help avoid similar problems in the future.

    4. Notes:
    1. Maybe the 18 waiting accounts will be created today or tomorrow. Even if that happens, it won’t solve anything. Waiting 4 months for a simple administrative task is not acceptable, and we need to fix that problem anyway.
    2. Account creation is not the only problem. Some people have been unable to upload packages or to vote for the DPL election, because their PGP key expired, and nobody updated it even if they have been asking for more than 4 months.

Things you should know before buying an Asus EEE PC

Posted on by lucas

I’ve been using an EEE PC since december, and people keep asking me if they should buy one. So here are some things you should know before making a decision.

Screen

The screen is small, but that’s OK. I couldn’t find many applications that didn’t work in 800×480. And for the remaining ones, you can usually use hacks such as running the app in a larger virtual desktop. Not ideal, but it works. Regarding usability, it’s not a too big problem. Jut expect to use many apps maximized, and use scrolling and virtual desktops a lot. (Important note: Xmoto doesn’t work!!)

Keyboard

The keyboard is probably a bigger problem than the screen (I hadn’t expected it to be a so big problem before buying the laptop). Every other keyboard seems to have HUGE keys after typing on the EEE PC keyboard. But with some training, you can type quite fast, even if probably not as fast as with a full size keyboard. The layout of the “special” keys is quite good (esp. the PgUp/Down, Home and End keys on the arrows. I should remap my other keyboard like that).

Battery, power consumption

One big problem with the EEE PC is that it consumes a lot of power while sleeping (using suspend to RAM). If you leave it sleeping for more than a few hours, expect the battery to be empty. That really sucks, especially since suspend to disk isn’t really a good option on such a device. Another problem related to power is that charging the battery is quite slow.

SDHC card reader

If you want to expand the storage capacity of your EEE PC with the SDHC port, choose your SDHC card carefully. There are only a few cards that work. The other ones will cause write error (they will work in read-only mode, so I’m using an external card reader to write, which sucks). A list of supported cards is available in the EEE PC manual.

Random Q&A

Q: I don’t have a laptop. Should I buy it?

A: If you want to do a lot of real work on it, it’s probably not a very good idea. There are faster (but bigger) laptops at around 450-500 EUR. They are probably a much better option. The EEE PC is perfect as a the second laptop you only bring to places where you want to stay connected, read email and surf on the web (like conferences and meetings).

Q: Does the VGA output works well?

A: Yes. And it uses an Intel video card which works like a charm with xrandr. I already gave several presentations and classes using it. The laptop can output 1280×1024 (maybe 1600×1200, not sure). The only problem is that you will only have the top-left part of your slides on the laptop’s screen, because of the smaller resolution there. Annoying if you plan to read your slides.

Conclusion

So, should you buy one? If you can’t wait, go ahead. If you can wait, it might be a good idea to wait until Asus fixes some of the problems (the SDHC card reader and the power consumption while sleeping are the most annoying ones).