56 thoughts on “Ruby packaging in Debian and Ubuntu: Mythbusting and FAQ”

1. Ouch, demotivating for sure.

   Also, ruby-lang.org state that 1.9.2 is “mostly compatible” with 1.9.1. Is that mean they are non fully API-compatible?

2. @Eugene: Indeed, there are some changes that can prevent working code on 1.9.1 to work on 1.9.2. For example, ‘.’ is no longer included in the load path, so relative requires need to be explicit. Whether this is an API breakage or not is questionable.

3. I think the main reason why people complain about Debian packages is because rubygems seems to be broken. When rubygems binaries work out-of-the-box and in the default paths and “update –system” is allowed, I guess many Ruby users will use the Debian packaged version since there is no need to compile anything… Besides that, several gems require a compilation phase and maybe merging ruby and ruby-dev would bring less surprise to newcomers trying to install their gems…

   Anyway, the main reason I prefer RVM over Debian package is the gemset feature. There are also great features in RVM. A single command changes your environment from Ruby 1.8.7 to 1.9.2 to JRuby 1.5.1… The rake tasks and Ruby scripts just work and I don’t need to worry about what executable to run. Actually, the main disadvantages of using RVM is that the compile phase is slow and that the ruby packages don’t get updated by ‘apt-get dist-upgrade’, which I do on a daily base in unstable repository.

   But I also have the Debian Ruby installed and it is useful, for instance, when I need to compile some Vim extension that needs exactly the same Ruby version for which vim support was compiled with (like the great command-t vim script).

   I can understand you are sad and demotivated, but I think there is space for both approach: system packages and RVM management. RVM really easy the migration path from Ruby 1.8 to 1.9 making it trivial to change our environment and allowing us to isolate gem versions. But after development is finished, system packages are faster to install and better integrated with Debian upgrade tools (this is the reason why I won’t be buying any Notebook with NVidia graphic cards anymore).

   I would just like to thank you for your efforts and suggest you to make the rubygems work in the standard way (default paths, updates, etc) and I guess Ruby developers will cease Debian’s packages attack.

   Congratulations for your great work!

   Thank you!

4. Ick, Ruby sounds like a mess. Given what you’ve written here, I wonder why anybody would want to use it… :-(

5. I wonder whether the harsh attitude towards Debian is from people who actually do things and could otherwise cooperate with us, or from random people who rant without actually knowing what they are talking about.

   Losing you in the pkg-ruby-extras teams would be a major defeat for everyone who cares about Ruby in Debian.

6. It does have to be realised that the bin wrappers outside of path problem is one that does really get in the way of the 90th percentile use cases. It’s very clear that this is one of the really major if not the major source of hostility by users. I’m glad it’s under discussion to move it back to /usr/local, and I provided feedback on the rubygems-developers list after getting the teams messages through by putting pressure on the ML moderator. As you know I’m doing what I can to be a responsive guy on the ‘other side’ of this, but I’m also trying to avoid more breakage, and as such I’ve scheduled a lot of the required integration patches / fixes for a February sprint in order to make sure we have ample time to research the plans such that we don’t break things for other packagers (besides Debian, that is). The result should be a system that’s more friendly for everyone.

   To be quite honest, the comment about smartphones and embedded systems is really an odd one. I don’t know of anyone in the ruby community who can use an unpatched, custom built ruby for these purposes anyway. Certainly the Debian / Ubuntu patch sets do not make the interpreter suitable for these use cases either, and as such I really see this as a boilerplate argument that doesn’t really have practical applicability. The most essential and foremost item bearing discussion would be the GC tuning that has to be done in these low space / low cpu / low ram scenarios. You’d probably also want the MBARI patches too. I’m not saying this to start another argument, I’m just saying it to point out that some of the people who turn hostile will not be helped to calm down when they read statements like this when they know them to be invalid.

   You know I’m trying my best with what time I have to try and repair relationships or at least be somewhat of a channel to at least sympathize with your issues on the RubyGems side of things, and I’m doing what I can to try and push core toward more sensible engineering processes, but we have a lot of work to do to reduce ill feelings on both sides. In the past we’ve all mentioned facts to each other that have caused an immediate defensive attack from the other side, and it’s going to take concerted efforts on both sides to stop doing that. I can’t make assurances for you that other members of the community will respond in kind, but I do have a little weight with them to at least try to calm the situation and ask them to step back and let me handle some of the issues. I just need to hope that I can provide you and the software with the time required.

   The RubyGems issues raised by gemcutter also did a massive amount of damage to reputations, as it caused a complete breakage for end users. They couldn’t install new gems using old RubyGems and because of the patches and release processes in their OS they couldn’t get new RubyGems without doing upgrades to the whole system. Obviously for these end users that was in many cases unacceptable. We can argue all day long about how wise/unwise those users decisions are, and philosophy about which facets of the different approaches are correct. None of this actually matters, in reality. Yes, the changes introduced by gemcutter were awful release engineering and caused a final-blow set of issues that are still appearing in the wild regularly. I spoke out against all of this at the time, but even being a committer to RubyGems and one of the folks that devotes a lot of time to community support, no one listened. I received hostility because I was speaking out against the latest shiny thing. As I’ve said on ruby-core recently, this attitude and this immature release engineering is almost unforgivable, but we have to. We have no choice, if we want to support users and help people get real work done, we have to move forward and make these issues go away somehow, or at least ease transitions. We cannot really afford, if we want to reach our goals (which are servicing users, for all of us), to keep arguing over these things. We have to educate our users, and make our choices more user friendly.

   I hope we can make leaps and bounds in the coming year, and make major improvements to the working relationships we’re trying to repair.

7. Lucas, thanks for all your great work on Ruby and Debian. I’ve been using Ruby 1.9.1 on i386 and amd64 for awhile and have run into a slew of problems, but using these two technologies is still completely worth it to me. I think the Debian team does an awesome job of getting an incredibly diverse array of technologies to play nicely together. Similarly, the Ruby community does an awesome job of pushing the envelop with development techniques.

   While lots of people complain about the line that Debian has drawn with regards to Ruby packaging, I think its perfectly justified and the right thing to do – its in the best interest of both Debian and Ruby because in the long run it will help reduce bugs and the amount of work needed to maintain the packages. I love developing in Ruby, but I’m disappointed Rubygems hasn’t complied with Debian’s policies. I completely side with Debian without hesitation.

8. Raggi posted that helpful comment while I was writing mine, and I agree his sentiment that Debian and Ruby have a bright future with lots of improvements coming within the next year.

9. It’s sad that this has come to such a bad state. I’m not a Ruby developer, but I am a long time Debian user and now a Ubuntu user. I’ve worked as a Linux Admin in the hosting industry for some time, and Ruby has largely been a PITA.

   I’ve never seen any software break itself so much between such trivial releases. Now, with that said, I also think the Rails framework for Ruby is awesome. I wish that there was a true equivalent of Rails for Python. Yeah, Django is close, but it’s not the same.

   Anyway, back to the topic at hand. If you give up maintaining Ruby for Debian then that will force everyone to install from source, which they already do (for the most part). What I don’t get though is why the Ruby devs are so insistent on bleeding edge, especially in production environments? Are they content with their language and packages being more of a hobby language and framework, or do they actually hope to have it become something stable and usable in large production systems?

10. @Lucas:

    You are write about Ruby…it’s really a mess.

    @Steve:

    Well, Puppet, one of my and other peoples main application written in Ruby needs it. I think Google (aka Andrew P. and Google Sysadmins) can sing a song about ruby and puppet.

    As long as you are not using Gems everything is fine. And Puppet and Ruby right now is just a find experience, I even tend to code some Japanese^WRuby scripts, even I don’t like that language.

    I’m thinking about an alternative to Ruby for Puppet…but Chef or cfengine is not a solution, regaring the all mighty ness of Puppet.

    I hope, Ruby Upstream Devs are thinking about their Userbase, to provide Stable Releases and Development releases like Python does. If not, it’s a non winning case for Debian and all serious Distros which want to provide stable releases.

    Regards,

    \sh

11. oh well, just s/write/right/ I’m tired so mistakes happen

12. Let me explain a bit this bleeding edge thing with Ruby. Lots of Ruby developers use Rails (I’m one of them).

    Rails 3 has just been released and has a lot of great enhancements and it is faster with Ruby 1.9, but at the moment the only 1.9 supported release is 1.9.2. Ruby 1.9 has support for system threads while 1.8 has not and it is also much faster. Unicode support is also much better for Ruby 1.9.

    Those are the reasons why Rails developer are on bleeding edge Ruby and Rails. They don’t need to but they opt to because of the great advantages of being on the edge currently. Both Ruby and Rails have evolved a lot lately. Maybe some day we stop this edge thingy, but there is still lots of improvements on the way (on the Ruby side, there is still space for ‘ação’.upcase become ‘AÇÃO’ instead of ‘AçãO’).

13. It’s great that 1.9.2 is faster than the others. That doesn’t make it production ready, and breaking any and all backwards capabilities isn’t helpful either.

    We can’t expect Ruby to be inline with Debian as they have no relation to each other. However, Debian being what it is, it would be extremely helpful to a lot of people if there was more cooperation. I can’t help but wonder if some of the issues on this stem from Ruby developers being more of a Mac breed and are clashing with the values of Linux users. ;)

14. @raggi: On Ruby on embedded devices. I’m talking about the higher end of embedded devices, aka the ones who already run Linux. I’m happily running Ruby on both by Nokia N900 and my Linksys NSLU2. I think that it would be quite a failure of Ruby not to be able to be useful as-is on embedded devices, when other similar programming languages are. Sure, it might not be perfect performance-wise, but that’s not a major problem.

    One thing to remember about portability issues is that they are usually common to several architectures or systems (think of endianness, word size, POSIX compliance, etc), so fixing it once might fix it for several architectures or systems. It might have looked completely useless to make sure that Ruby worked on ARM or mips 5 years ago, but now that there are ARM-powered netbooks, it doesn’t look ridiculous anymore, and all the work done on ARM while it looked useless can be re-used now to get Ruby working on such netbooks in a very short timeframe.

    A debian-specific problem is that we support all our packages on all architectures (or at least, try to). So if Ruby doesn’t work on one of the architectures, it might prevent many more applications to grow a dependency on Ruby. The fact that there are no so many applications written in Ruby in the wild might be related to the fact that Ruby is not very dependable. For example, the Debian infrastructure uses puppet, and the Debian sysadmins have gone through many issues on getting Ruby to work properly on Debian GNU/kFreeBSD. The root issue (identified by the Debian kFreebsd porters) was that the threading code was using sigprocmask(), which has undefined behaviour in threaded programs. (see http://redmine.ruby-lang.org/issues/show/3779)

    Caring about portability is just a good way to improve code quality.

15. @Greg: the fact that many Ruby developers have a web development background and not a UNIX background is clearly one of the reasons why it is harder to deal with that community than with the Perl or Python communities. And it is also true that many Ruby developers are using Mac OS X on their development systems, and don’t feel very concerned by Linux distributions issues.

16. Lucas: I’m curious, do the 4mb segments not cause you pain for use in environments that only have limited ram?

    Interesting issues under kFreeBSD, I’ve been using ruby on native FreeBSD for a long time and have come across similar issues before. Sadly there’s no accounting for MRIs instabilities eh :-/

    Absolutely agree with your sentiments about portability :-)

17. Lucas – a very large number of companies I know of deploy to ubuntu systems but use rvm or manual builds, I agree that most of the devs work on OSX, but to say that most don’t care about the Linux side, I’m not so sure. Of course we both lack real numbers, so it’s hard to say. I definitely think that the more recent builds will reduce peoples need for manual building, at least I can hope that. I’d also suggest that some of the historical issues actually account for part of the reason rvm receives such significant donations, despite being just a collection of sh scripts.

18. @raggi: my use of ruby on such platforms is rather limited, but no, no problems seen.

    Note that Debian GNU/kFreeBSD uses the freebsd kernel with the GNU libc, so it might not exhibit the same issues as stock FreeBSD.

    Regarding large companies deploying Ruby using distribution packages vs rvm, there are companies on both sides. For example, I know from the investigation of http://redmine.ruby-lang.org/issues/show/2739 that Google is using puppet and the Ubuntu packages for Ruby internally. (I don’t know the scale of the deployment, though. But the fact that they ran into this issue before everybody else makes me think that it’s not too small).

19. For me most of rants from Ruby people (or Python, to a least extent) come from people that might be excellent developers but have no idea regarding system administration and best ingeneering practices (and I say “might be excellent developers” out of courtesy; I don’t really think someone can be an excellent -even good, developer disregarding sysadmin and proper engineering practices).

    Some examples:
    * Not coping with distribution providers installing at /var/lib/ruby and forcing instead /usr/local. That’s so obnoxius it hardly needs to be debated. Haven’t they heard about FHS or proper developing practices so it’s easy to “rebase” a deployment?
    * Forcing “bleeding edge” to everybody. It’s obvious they never needed to support installs on the hundreds (or even in the tens), but it’s obvious they never heard about stability and QA. It’s lame and backwards some 20~30 years that their idea of “support” is returning to the days where compiled apps needed to be statically linked to libraries (having to deploy almost every single app with their own bunch of packages it depends on because no one takes care of backward compatibility is just the same, with exactly the same problems about supportability than the static library environment).
    * You told that current Ruby on Squeeze is out of a daily snapshot. Why??? May it be because no one from Ruby is in the position of saying “*This* is stable; *this* is going to be supported for the next X years and we guarantee *this* not to change API within those X years; only bug corrections will be produced and they’ll be at the extraversion level”? Because this would be a rather serious “warning: lame project” indicator.

20. @sysadmin: on Ruby in Squeeze. It is actually Ruby 1.9.X that is out of a daily snapshot. What happened is:
    – we initially planned to release with Ruby 1.9.1.x
    – the squeeze freeze was delayed
    – a newer GCC was uploaded (minor release in the 4.4 branch)
    – Ruby 1.9.1.x was failing to build with the newer GCC ( http://redmine.ruby-lang.org/issues/show/2727 ; hard to investigate because hard to reduce to a simple test case, still not fixed in 1.9.1.x)
    – We decided to go with 1.9.2.x, since it was building fine
    – During Debconf, I did the initial work on switching to 1.9.2, but 1.9.2 was not released yet at the time. So I used an SVN snapshot from a couple of weeks before the freeze.

    Ruby 1.9.2 has now been released, and after some additional work, it got uploaded to Debian unstable today ( http://packages.qa.debian.org/r/ruby1.9.1/news/20100912T150314Z.html ). Now I just need the debian release team to accept it into squeeze (but I need to make sure it builds everywhere first).

21. @sysadmin

    If those points were ever met, Ruby, and more importantly, RoR would become dominating in the web market.

    I still prefer Python to Ruby for sysadmin stuff, but as a web framework Rails adds so much to Ruby. Ruby could become the next PHP in terms of people jumping on the bandwagon. I think what largely holds it back is attitude. Aside from what we touched on already, you have tools for Ruby and Rails development such as Textmate that will stay OS X only because “that’s how they like it”.

    Puppet is awesome, and I admit that I didn’t get to use it as much as I wish I could have. Now that I’m no longer doing Linux Admin duties I’m focusing more on learning how to program, primarily in C++ and some C# at this time.

22. I prefer Debian provided packages any day. I don’t want to start my (say) Rails installation with installing dev libraries and compiling. And compiling each time I need to add another gem. I want it just work. And it does for me.

    // Just a Debian user.

23. I have a question about the following statement:

    “but we still have serious problems with 1.9.2, since the test suite fails on sparc, hppa, ia64 and Debian GNU/kFreeBSD”

    Can you explain this in more detail? Thanks.

24. Hi Lucas,

    your blog post is another item in my collection, why Ruby is rather the problem then the solution. If it could make your decision easier: You’d make me happy, if Ruby would be dropped from Debian.
    If Ruby is not part of Debian, then it makes it much easier for everybody to keep it out of the company. – A sysadmin may be able to say, what’s not in Debian won’t be deployed!
    Please don’t take this personal, but there are not only friends of ruby.

25. Thomas, the problem is there are useful applications that are build on Ruby, namely Puppet. I’m not sure if you’re aware of what Puppet is or does, but it’s really a great tool to sysadmins. Yes, you could accomplish the same thing with home grown scripts and such, but it seems a waste to trash Puppet simply because the people who constantly develop Rails applications need the most bleeding edge.

26. Lucas,

    I’ve been using Ruby since 1999 and Debian since 2004, and I’m grateful to the numerous volunteers who have made both projects a possibility and a reality.

    Regarding someone calling Debian-specific changes unforgivable, I think it’s fairly well known on the mailing lists that this person has a history of using bombastic and strident language. As such, I think it easiest to simply filter out this sort of noise when reading this person’s posts.

    Regarding: “The Ruby developers (or expert users) community is generally very hostile towards Debian. Many harsh words and insults are used in discussions mentioning Debian in the ruby-core@ mailing list, and there are frequent recommendations to avoid the Debian packages and install from sources.”

    I think this is largely a problem of historical inertia, however regrettable. For instance, it would be completely false to suggest that the aforementioned hostility mysteriously appeared one day out of a vacuum. My belief is that the hostility reflects a perception that developed over the course of years of fielding questions from baffled users on ruby-talk, who didn’t understand why their ruby installation on debian was missing half its standard library.

    Please note that the purpose of my comments here are not to assign blame, and that I personally value both Debian and Ruby very highly.

    Circa 2004, there was no apt-get install ruby-full. If one wanted a ruby installation on Debian that was the SIMPLE EQUIVALENT to ./configure && make && sudo make install, one needed to apt-get something like the following (and I’m not sure this is complete.)

    libbigdecimal-ruby1.8 libcurses-ruby1.8 libdbm-ruby1.8 libdl-ruby1.8 libdrb-ruby1.8 liberb-ruby1.8 libgdbm-ruby1.8 libiconv-ruby1.8 libopenssl-ruby1.8 libpty-ruby1.8 libreadline-ruby1.8 librexml-ruby1.8 libruby1.8 libsdbm-ruby1.8 libsoap-ruby1.8 libstrscan-ruby1.8 libsyslog-ruby1.8 libtcltk-ruby1.8 libtest-unit-ruby1.8 libtk-ruby1.8 libwebrick-ruby1.8 libxmlrpc-ruby1.8 libyaml-ruby1.8 libzlib-ruby1.8 rdoc1.8 ri1.8 ruby1.8 ruby1.8-dev ruby1.8-elisp ruby1.8-examples

    It was not uncommon at the time to read posts like the following, saying things like “Debian vs. Ruby is a Tale of Woe of epic proportions.”

    http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/153779

    That person, in a fit of desperation, was resorting to the following:

    for pkg in $(apt-cache search ruby | grep -E ‘^lib’ | grep -v ‘ruby1.6’ | awk ‘{print $1}’ | wc -l); do echo “Installing $pkg…”; sudo apt-get –yes –force-yes install $pkg; done

    Whereas many of us (myself included) simply advocated: ./configure –prefix=/opt && make …

    We can’t change the past. And I hope your Mythbusting FAQ will be helpful in alerting those who retain lingering hostility from the early days, that many of our negative perceptions about Ruby on Debian from that time may no longer reflect the present reality.

    And once again, I’d like to thank all the volunteers who have made both Debian and Ruby possible.

    Regards,

27. The reason for the push towards Ruby 1.9.2, is it includes all the performance improvements of 1.9.1 and passes 99% of RubySpec. Ruby 1.9.1 did not pass all of RubySpec and had bugs; a couple of which segfaulted active-support 3.0.0 testing suite. This was why Rails3 explicitly dropped support for 1.9.1. Additionally, most people seem to be running 1.8.7, and if they were crazy enough to use 1.9.1 they will probably upgrade to 1.9.2 anyways.

    Ruby 1.8.6 also got the axe (by Rails3) in favor of 1.8.7, since 1.8.6 had many bugs and performance problems. Ruby 1.8.7 also includes backported features from 1.9.1, as a means to transition developers into 1.9.x. Other projects (such as DataMapper) are following suit by dropping support for Ruby 1.8.6 and 1.9.1. (Did I mention that it’s an utter pain to develop against 1.8.6?)

    It was also announced by the ruby-core team that Ruby 1.8.8 will be the last version in the 1.8.x family, and will only contain bug-fixes. This means 1.8.x is reaching end-of-life, and that it’s time to start porting any old code to 1.9.2. Porting to 1.9.2 is easy, assuming you have good test-coverage on your project or previously ported to 1.9.1.

    For those who say RubyGems is “broken” or could be improved, please submit bugs or patches. RubyGems can always use further improvement/scrutiny.

    (Why is rubygems still using rubyforge.org? They should be using the GitHub Issue tracker at http://github.com/rubygems/rubygems.)

    Personally, I think a huge problem with RubyGems and installation are Ruby C extensions. C extensions have no (real) way of specifying dependencies on system libraries, but require a compiler and header files to compile against. Not to mention mkmf, which generates the Makefiles for C extensions, is some hideous Ruby code. That said, I’m a huge fan of Ruby FFI bindings, which require no compilation and are supported by all Ruby implementations.

    I find it funny that Ruby is not officially supported for Debian/x86-64. I started using Ruby 1.8.6 on x86-64 back in 2005.

    Yes, Ruby, RubyGems and Rails have made mistakes or put out bad releases in the past. It seems now that the Ruby community is working hard to improve the user/developer experience. As a result, we now have http://rubygems.org, http://rubydoc.info, RVM, memprof, Bundler, Rails3, Ruby 1.9.2 and RubySpec.

    Oh, also, don’t feed the trolls.

28. I should point out that “officially supported” is a somewhat misleading label used by ruby-lang.org. As I recall, ruby-core runs their current Continuous Integration server on Debian IA32. This is why ruby-lang.org states that Debian IA32 is “officially supported”.

    Applying their definition of “supported”, Mac OSX 10.5 and 10.6 (the dominant OS amongst Rubyists) are not even supported.

    There was also discussion in the last ruby-core meeting minutes of setting up a new Continue Integration server.

    Also, I don’t think many Rubyists endorse breaking APIs on minor versions. http://semver.org is very popular amongst Ruby project maintainers.

29. @andrew: the Debian packages are automatically built on all supported architectures. The status page to follow the status is https://buildd.debian.org/pkg.cgi?pkg=ruby1.9.1.

    The ruby1.9.1 package (for Ruby 1.9.2) currently enables the bootstrap testsuite, and consider its failure as fatal, and the full test suite, without considering its failure as fatal (because else, we wouldn’t be able to ship it).

    But there are currently test suite failures on several architectures, that seem to indicate that Ruby is broken on those architectures. There are open bugs for each of them: http://bugs.debian.org/cgi-bin/pkgreport.cgi?repeatmerged=no&src=ruby1.9.1

    Looking at some failures and investigating them is a good way to help. it is possible that the sparc and kfreebsd failures are actually caused by the same issue. The ia64 one has apparently been fixed since the bug was filed (the test suite no longer segfaults).

30. Perhaps there’s a definition of “myth” I’m not aware of? Even the one that turns out to be not true is only not true quite recently, so it was very much true at the time people talked about it.

31. Lucas, I’d like to thank you for your work on Debian (although I haven’t benefited from it since I don’t use Debian) and especially for raising awareness of these issues in a wider audience. I do agree that these packaging and compatibility issues need to be improved in order to get Ruby where many people believe it should be – in the same position that Perl and Python are today, i.e. first class choice for system administration scripting and application development.

    In order to improve things we should take a brief look at the root causes of the current situation and see whether we can remedy them. To me it seems there are two major issues here:

    1. Ruby versioning and compatibility. The switch from 1.8 to 1.9 was a major one – not because of the changed runtime system but because of the dramatic changes of the language which create incompatibilities. I’m afraid, 1.8.7 – which was intended to ease migration – probably made things harder because now there was the “real” 1.8 (meaning 1.8.6), the new 1.9.x and a hybrid version 1.8.7 between them.

    2. Gem package management. Personally I was slow to adopt gems because having an independent package management did not feel right to me. At the time the Gem system was conceived mature package management systems for part of the operating systems that Ruby was used on were in place already. OTOH there are other operating systems which still do not have a package system like Debian’s and Red Hat’s (I am thinking “Windows” of course). I do believe the independent existence of Ruby Gem system is what causes most headaches for you as a maintainer of an OS with a proper package management system.

    Item 1 cannot be easily changed because history is as it is. In hindsight it might have been better to call what has become 1.9.x “2.x” because of the language changes. I hope this is handled differently during future language changes.

    Item 2 is probably also difficult as well but I would like to make a suggestion and hear what others think about this. I am pretty sure that this idea will earn quite a few spontaneous rejections but let’s see. There are two main users of Gems: Gem authors and people who actually only use Gems. For the latter group (which is likely larger) the main advantage of Gems is that they can say “gem install foo” and have the “foo” gem installed on their system ready to work. I am pretty sure that these people do not care much about the underlying mechanics. So what about having a version of the “gem” command that uses the system’s package manager to resolve dependencies and load other packages? The most dramatic difference will be that there is no user level package management as there is with the current Gem implementation. However, I am not sure that this is a bad thing per se.

    Gem authors could certainly be helped by providing the appropriate tools to do the gem creation and packaging. An alternative would be to create a bridge between gem repository and the OS’s package repository which would provide an automated or half automated migration of gems to the package system reducing the job of package maintainers to testing and selecting the proper version (which is still hard enough).

    While I do think that this suggestion will create some efforts on several sides it might be a good remedy in the mit term: there would be no more second package management system on OS which do have their package management already, others could still work with the current Gem logic. And life would not become harder for Gem authors.

32. As a Rails developer, I never had any problem with ruby / rails on Ubuntu.
    All is working just fine.
    You’re doing a great job, thanks for all.

33. Preamble: I can only speak for my own experiences, not “the community”. I am both a programmer and system administrator, although until recently most of my sysadmin experience was in *BSD systems and RPM-based Linux distros.

    I 100% support the idea of keeping “stable” versions of software, and the major breakages between certain patch-level releases of 1.8.6 demonstrates why this is important. Ruby itself has let down its users badly here, by adding all sorts of kitchen-sink changes into what IMO should have been purely maintenance branches, and it falls to system packagers like Debian to fill that void.

    However, I also completely understand and basically agree with the “unforgiveable” rant.

    For me one of the great things about Ruby is its completeness. The source is about one third of the size of Perl, but you get a whole lot more standard functionality, like Net::HTTP and Net::HTTPS, out of the box.

    So when I first installed the Ubuntu ruby package, and found lib/net/https.rb was completely missing, my knee-jerk reaction was “WTF?” It looked just like a packaging error.

    After calming down, and considering that maybe there was a reason for this omission, I found the libopenssl-ruby package. There is logic behind it – https.rb wouldn’t be very useful without openssl.so – but it was surprising to me, when other systems I had worked on didn’t have this issue. (That is: “yum install ruby” was fine. “apt-get install ruby” wasn’t)

    The same applies to putting rubygems under /var. It gives the impression that Debian packagers “know better” than the original software author, and this comes across as arrogant. Put more mildly: I think Debian aims for consistency within a Debian system. So this decision supports POLS for Debian users who don’t know Ruby, but it violates POLS for Ruby users who don’t know Debian. The fact that /var/lib/gems/1.8/bin is not in $PATH is just a headache which you don’t get on other systems.

    As for disabling “gem update –system”, this seems like OS policy overriding administrator policy, putting unnecessary obstables in the way. Look at this:

    
$ sudo gem install rails
ERROR: Error installing rails:
i18n requires RubyGems version >= 1.3.6
$ sudo gem update --system
ERROR: While executing gem ... (RuntimeError)
gem update --system is disabled on Debian. RubyGems can be updated using the official Debian repositories by aptitude or apt-get.
$ apt-cache show rubygems
...
Version: 1.3.5-1ubuntu2


    I’d say that “throw away the Debian packages and install from source” is a legitimate answer to this one, when you have a job which needs to be done.

    Now, I don’t think you can dismiss any of these points as “myths” by pointing to some future, non-released version of Debian which fixes them. On the contrary: by doing so you are acknowledging that these are genuine problems with current releases, and you are also admitting that these are the result of bad decisions (because they will be reversed in future).

    But I also understand you’re in a position where it’s hard to win. A Debian package has to support two distinct use cases: providing ruby support for other packaged applications which depend on it (e.g. puppet); and providing a package for ruby end-users, those who write and deploy their own apps in ruby.

    I don’t envy your job, but listening to your users’ needs rather than dismissing them is important. I am glad to see that steps are finally being taken in that direction.

34. @Brian I wish I could upvote your comment, well said. Additionally, I always disliked having to set the RUBYOPT env variable to auto-load rubygems. Debian kept rubygems separate from the ruby-1.9.1 package, after rubygems was explicitly merged into upstream ruby-stdlib so users wouldn’t have to mess with RUBYOPT. I remember Gentoo would simply add RUBYOPT="-rrubygems" to the system env.d files. I’m not sure if this too is an old Myth.

    Per disabling gem update --system, I would like to see a pluggable update system for RubyGems. This way Debian, RedHat, Arc, Gentoo, etc could override the updating of RubyGems, and have it call out to the system package manager.

35. I am one of the persons that is constantly put off by the problematic process to install Ruby on Debian. That said, the situation is constantly getting better. The ruby-full package did a lot in that area. But why is there no ruby19-full package? The 1.9 support in Debian feels like history repeating. Its not like Ruby 1.9 was released yesterday. I do understand that Debian does not push Ruby 1.9.2 to their repositories immediately (but calling it “not stable” might not be good as well). Also, the ruby-full description calls 1.8 “the current stable version” of ruby, which I think is up to Ruby Core to decide. They flag 1.9 as “the current stable”.

    But I also think that this is a heavy cultural clash between to strong-headed communities. The biggest problem is that the Ruby Ecosystem did a tremendous sprint during the last few years which makes it hard for (conservative) package maintainers to keep up. RubyGems might be a chore to the Debian maintainers, but it is a necessary one: libraries iterate and release fast and are superseded even faster, sometimes. Considering the average size and scope of a Ruby Library, this is understandable. We just need a system that can keep up with that speed – and this one is not apt-get.

    But however, my biggest problem this year was the dash through all minor versions of RubyGems 1.3.x, with all kinds of libraries suddenly introducing dependencies to just-released versions of RubyGems. I can fully understand that Debian does not want to follow this development and think it was unwise of the Ruby Community to force that push in such a central library. I do hope that this situation will improve with RubyGems now being a part of Ruby.

36. if the behavior of ruby gems is different from installing gems from source, or the default ruby package doesn’t include libraries rails requires, it seems to me that many ruby developers will probably continue to avoid the debian ruby packages. Especially because of how horrendous the experience was “pre 2005,” as you say.

37. @postmodern: Thanks for your support.

    Regarding gem update –system: IIUC, it does not “replace code from the Rubygems Debian package” as Lucas said, but just installs a newer version in /var/lib/gems/1.8/rubygems-update-x.x.x/

    Of course, it is a potential source of confusion to sysadmins for dpkg to report version X of rubygems, where version Y is actually live. But IMO that should be a choice for the well-informed system administrator to make, and it comes across as arrogant for it to be forbidden.

    After all, the reason I use gems instead of apt packages of ruby libraries is because Debian (quite rightly) won’t keep up with the latest-and-greatest of these. By including rubygems, Debian is allowing people to install versions of many libraries which are different to those available in the apt repositories. So why shouldn’t that right extend to rubygems itself?

38. Lucas,

    This is an excellent exposition and clarifies many common misconceptions with system Ruby packages in general. I firmly believe that all Ruby developers should read this post. One point I try to make is that system Ruby packages must not break any packages that depend on them, which is why it make sense that system packages will always have a very controlled release cycle. This is the correct way to do (system level) Ruby package management.

    RVM enables users to be able to run any arbitrary Ruby environment isolated on a per-user (user level) basis so as to not disrupt the system Ruby. In other words, let system Ruby be system ruby, dedicated to and for the operation of the core OS while separately enabling arbitrary combinations of Rubies + Gems. This yields the best of both worlds, namely stable OS and specific customized application environments such that one does not impose requirements on the other. Thus it should be ‘RVM with system Ruby’ not ‘RVM vs system Ruby’.

    Lucas, it is very easy to get discouraged. You are doing the community a great service regardless of their realization of it. I also understand that it is not easy to let community comments just ‘roll off’, especially when they make generalized statements without obviously knowing all that is involved and backing their comments up with well thought out details :-p

    Keep at it Lucas, you are doing a great job.

    ~Wayne

39. I am using Ruby for around ten years and Debian even longer. I can understand the issues of debian-conform packaging for any environment providing its own package management (ruby, tex, R, whatever…). There always are issues.

    I want to thank you a lot for your work. I like to have Debian Ruby packages where I dont have to care about security fixes on my own, where I even don’t need compilers installed on my server etc.
    So there are users outside who really appreciate what you are doing!

40. I say drop support for architectures that they don’t support.
    Will non i386 ruby users be angry? Yes.
    Will it be Debian’s fault? No.

    Isn’t the policy to make sure it compiles for _supported_ architectures? They don’t support it.
    Let it die.

    I’ve never been a fan of the language, and it seems that the community is poisonous.

41. I’ve heard of it before, but still I’m rather surprised that so many people seem to be having problems with ruby on Debian.

    I’m using this combination for more than five years now to do Rails development and it is just fine.

    For my own development I’m using gems, not the packages, even where they are available. For my work, bleeding edge versions often are suitable or even necessary. First and foremost, however, I have to keep in sync with coworkers on other platforms.

    I’m happily using packaged ruby libs where they are required by other packages that I’m using for their functionality and not because they’re incidentally written in ruby.

    As for gem’ed binaries being installed in /var/lib/gems/1.8/bin, is this really causing anyone trouble? If it does, I wonder whether these people are well-advised to install gems at all, if they’re not able to add that directory to their path.

42. Lucas, I really hope you don’t quit your important work. As someone who came to Ruby from the Perl world, I am often amazed at the disarray of the ruby toolchain. I feel the problem is largely a cultural one and I think Ruby only improves when organizations like Debian insist on a bit more maturity than what we’ve seen from Ruby since it “hit the big time.”

    Obviously, you gotta do what’s healthy for you… but if your demotivation is based on lack of appreciation from the community, I hope the myriad of supportive comments on this post demonstrate that your work is greatly appreciated by those who care about your goals. Someday Ruby will grow up, and when that day comes it’s going to be because of the hard work of people like you.

43. Hi Lucas,

    Thank you for your effort and communication.

    In general, when things are not what people expect, they tend to channel that frustration into hostility. That’s a shame, but i am guilty of that as well.

    As for Ruby. It’s not so much that it’s different from what people expect, but mainly because the only workaround people tend to know, is to just compile ruby & rubygems themselves.

    The debian philosophy was always to have a very consistent platform and if that meant diverting packages from upstream, that so be it.

    The developers working with ruby, the ones using for hosting say rails Applications, tend to care more about the consistency of the ruby platform, than the consistency of the debian platform.

    Some of my team-mates work on OSX, some on Arch Linux, some on Ubuntu, some on Debian.

    And like you pointed out, the ecosystem of Ruby is already quite fragmented. The complexity of also having debian specific versions just adds to the confusions.

    And there isn’t any clear documentation about how the debian ruby packages are different from upstream.

    This creates the situation that repeating the same steps on one platform, does not have the same affect on another platform.

    In the end, I would like to thank you for the efforts. But I would also like to state that, unless the packages are 100% upstream, with no diversions, we will always compile our own.

    This is not just about choices about path locations. When you develop on one system and want to deploy on another system, i want to know that the development system has all the bugs that the deployment has.

    So, even custom debian patches for crasher style bugs, are not really appreciated.

    I understand that our priorities do not line up with those of debian. Why should debian care for any scenario other than developing on Debian release X and deploying on Debian release X?

    So perhaps, the current situation (all the ruby/rails webdevs compiling upstream ruby+gems themselves) is the right way for the community to do this.

    And the ruby packages in the repositories should focus on the packaged desktop and server applications that depenend on the ruby toolchain.

    Because I think, that using a toolchain to run some other packaged application is just a very different use case from developping and deploying your own applications.

    The only true proper debian way would be to create .deb packages of our websites and deploy them like that on the server.

    Perhaps, and this is just a random suggestion, it’s a good idea to have two sets of packages. A debian-ruby and an upstream-ruby. The upstream-ruby would be a garantueed 100% debian patch free version for web developers, whereas the debian-ruby package could be contain a patched ruby interpreter (without the gems tool) which is used by ruby desktop and server applications that are packaged in the repository.

44. I have found that pulling the packages for rubygems from the latest version of Debian into whatever version of Ubuntu I am using works great.

    I think a lot of people don’t realize this.

    It also compounds when people have used both src and package installs of rubygems on the same machine. Even if one is removed it often results in (solvable) madness. :-)

45. In the end, I really cannot see why do Ruby developers have to complain about Ruby and Rails in Debian/Ubuntu: it’s like every other program: if you want to be always on the bleeding edge, then you have to compile and install applications from source; if you want stability, then you can use the provided packages.
    If Rails wants to grow in Web sites usage more and more, then it has to avoid breaking API compatibility even between micro versions; a distribution must avoid breaking compatibility between updates, it is so obvious that there is no reason to complain at all. If I have a production site, then I don’t want my site to crash after an update.
    And about the number of packages: it is just the debian way; if I use Slackware, for example, I have only one package “kdelibs”, if I use debian/ubuntu, then “kdelibs” is split into many many packages, so everyone can install only what is really necessary; if someone wants just everything, that’s way metapackages exist.

46. I’m one of the Google sysadmins responsible for our Puppet deployment here, and as there have been a few references to us, I thought I should speak up.

    First, thank you Lucas for all your work. It’s a difficult position to be in, and I’m happy to see so many people trying to be constructive about repairing the issues we’re all facing.

    I’m saddened to see how demotivated you’ve become, and we should all do a better job of helping out with debian-ruby.

    We’ve never run an alternative Ruby stack for Puppet clients, and have always stuck with the Ubuntu packages.

    For our servers we’ve spent time experimenting with Ruby Enterprise Edition, primarily due to the pthreads/rt_sigprocmask performance issue. When we were running Dapper server, the performance difference with Ruby EE vs the Ubuntu packages was astounding, and there was a clear win in going that way.

    Once we’d migrated to Hardy we saw much less improvement, and decided to stick with the Ubuntu packages on the servers. This was mainly because we didn’t want the management overhead of packaging up Ruby EE. Although they produce a deb, it’s monolithic and installs into /usr/local, and I don’t want to manage multiple stacks.

    (John Leach/Brightbox are doing a good job of producing debs of Ruby EE that replace the system stack, which is a much more friendly way of distributing Ruby EE)

    As we migrate to Lucid, we’re facing this decision again, and I’m hoping things work out with squeeze/sid or maverick packages, as I’d much rather work with Debian/Ubuntu upstream than Ruby EE.

47. Can I just correct something I wrote above.

    ” Regarding gem update –system: IIUC, it does not “replace code from the Rubygems Debian package” as Lucas said, but just installs a newer version in /var/lib/gems/1.8/rubygems-update-x.x.x/ ”

    Well, I just did a gem update –system on an Ubuntu Lucid box, by patching /usr/lib/ruby/1.8/rubygems/commands/update_command.rb (removing the ‘fail’ statement and inserting the original code which had been removed at that point)

    This appeared to work, but then I ended up with gems being installed under /usr/lib/ruby/gems/1.8/gems, and the code for rubygems itself was installed in /usr/local/lib/site_ruby/1.8

    I guess this demonstrates the real reason for Debian disabling the update mechanism: it’s because if an update were installed, it would be missing Debian’s mods.

    So the sysadmin has two choices:

    (1) go and find new .deb files for rubygems, rubygems1.8 and rubygems-doc, which eventually I got from http://gb.archive.ubuntu.com/ubuntu/pool/universe/libg/libgems-ruby/ – then rubygems works in the way Debian decided is best

    (2) install rubygems from source, in which case rubygems works in the way its authors intended.

48. Pingback: Noticias del Proyecto Debian – 21 de septiembre 2010 | Gustavo Pimentel's GNU/Linux Blog

49. I just want to thank you, man. I maintain ruby for ALT Linux and I took some patches for redmine, packaging rails applications was easy. And I really liked you packaging policy (though I can’t agree on everything).

    I am doing this only for my personal fun, since tr00 rubysts doesn’t give a sh*t about “packages” and “distributions”. It’s fun when you make several people happy with redmine or alexandria packages. It’s fun when you can make several people happy with fixed bug (and in would be OK if upstream just silently ignore you).

    Do what you have (and like) to do and ignore that herd of brain-damaged monkeys.

50. The frustration is also felt on the python side of the fence see http://www.dzone.com/links/r/the_distros_have_killed_python.html

    We need to find a solution for Python and Ruby (and lua etc.)

Comments are closed.