There’s a pattern that comes up from time to time in the release management of free software projects.
To allow for big, disruptive changes, a new development branch is created. Most of the developers’ focus moves to the development branch. However at the same time, the users’ focus stays on the stable branch.
As a result:
- The development branch lacks user testing, and tends to make slower progress towards stabilization.
- Since users continue to use the stable branch, it is tempting for developers to spend time backporting new features to the stable branch instead of improving the development branch to get it stable.
This situation can grow up to a quasi-deadlock, with people questioning whether it was a good idea to do such a massive fork in the first place, and if it is a good idea to even spend time switching to the new branch.
To make things more unclear, the development branch is often declared “stable” by its developers, before most of the libraries or applications have been ported to it.
This has happened at least three times.
First, in the Linux 2.4 / 2.5 era. Wikipedia describes the situation like this:
Before the 2.6 series, there was a stable branch (2.4) where only relatively minor and safe changes were merged, and an unstable branch (2.5), where bigger changes and cleanups were allowed. Both of these branches had been maintained by the same set of people, led by Torvalds. This meant that users would always have a well-tested 2.4 version with the latest security and bug fixes to use, though they would have to wait for the features which went into the 2.5 branch. The downside of this was that the “stable” kernel ended up so far behind that it no longer supported recent hardware and lacked needed features. In the late 2.5 kernel series, some maintainers elected to try backporting of their changes to the stable kernel series, which resulted in bugs being introduced into the 2.4 kernel series. The 2.5 branch was then eventually declared stable and renamed to 2.6. But instead of opening an unstable 2.7 branch, the kernel developers decided to continue putting major changes into the 2.6 branch, which would then be released at a pace faster than 2.4.x but slower than 2.5.x. This had the desirable effect of making new features more quickly available and getting more testing of the new code, which was added in smaller batches and easier to test.
Then, in the Ruby community. In 2007, Ruby 1.8.6 was the stable version of Ruby. Ruby 1.9.0 was released on 2007-12-26, without being declared stable, as a snapshot from Ruby’s trunk branch, and most of the development’s attention moved to 1.9.x. On 2009-01-31, Ruby 1.9.1 was the first release of the 1.9 branch to be declared stable. But at the same time, the disruptive changes introduced in Ruby 1.9 made users stay with Ruby 1.8, as many libraries (gems) remained incompatible with Ruby 1.9.x. Debian provided packages for both branches of Ruby in Squeeze (2011) but only changed the default to 1.9 in 2012 (in a stable release with Wheezy – 2013).
Finally, in the Python community. Similarly to what happened with Ruby 1.9, Python 3.0 was released in December 2008. Releases from the 3.x branch have been shipped in Debian Squeeze (3.1), Wheezy (3.2), Jessie (3.4). But the ‘python’ command still points to 2.7 (I don’t think that there are plans to make it point to 3.x, making python 3.x essentially a different language), and there are talks about really getting rid of Python 2.7 in Buster (Stretch+1, Jessie+2).
In retrospect, and looking at what those projects have been doing in recent years, it is probably a better idea to break early, break often, and fix a constant stream of breakages, on a regular basis, even if that means temporarily exposing breakage to users, and spending more time seeking strategies to limit the damage caused by introducing breakage. What also changed since the time those branches were introduced is the increased popularity of automated testing and continuous integration, which makes it easier to measure breakage caused by disruptive changes. Distributions are in a good position to help here, by being able to provide early feedback to upstream projects about potentially disruptive changes. And distributions also have good motivations to help here, because it is usually not a great solution to ship two incompatible branches of the same project.
(I wonder if there are other occurrences of the same pattern?)
Update: There’s a discussion about this post on HN
Another situation worse than these is Rust’: you have a stable channel but basically every popular framework makes use of features from the nightly channel. I think distros can’t ship nightly Rust as the only Rust as some developers are saying, but if they could, well, software distributors could be in a great position to potentially ship ~stable versions of nightly rustc.
Speaking with my application developer hat on: I’ve never been contacted by a distro about breakage due to an API change.
Speaking with my application user hat on: it’s not fun having things break when you just want to get on with your work.
Would you consider OS X an example of this? Users were stuck on OS 9 for a long time while OS X was being developed — 10.2 was the first version that was even remotely usable as a daily driver.
Then again, while it is technically possible to make a continuous refactoring from OS 9 to OS X, I’m not sure it’d be preferable. There is so much that is completely different between these environments that it would take forever, and require many times more work, both on the part of Apple and their developers and users.
At some point, it’s gotta be more worthwhile to just yank the tooth out, right?
You can add TYPO3 cms version 5 that became Phoenix and eventually died.
Windows 95 and NT did the same too. We suffered for 10 years.
I would resume it like this: backward compatibility is essential even if progranmers don’t like it.
Python has greatly benefited from the stability of 2.7 allowed by branched development on 3.x, proving that once a tool gets good enough, reliability is seen as far more important than improvement. And the resulting compatibility libraries show that all of the improvements of the past eight years don’t really amount to much. This point was driven home when I saw the early adopter fans drooling over 3.6’s new f’…’ formatting strings as if they are some kind of revolution. Long story short, they aren’t.
I’m even more worried about Rust, it seems to be in a real version crisis.
I am only speaking from a very cursory experience, but trying to use some crates via cargo has led me into the dark territory of not just dependency on nightly rustc, but on nightlies of a timestamp (and specifically not earlier or later nightlies.)
To me this seems like insanity, before I started really exploring Rust, it seemed like a really nice option, now though I’m terrified of this version landscape.
What’s the alternative that you suggest? Never release breaking changes?
I think those who proclaim Rust has a problem here should look more closely. Yes, there are some crates which are nightly-only. No, the most relevant crates like serde and Diesel run on stable just fine (though they take an extra code gen step). Rust nightly is 100% compatible with stable (although using a few APIs may elicit deprecation warnings).
Sure, there are some features that haven’t been stabilized yet, but the Rust team is introducing new features at an awesome pace. For example, the next release, due in roughly 6 weeks, will have custom derives (a.k.a. macros 1.1), and at that point both serde and Diesel will run on stable without an extra step.
@Bl@ster & @Jasonm23 : I don’t think you understand the Rust model about Nightly vs Stable.
Nightly is a sandbox for experimental feature that may be included in the language one day, or may not. There is no stability guarantee at all in the Nightly version. It’s OK to be using nightly for toy projects or experiments, but it’s never been intended for production use and you should not use it this way !
If you encounter crates (libraries in rust parlance) that requires nightly, you must consider them as experiments with no stability guaranteed either, they are not intended for production usage (otherwise they would have a «stable compliant» version in parallel, see Serde or Diesel).
Don’t use Rust nightly for business/production stuff !
Using Nightly should never be mandatory, and if you are facing a situation where you need Nightly for a feature (let say SIMD support, or using an alternative allocator), then it means that Rust is not yet ready for your use-case in production. Feel free to perform experiments on Nightly and give feed-back to the community, to help Rust become the language you need but don’t use it in production now !
This is a public service announcement.
@StyMaar: given that rust stable is probably never going to support compiler plugins, and plugins being really-very-incredibly-useful, I’d actually argue for the reverse: use nightly rust only, ignore the lack of stability guarantees; the core features are stable enough, anything really unstable is feature-flag-gated so you can use nightly without worrying about it.
@Jacek Lach Check https://github.com/rust-lang/rfcs/blob/master/text/1681-macros-1.1.md, compiler plugins feature are intended to be stablized.
Yes, and to second what York Xiang says, 11 days ago, full compiler plugins were accepted as an RFC: https://github.com/rust-lang/rfcs/blob/master/text/1566-proc-macros.md
So, “never going to be stabilized” is definitely false; macros 1.1 will be in stable as of the next release, which is the most common use of plugins, and then the full plugin infrastructure will land sometime in the future. It’s on track to become stable eventually, even though we can’t say when at this time.
Unrelated to Rust, I’m also not sure that comparing Ruby 1.8 -> 1.9 and Python 2 -> Python 3 is a good comparison; the Ruby world has virtually totally moved on to the new version, but the Python community is still fractured, with many who claim they will never move on. Python 2 is EOL in 2020, Ruby 1.8.7 was EOL’d two years ago.
Plain and simple, nobody uses a software that breaks often. You can play with it, at best. Plus, the “fix” part doesn’t happen in terms of hours or days, it can take weeks or months. The probable result is people not updating. Suicidal theory. It makes sense only in the “bazaar” where everybody does what he likes and who cares of the rest.
Elektra had the reverse issue: during 0.6 Elektra broke its API so often that it lost most of its user base. Some of the breaking changes would have been avoidable if they had been excluded in releases.
So I definitely do not agree to “break early, break often” and since Elektra 0.7, this policy has changed, slowly catching up again.
But I agree that “unstable” branches are not a solution. Instead we need proper migration strategies. For example in 0.8.15 Elektra changed the file where the mountpoints are stored without any breaking change visible to users: When mounting the first time the new file is used, otherwise Elektra will continue to use the old file. And there is a script to migrate to the new format (kdb upgrade-bootstrap).
Btw. I wonder why nobody mentioned KDE3->KDE4->KDE5 transitions?