There’s been quite a lot of Google talk of late, both out on the wild and wooly interweb and right here on this humble little blog. It’ll take a back seat, no doubt, to tomorrow’s news, which will see Europa, the GPLv3 and the iPhone unleashed upon the waiting masses, but it’ll pick back up. Particularly if we do switch to Apps, because the testing experience thus far has been an interesting one. Mostly positive, particularly on the migration front, but with the requisite ups and downs.
Most of that news is, of course, well documented. Whether it’s the release of Desktop on Linux (CNET’s got that), the introduction of IMAP migration (I caught that one via Matt), or the rumors of a Grand Central acquisition (saw it first on Google Operating System), someone’s talking about it. Even something you’d think would be of interest only to a technical audience – Steve Yegge’s port of Rails to Javascript – got a lot airtime.
What’s gone under the radar, in my opinion, are Google’s Linux repositories. For once, it’s not really Google that’s the story here: it’s the mainstreaming of the repository overlay approach. This, as mundane and esoteric as it sounds, could be a significant development for commercial ISVs. Confused? Let me explain.
For those of you unfortunate enough to never have had the benefit of using package management systems, my condolences. From a users perspective, package management systems are revelatory the first time you use them. “You mean I don’t have to go to a website, download some random file, and double click to install it? Or worry about keeping that application updated over time? It’s like magic!” Like any other sufficiently advanced technology, yes, yes it is. Just like magic. Or, in the words of Tim Bray, “unreasonably f***ing great.”
These magical package management systems leverage, typically, community based repositories of, well, packages. Each package tells the operating system important things like what it is, where it needs to be installed, and most critically – what else needs to be installed for it to run. Lamentably, at least in Linux land, the packages themselves vary from distribution to distribution, in such a fashion that Debian, Fedora and Gentoo each need to have an Apache package maintainer, a MySQL package maintainer and so on. In spite of this regrettably redundant effort, the repositories are broad and far more complete than any commercial equivalent, with even the smallest repositories typically containing packages numbering in the thousands.
As diligent and responsive as the package maintainers from the respective distributions and operating systems are, however, there are always applications that someone would like to see included in a repository that are not present. Maybe they can’t find a maintainer, maybe they don’t comply with the licensing terms required for inclusion, or maybe they’re just not that popular – who knows. The important thing, from a user’s perspective anyway, is that they’re not there. Java, not an unpopular technology despite its flaws, only got into repositories this past year, after significant changes to its licensing. Presented with the lack of the application they’re looking for, the user has a choice to make: do I install this application separately and maintain it myself, or do I wait for it the package? Ok, yes, there’s a third: install an alternative application that is in the system, but forget that for now.
Seen from this angle, it’s apparent that the operating system/distribution and its maintainers are a control point. Not in a malicious sense, mind you, as the maintainers are in large part volunteers performing a public service; a public service that permits lazy users like me to install entire stacks from the command line. But a control point nonetheless. If you want access to their users, you’re obligated to submit to their process. Although those on the wrong side of a tangle with Debian would probably take exception to this assertion, I think it’s to the distros credit, in fact, that they decline to throw their weight around as heedlessly as does, say, Wal-Mart when it arbitrarily decides not to carry books or music that doesn’t meet its “moral” standards. They have strict rules, yes, but you know that going in and you know they play by those same rules.
Given the fact that the distro folks I know are all excellent people, I won’t push the Sprawl-Mart analogy further. But while the distributions may only be part time moral arbiters, they are – like it or not – full time gatekeepers to sizable communities, not to mention potential markets. For this reason, we’ve strongly encouraged our ISV customers with credible and viable open source projects to pursue relationships with the various distributions; have facilitated some of those very conversations on their behalf, in fact.
But what if that control point became less important over time? This is, I believe (and he can correct me if I’m wrong), the future that Jeff Waugh, formerly of Ubuntu, was discussing in this post. Anthony summarizes Jeff’s views with the following:
I think it’s fair to sum Jeff’s main point about the future of distros as one of “disintermediation” – that is removing, or at least minimising, the barriers between the authors and users of a piece of software, and in general getting closer and quicker collaboration between everyone involved and interested in developing and maintaining that software…
Jeff had a bunch more ideas in that vein – like not bothering with a central archive, but having users collect the packages they’re interested in from upstream sites all over the place, standardising, reducing or removing the “control” information for packages so that creating the correct packages for 80% of free software (that uses ./configure in the usual way, eg) was a complete no-brainer, and perhaps viewing distributions not so much as the gatekeepers and central players of the open source world, but perhaps more as systems of providing resources, assistance and opportunities to software authors who deal with their users directly – and in so doing have more of an opportunity for their users to actively participate in the project’s development, and have more feedback (and control) of how their project is perceived by its users.
Technically, there’s nothing preventing this disintermediation now. Donnie and the Gentoo crowd know this approach well as “overlays”. My initial implementation of Beryl (a Linux UI enhancement), as an example, was via a repository outside of the Ubuntu mainstream (Quinn’s, as I recall).
Further proving the point, to finally get back round to the topic at hand, are Google’s Linux repositories. Rather than fight half a dozen pitched battles to persuade distributions to include its software – battles it was likely to lose, in many cases – Google chose the MacArthur route. It bypassed the control points entirely, offering its software directly to end users, neatly packaged in the format of your choice (except for the poor Gentoo users). I’m sure they’re not the first commercial firm to take this approach, but I’d bet they’re the biggest (happy to be corrected on that score, trust me).
The implications of this are profound, I think – for users, maintainers, and ISVs alike. For the users, it’s certainly a short term win, in that they get access to applications more quickly via the standard installation mechanism. Longer term, this has the potential to be a Categorical Imperative type problem, but personally I’ll cross the bridge of too many repositories to manage when I get to it. The maintainers likewise may see a short term win – less packages to be maintained, but more software available – partially offset by longer term concerns: what does this mean to the importance of the distro? For ISVs, it’s an approach I expect to see increasingly adopted, if only as an interim step towards full product adoption. In fact, I’d recommend it.
In any event, this is a trend to watch – and one that other, smarter folks have already predicted. I don’t know that anyone yet has the answers here, but that it will impact the future of the distributions – as Jeff contends – seems to me inarguable.
