Blogs

RedMonk

Skip to content

Google and the Future of Package Management

There’s been quite a lot of Google talk of late, both out on the wild and wooly interweb and right here on this humble little blog. It’ll take a back seat, no doubt, to tomorrow’s news, which will see Europa, the GPLv3 and the iPhone unleashed upon the waiting masses, but it’ll pick back up. Particularly if we do switch to Apps, because the testing experience thus far has been an interesting one. Mostly positive, particularly on the migration front, but with the requisite ups and downs.

Most of that news is, of course, well documented. Whether it’s the release of Desktop on Linux (CNET’s got that), the introduction of IMAP migration (I caught that one via Matt), or the rumors of a Grand Central acquisition (saw it first on Google Operating System), someone’s talking about it. Even something you’d think would be of interest only to a technical audience – Steve Yegge’s port of Rails to Javascript – got a lot airtime.

What’s gone under the radar, in my opinion, are Google’s Linux repositories. For once, it’s not really Google that’s the story here: it’s the mainstreaming of the repository overlay approach. This, as mundane and esoteric as it sounds, could be a significant development for commercial ISVs. Confused? Let me explain.

For those of you unfortunate enough to never have had the benefit of using package management systems, my condolences. From a users perspective, package management systems are revelatory the first time you use them. “You mean I don’t have to go to a website, download some random file, and double click to install it? Or worry about keeping that application updated over time? It’s like magic!” Like any other sufficiently advanced technology, yes, yes it is. Just like magic. Or, in the words of Tim Bray, “unreasonably f***ing great.”

These magical package management systems leverage, typically, community based repositories of, well, packages. Each package tells the operating system important things like what it is, where it needs to be installed, and most critically – what else needs to be installed for it to run. Lamentably, at least in Linux land, the packages themselves vary from distribution to distribution, in such a fashion that Debian, Fedora and Gentoo each need to have an Apache package maintainer, a MySQL package maintainer and so on. In spite of this regrettably redundant effort, the repositories are broad and far more complete than any commercial equivalent, with even the smallest repositories typically containing packages numbering in the thousands.

As diligent and responsive as the package maintainers from the respective distributions and operating systems are, however, there are always applications that someone would like to see included in a repository that are not present. Maybe they can’t find a maintainer, maybe they don’t comply with the licensing terms required for inclusion, or maybe they’re just not that popular – who knows. The important thing, from a user’s perspective anyway, is that they’re not there. Java, not an unpopular technology despite its flaws, only got into repositories this past year, after significant changes to its licensing. Presented with the lack of the application they’re looking for, the user has a choice to make: do I install this application separately and maintain it myself, or do I wait for it the package? Ok, yes, there’s a third: install an alternative application that is in the system, but forget that for now.

Seen from this angle, it’s apparent that the operating system/distribution and its maintainers are a control point. Not in a malicious sense, mind you, as the maintainers are in large part volunteers performing a public service; a public service that permits lazy users like me to install entire stacks from the command line. But a control point nonetheless. If you want access to their users, you’re obligated to submit to their process. Although those on the wrong side of a tangle with Debian would probably take exception to this assertion, I think it’s to the distros credit, in fact, that they decline to throw their weight around as heedlessly as does, say, Wal-Mart when it arbitrarily decides not to carry books or music that doesn’t meet its “moral” standards. They have strict rules, yes, but you know that going in and you know they play by those same rules.

Given the fact that the distro folks I know are all excellent people, I won’t push the Sprawl-Mart analogy further. But while the distributions may only be part time moral arbiters, they are – like it or not – full time gatekeepers to sizable communities, not to mention potential markets. For this reason, we’ve strongly encouraged our ISV customers with credible and viable open source projects to pursue relationships with the various distributions; have facilitated some of those very conversations on their behalf, in fact.

But what if that control point became less important over time? This is, I believe (and he can correct me if I’m wrong), the future that Jeff Waugh, formerly of Ubuntu, was discussing in this post. Anthony summarizes Jeff’s views with the following:

I think it’s fair to sum Jeff’s main point about the future of distros as one of “disintermediation” – that is removing, or at least minimising, the barriers between the authors and users of a piece of software, and in general getting closer and quicker collaboration between everyone involved and interested in developing and maintaining that software…

Jeff had a bunch more ideas in that vein – like not bothering with a central archive, but having users collect the packages they’re interested in from upstream sites all over the place, standardising, reducing or removing the “control” information for packages so that creating the correct packages for 80% of free software (that uses ./configure in the usual way, eg) was a complete no-brainer, and perhaps viewing distributions not so much as the gatekeepers and central players of the open source world, but perhaps more as systems of providing resources, assistance and opportunities to software authors who deal with their users directly – and in so doing have more of an opportunity for their users to actively participate in the project’s development, and have more feedback (and control) of how their project is perceived by its users.

Technically, there’s nothing preventing this disintermediation now. Donnie and the Gentoo crowd know this approach well as “overlays”. My initial implementation of Beryl (a Linux UI enhancement), as an example, was via a repository outside of the Ubuntu mainstream (Quinn’s, as I recall).

Further proving the point, to finally get back round to the topic at hand, are Google’s Linux repositories. Rather than fight half a dozen pitched battles to persuade distributions to include its software – battles it was likely to lose, in many cases – Google chose the MacArthur route. It bypassed the control points entirely, offering its software directly to end users, neatly packaged in the format of your choice (except for the poor Gentoo users). I’m sure they’re not the first commercial firm to take this approach, but I’d bet they’re the biggest (happy to be corrected on that score, trust me).

The implications of this are profound, I think – for users, maintainers, and ISVs alike. For the users, it’s certainly a short term win, in that they get access to applications more quickly via the standard installation mechanism. Longer term, this has the potential to be a Categorical Imperative type problem, but personally I’ll cross the bridge of too many repositories to manage when I get to it. The maintainers likewise may see a short term win – less packages to be maintained, but more software available – partially offset by longer term concerns: what does this mean to the importance of the distro? For ISVs, it’s an approach I expect to see increasingly adopted, if only as an interim step towards full product adoption. In fact, I’d recommend it.

In any event, this is a trend to watch – and one that other, smarter folks have already predicted. I don’t know that anyone yet has the answers here, but that it will impact the future of the distributions – as Jeff contends – seems to me inarguable.

Categories: Open Source.

  • Pingback: Arcadian Visions » Managing Google’s Packages

  • http://michaelbernstein.com Michael R. Bernstein

    So, there is a natural infrastructure play here: Google can offer integrated repo mgmt. to their Google Code project hosting.

  • http://robilad.livejournal.com Dalibor Topic

    I think, like the Google Pack, Google could actually take over the (presumably financially interesting) role of being the default channel for ISVs for the cross-platform software management experience. But, like Google Pack, I doubt this will go very far beyond the initial splash.

    Google has a similar problem like many proprietary software makers wrt their Linux offers: getting the software out there in a way that does not make the users suffer. They could go to distributions directly, but in Google’s case, that would almost certainly be a failure, given that the Picasa EULA requires distributors to do some rather outlandish things, like having to bind their organizations to Google’s EULA, or mandating that a single copy (+ one backup copy) of the software exists, which makes mirroring of the distribution pointless, etc. The proprietary distribution model with a single, controlled point of distribution, and the Linux distribution model with many autonomous points of distribution, don’t play well with each other, and licenses that try to enforce the former model can’t really fit into the later.

    One major problem with the routing around the distributions in order to preserve restrictive licensing is that one’s own packages can’t rely on the distributions caring about them (because they don’t know they exist), i.e. in the long run stuff breaks, because the underlying package dependencies change, and then users suffer from a bad upgrade/installation experience, quite unlike magic. The external proprietary software repositories for Fedora, for example, have long suffered from such ‘platform drift’ effects.

    The LSB is supposed to make that problem go away for ISVs, I guess. I have no idea how well it does, as I’ve never heard of an LSB-compliant package being advertised as such. So I guess it’s not being marketed to me, as an end user. :)

    On the other hand, the Linux channel is the only channel where ISVs can ‘make a deal’ with the channel operators, and try to establish their products as part of the ‘magic’ out-of-the-box experience of the platform. Neither Microsoft nor Apple are really interested in that. Cutting themselves free from the necessity to cooperate with distributions can be an initially appealing proposition, but in general that also means being cut off from the user support channels that distributions offer, and requires an investment in one’s own infrastructure.

    Whether that makes sense is a matter of the community & branding strategy, I guess.

  • Pingback: Daily links - 29.06.2007 : TrendPlex

  • Pingback: » News to know: iPhone alternatives; Desktop virtualization diaries; Hacker’s road to riches | Between the Lines | ZDNet.com

  • http://spyderous.livejournal.com Donnie Berkholz

    Sun Studio’s Linux version is available as a big pack of RPMs. 75 or so.

  • Jeffrey W. Baker

    You know, disintermediation might sound good when you say it, but I think it’s a bad idea. The nice thing about the Ubuntu or Debian process is there exists a layer of people capable of making the decision that some software is not good enough to expose to the average user. If the software fails to build on some architectures, it won’t make it in. If the software has some grave bug (removing another package’s files, for example) then it won’t go in. The package maintainers are responsible for timely security fixes that sometimes even the upstream provider won’t fix.

    I wouldn’t trust Google with that role because their incentives are not properly aligned with my goals as a user, and they don’t have any track record of performing that function.

  • Pingback: » Google to the rescue: FETCH! With Ruff Ruffman CQ2