Linux, Solaris & Windows: The Application Management Q&A

Installing MySQL on Gentoo

Originally uploaded by sogrady.

As one prominent member of the open source community and I were discussing last week at OSCON, it’s human nature to occasionally take important things for granted, and not give them the attention that they deserve. Such is the case, in my opinion, with the application management situation in the Linux and, to a lesser extent, Solaris worlds. It’s my belief, as I mentioned to folks from Sun and the OSDL last week – and Red Hat the week before, that both of these operating systems have a significant advantage over Windows in the application management department. To explain why, let’s do a Q&A.

Q: Before we get into the why, can you explain what you mean by application management? The term seems to mean different things to different people.
A: True enough. For my purposes here, I’m simply defining application management as the ability for an end user (of any type, desktop or server, test or production) to procure, install and manage an application that sits on top of the OS. That could be anything from end user note taking, PIM or office productivity applications to higher end database, web or application servers.

Q: Ok – given that definition, how do Linux and Solaris have an advantage over Windows?
A: Well, to answer that let’s first look at the application management process for Windows. First off, the process is distinct as one might expect for Microsoft applications versus non-Microsoft applications. For the latter, software is procured, either through physical media or download (often hunting through pages of sites to find the actual download), and installed via some form of installer software. Typically, this software registers itself with Windows and is centrally removable via the Add/Remove Programs facility that the operating system provides.

But from that point forward, Windows as an operating system does little, if anything, to assist in the ongoing management of that software. Patching, application updates, and other modifications to the installed applications are essentially left to the user, the application provider, or third party software to manage. For Windows applications, the situation is somewhat improved, as much (but not all) of the Microsoft portfolio can be managed on an ongoing basis via the Office and Windows Update web services.

Application providers have begun to realize the impact of this and are scrambling to provide their own update facilities; Firefox may be the most notable recent example. But the fact of the matter is that a great deal of Windows based applications provide no such facility, and as a result become out-of-date, and worse, may become vulnerable due to a lack of security patches. Anyone who maintains computers for friends and family, or servers for the home office or small enterprise likely knows what I’m talking about.

Further, end users seeking software functionality without an idea in mind of the application they want to use have no way, in Windows, for seeking potential solutions. At that point its consultants, or maybe Google. It’s not possible, for example, to query Windows and say, “I need an SSH client – what do you have available?”

In short, the application management solutions for Windows are barely adequate in the case of Microsoft packages, and even less so with the volume of non-Microsoft software written for Windows.

Q: With you so far, but how are Linux and/or Solaris any better?
A: Well, its mostly because both have evolved over time an infrastructure that streamlines the discovery, installation and ongoing management of applications. Ironically, this is likely a result of the fact that the typical installation for Unix based operating systems (./configure, make, make install) is more difficult for non-technical users than simply clicking on a .exe file in Windows.

Let’s take Linux first, as it has in my opinion the more sophisticated application management options at the present time. I’ve written on several occasions in the past (see here or here) about Gentoo’s Portage, which essentially is a package/application management application at the heart of the distro that leverages a massive, community maintained library of applications available for the platform. You simply tell Portage what application you want, and it will

1. Determine what other applications are needed
2. Download them all for you
3. Install the application and its dependencies
4. Allow you to update and/or remove them as necessary via a central database

Need Apache? It’s in there, with multiple versions. MySQL? Ditto. (see inset picture) Need an SSH client or library? A quick query of Portage turns up over 20 SSH related tools. Virtually anything you might want is in there, or will be soon.

Gentoo is certainly not the only distribution with such a facility, of course. Most distros that I’ve worked with have some equivalent functionality; Debian and Ubuntu share apt-get, Fedora has yum, NLD/SuSE has Red Carpet, and so on. The concept then is hardly a unique one; the differences in the individual implementations seem to be first in the granularity and functionality of the tool involved, and second in the size of the application library behind the tools.

And speaking of library size, that brings us to Solaris. Solaris and its open-source instantiation OpenSolaris do not have as yet – with the exception of efforts like Schillix – a sizable community of different Solaris distributions (though Jonathan Schwartz did speak positively of the this possibility at OSCON). While they lack variety there, however, they have at their disposal a tool similar to Debian’s apt-get called pkg-get, which leverages the library of Solaris-compatible applications maintained by the folks at Blastwave.org. Compared to the library behind a facility like Portage, however, the Blastwave library is relatively small (some 300+ applications, last time I checked Dennis Clarke wrote in to provide a number of 1100+ applications in the Blastwave library, which is supported by my unofficial count of 1115 applications as of this morning – taken from here. Thanks for the update, Dennis. As for the original comparison of libraries, I did a similar check on packages.gentoo.org at the same time and received a count of 9928, so I believe the point still stands. It’s important to note, however, that Gentoo has been around a lot longer and therefore should be expected to have the more sizable library.).

Q: But isn’t there the possibility that these application management facilities will compete with commercial versions, such as the Red Hat network?
A: Great question. The answer, in my opinion, is no. Here’s why:

1. I accept as a given that community maintained libraries will always be more comprehensive and long tail-ish than commercial equivalents
2. I further accept as a given that big commercial entities would prefer to pull their version of, say, MySQL from a commercial network versus a library maintained by individuals without commercial needs in mind
3. I accept as likely that commercial ISVs would prefer to work with commercial networks

In short, I think there’s an excellent opportunity for commercial and community networks to function seamlessly side-by-side, each serving different needs and having different strengths. For example, I spoke with Red Hat two weeks ago about updates to the Red Hat Network and recommended that they provide hooks or APIs from yum into the network. Either network by itself is incomplete, in my view. Commercial networks will always lag non-commercial ones in terms of application breadth and depth, and non-commercial networks are unlikely to be perceived as a viable basis for higher end enterprise needs.

Q: So what’s the catch?
A: The catch, at least as far as Linux is concerned, is that each individual distribution is effectively creating redundant libraries. This means that at least some of the effort poured into the distributions mentioned above is wasted effort. Because each distribution infrastructure operates differently (e.g. source code for Gentoo, versus RPMs for Fedora) every distro needs to repackage the individual applications for their own Linux flavor. Thus far this has proved to be sustainable – it’d be difficult to argue against the strength of the Debian or Gentoo libraries – but it also seems clear that if, in the longer term, a standard packaging mechanism could be developed and shared amongst the various distributions that’d be a big time saver for the individual communities.

In this respect, Solaris/OpenSolaris would seem to have something of an advantage in that they can solve this problem before different packaging systems proliferate by annointing the preferred approach. Maybe there’s room for both pkg-get and Portaris, but having a standard package format would be hugely benefical I think. That point is important, because I’m not saying any one of the package applications – apt-get, Portage, Red Carpet or yum should be adopted universally; rather I’m saying that having them all operating off of the same package structure – and therefore, library – would be ideal.

Q: Do you think it’s realistic that the different distributions can agree on a standardized method of distributing packages? Isn’t that what RPM was for?
A: No, and yes. I don’t think that at least in the short term its feasible that all of the different distros can coalesce around a single way of doing things; they’ve evolved to meet their own needs, their are backward-compatability and dependency issues to consider, and then there are the sea of existing packages. But I don’t think this is necessary. As to the second question, yes RPM was designed by Red Hat I believe at least in part to solve some of these very problems. And because it enjoys widespread commercial support, this might be a format to look to as far as standardization goes – particularly given its adoption within the LSB. The format is just part of the solution, not the solution itself.

Q: Why don’t you think it matters that the Linux distros can’t agree on a single format?
A: Because the larger distributions are doing just fine maintaining their own libraries at the moment. All of those mentioned here have very vibrant application ecosystems served by their different application management systems. As it stands right now, it’s easier to maintain a complex application portfolio with respect to updates and patches on Linux or Solaris than it is on Windows. As alluded to above, however, I do believe that Linux and potentially Solaris should market this strength more aggressively than they have in the past.

Q: In what respect?
A: When I talk about Portage to people, their eyes tend to glaze over and they start fidgeting. When I show it to them, however, they get it right away. The fact that, with a single command, I can update every application on my machine – or any subset I choose – is immensely interesting to technologists who appreciate the difficulty in maintaining applications over the longer term. Because its boring to talk about, however, it’s my opinion that many Linux advocates have largely ignored the value of the functionality – particularly given the lack of a Windows equivalent.

To be clear, however, I’m not recommending that Linux trumpet a “forest of applications” message. As Asa apparently noted during his OSCON keynote (missed it), the sheer volume of choices within Linux can serve as a deterrent to potential adopters. Instead the focus should be on maintaining the applications you need, longer term. Take Ubuntu as an example; this distro makes choices for the user (GNOME, Firefox, and so on), very much in the Firefox style of keeping things simple. But the apt-get infrastructure underlying Ubuntu can make maintaining that selection of applications very easy, while simplifying the process of dropping in a new application or two when necessary.

Q: Could Windows offer similar functionality?
A: They undoubtedly could, but several factors argue against it. First, the sheer volume of Windows applications is overwhelming. Maintaining that massive a library of applications would be difficult, even for a sizable community. Not impossible, but difficult. Second, it’s unclear to me how adept the Office/Windows Update infrastructure would be at managing non-Microsoft packages and applications. I frankly don’t know whether or not they have or could add that capability. Lastly, the model works well for Linux and probably for Solaris because they don’t have the same ambitions as Windows, feature/function-wise. There would seem to be more applications that competed with features in Windows than in Linux or Solaris, which would seem to dampen any enthusiasm by Microsoft for such an effort. All that said, however, could Microsoft offer more of a commercially oriented network, featuring its own products and certain tight partners? Certainly.

Q: Where might such an effort to standardize application management reside?
A: Unclear. Well, for OpenSolaris I’d expect it to start with the CAB. In the Linux world, which as mentioned seems to be trucking along fine without it, I can see different opportunities for different parties (might help the non-commercial folks gain weight as a more unified institution when competing against the big two, for example). For macro level conversations, however, I’d imagine some combination of the LSB and OSDL would have to be involved. When I spoke with them, the OSDL recommended the LSB. Irrespective of whether Linux addresses this on a cross-distro level, however, the advantage is still one that should be pressed.

Q: Any last thoughts?
A: Just that I believe that its important for commercial entities to acknowledge the community in these sorts of efforts. There are hundreds of PMC’s and devs out there busting their asses to bring users such as myself an outstanding end user experience. Rather than investing time and effort into competing with these communities, why not try and understand how to work alongside them, leveraging the different strengths each party brings to the table?

Update: Corrected the number of available applications within the Blastwave.org library, per comments from Blastwave’s Dennis Clarke. Provided a fixed Gentoo library number while I was at it. Appreciate the feedback, Dennis.