To judge by the vigorousness of the defense of its brand in recent weeks, it seems reasonably safe to conclude that open source can be evaluated as an institution in certain contexts. Granted, it’s a problematic approach generally given the fact that open source is not a single organization, but rather a loose federation of many individual institutions each with its own goals, principles and philosophies.
And yet for comparative purposes it occasionally is useful to look at the performance of open source as a whole relative to the commercial, closed source marketplace: itself a massive collection of independent entities, notably.
The occasion of OSCON is, in many respects, an entirely arbitrary endpoint for a check on the health of the phenomenon it is ostensibly organized around. At the same time, as Tim puts it, OSCON is a relatively unique gathering of the tribes, which perhaps makes such a system-wide evaluation entirely appropriate.
Depending on which particular business school text you pick up, you might have seen the organizational lifecycle stages described as some approximation of the following:
- Startup
- Growth
- Maturity
- Decline
We must of course acknowledge the glaring impedance mismatch between mixed motive movements such as open source and profit-centric enterprises. Undoubtedly, open source will occasionally, even frequently, follow a different trajectory than will closed source alternatives.
Still, if we may see differences between open and closed source businesses in how they experience those respective stages, it is nonetheless useful to ask the question: if open source did progress through a normal organizational lifecycle, where would it be today?
The LAMP Stack
With the term itself twelve years old come January, we can probably conclude that open source has passed successfully beyond the startup stage of its existence. The multi-billion dollar revenue stream generated by projects such as Linux validates this assumption.
If not startup, however, where is it? Growth? Maturity? Decline? It’s an interesting question. What’s more interesting is how you try to answer it.
Metrics in the open source space have always been problematic. Unlike paid software which can be assessed with moderate confidence via revenue metrics, it is inherently difficult if not impossible to track with any reasonable level of precision open source adoption, usage and so on. Proxies such as project download counts occasionally used by commercial entities or project backers are not entirely useless – particularly when you have a history – but they are woefully underrepresentative at best, fatally flawed at worst.
While we have our own means of tracking such things, we will occasionally turn to basic tools such as Google Trends for a sanity check.
As noisy as that is as a data source, it’s an interesting proxy itself for general traction. Search traffic, as a fundamental part of the experience for users and developer alike, tells us something about the activity around a given project. Provided, of course, there are no lurking variables from a query perspective; there’s a reason I never use Google Trends for LinkedIn’s Voldemort key value store, as but one example.
What can Google Trends tell us, then, about open source? Two weeks ago the folks from Black Duck kindly asked me to explore a related question: what does the future of open source look like? The obvious starting point for me was the venerable LAMP stack. As the most fundamentally popular combination of open source software, it seemed logical to look at the future of the space by looking at the relative performance of its most visible projects, as measured by Google’s multi-year dataset. The results were frankly surprising.
Here’s Linux.
Here’s Apache.
Here’s MySQL.
And here’s PHP.
Curious results, I think you’d agree. To assess whether or not this trending was unique to LAMP, I looked at “open source” as a query string.
And the GPL.
As an aside, just in case the Eclipse community is wondering why they weren’t polled, all I can say is that it’s your fault for naming yourself after a teen vampire movie.
Back to the graphs: it is certainly possible to argue that the project-specific results are an artifact of project-specific variables. The rise in popularity of distros such as Ubuntu, for instance, could certainly be involved in the decline in Linux-specific searches. But the fact that we’re seeing virtually the same trend line across all of the queried terms seems to indicate the presence of a larger pattern. The question is what that pattern means.
Curiously, Microsoft might hold some answers for us.
The Microsoft Connection
After a decade or so of stagnancy on the financial markets, we’re beginning to see more public criticism of Microsoft CEO Steve Ballmer. While there are a variety of arguments with his tenure ranging from the absurd to the substantive, the central point of contention in recent weeks has been the underperformance of the firm’s share price (see here or here). Both pieces use a graph created by Erik Pukinskis, which I don’t have permission to use and thus will not reproduce here. The basis for it, however, is the graph below. For context, Steve Ballmer was promoted to the Chief Executive position in January of 2000, according to Wikipedia.
Essentially, Ballmer’s tenure has been marked by a flat to slight decline in the company’s share price. Record results or no, Microsoft has yet to convince the street of its prospects for growth. The CEO’s defenders have hastened to point out that his ascension came during the height of the government’s anti-trust activities, arguing that fall out was inevitable, as was some sluggishness on the financial markets. But what the share price reflects, both to myself and another CEO I spoke with last week, is nothing more or less than a mature software business.
Ballmer once asked his own employees an interesting question: “we’ve had the two most successful software products in history: what makes you think there will be a third?” The lack of an obvious response to that question means that growth has been more difficult to come by, which in turn implies that Microsoft has transitioned from growth into a mature phase of the organizational lifecycle. Or at least that that’s the market perception: doubtless some at Microsoft would take exception to that idea.
Microsoft’s financial fortunes are not, of course, the focus of most of those within the larger open source community. But it is interesting, in spite of the different metrics, that the trendline of its share price – flat to declining – is not terribly dissimilar from the search trendlines we see for the various open source communities we’ve looked at.
The question, then, is this: assuming for the sake of argument that Microsoft is in a mature market stage, is open source as well?
The Growth
The short answer is that, at least by the very basic Google Trends metrics we’re using here, it is not. We know that usage of the vanilla Linux search term is declining. But what about Linux in conjunction with the cloud?
A healthy, growth-oriented trendline. And how about in the mobile space?
We can argue over how “open source” Android really is, but we can’t argue that it is based on Linux and that you can, in fact, get at least portions of the source. And again, that it is the opposite of flat-to-declining traffic. How about databases? Are they experiencing the same general abatement in interest that the longtime standard, MySQL, is above? Far from it.
We’ve written extensively about the explosion of interest in non-traditional (read: non-relational) datastores, and the Google Trends data reflects this. Growth for NoSQL is, effectively, growth for open source because virtually all of the significant projects in the space are open source. Mike Stonebraker, in his talk at GlueCon, stated definitively that you cannot be competitive as a new project in the database space without being open source.
Consider the trajectory of Hadoop.
Besides disproving vague assertions of some sort of open source malaise, Hadoop is evidence of the importance of open source. Would it be the growing standard for large dataset processing that it is if it was a closed source product? It’s difficult to say, but certainly there is no similarly popular commercial closed source startup.
The Net
While selective interpretation of data such as that collected from Google Trends has and will continue to fuel the kind of rhetoric that claims that open source is in its decline phase, the truth is that it is more likely to be early stage growth. That leaves considerable runway for those in and around the open source community moving forward.
It’s undeniably true that certain projects may have peaked from a relative popularity perspective: it’s difficult to see technologies like Java or MySQL (coverage) becoming more popular than they are today from a percentage of marketshare perspective. Increased competition and accelerating fragmentation are driving the market away from consolidated LAMP-style stacks towards more specialized, heterogeneous alternatives (coverage). But it is equally true that while the growth rate may be slowing for already ubiquitous technologies like Apache, MySQL and PHP, it’s accelerating for a variety of technologies in every area of the market from mobile to cloud. Where certain projects within the open source community are seeing slowing growth and a transition to more static market positions, there are a variety of other open source projects that are seeing dramatic spikes in traction.
Open source has been successful enough that it is essentially impossible for closed source infrastructure software startups to compete effectively, which is why we see fewer and fewer of them. Closed source software is not going away, clearly, and in addition to being an enterprise and consumer fixture it is still overwhelmingly the leading money earner by several orders of magnitude. But if open source has been this successful absent the types of revenue models we envision that more properly align its strengths with customer needs (coverage), what might we see in future?
The Outlook
More open source. Quite a lot more, probably.
Not strictly because of the inevitable innovation we’ll see in revenue models for commercial open source organizations. That will certainly add vital oxygen to certain open source ecosystems, but it won’t be a panacea. The business of selling open source software, remember, is dwarfed by the business of using open source software to produce and sell other services (coverage). And yet historically, most of the focus on open source software has accrued to those who sold it. Today, attention and traction is shifting to those who are not in the business of selling software, but rather share their assets via a variety of open source mechanisms.
Why would commercial organizations willingly cede the fruits of their labor to a market that might include their competitors? Because for software that is non-differentiating, that is not a competitive advantage – which for most non-technology firms is virtually all of their software – it will cost more over the longer term to author software privately than it would publicly. Facebook and Twitter demonstrate this quite adequately (coverage), true, but it’s not just the web firms. We see it when a hosting company (Rackspace) and space agency (NASA) jointly author a cloud computing stack that neither intends to create a software sales business around. We see it when Lockheed Martin launches an open source social networking project. And so on.
None of these can be characterized as decisions driven by idealism or emotion; they are simply the most logical means of developing software for companies that aren’t in the business of selling software. Make no mistake: we’re seeing a resurgence of roll your own software (coverage). The difference this time around is that by sharing the code developed internally as open source, it becomes possible to amortize the development costs across multiple organizations with similar needs. Worst case, you have the opportunity to lower your costs of talent acquisition; this, presumably, is one of the justifications for Google sharing details on its MapReduce and Pregel processing approaches.
The outlook for open source software, then, appears to be this: maturity in certain market segments, outpaced by growth overall.
Which is really excellent news, because like Tim, OSCON’s my favorite conference.
Disclosure: We have multiple clients in the Linux space, and we worked with MySQL prior to and subsequent to its acquisition by Sun, which was also a client. Also in the open source database family, Basho, author of Riak, and Cloudera, backer of Hadoop, are RedMonk clients. Rackspace, one of the authors of the OpenStack project, is a RedMonk client, as is Microsoft.
