“IBM designed IMS with Rockwell and Caterpillar starting in 1966 for the Apollo program. IMS’s challenge was to inventory the very large bill of materials (BOM) for the Saturn V moon rocket and Apollo space vehicle.” – Wikipedia, IBM Information Management System
While the practice has a long history, the formalization – and more particularly, popularization – of open source is a relatively recent development. Until the process of releasing of Netscape Navigator rallied supporters of the term open source, the industry lacked a consistent vocabulary for dealing with the concept of source code available under a license. Which gave proprietary software the stage by default.
In the absence of a widely understood model for the generation and consumption of shared code, innovation in software was largely the product of commercial vendors producing proprietary software. Starting from scratch, with few exceptions most organizations outside of finance and defense could not reasonably expect to sustainably compete with vendors whose business it was to write software. Rather than try, businesses predictably chose to outsource the process of development to vendors like IBM, Microsoft, Oracle and SAP. This is the dynamic that gave us a PwC Top 10 Global Software leaderboard with a median age of 34.5 years.
With open source systemically lowering the overhead associated with development over the last decade, however, in house development has become increasingly viable from an economic perspective. The inevitable result has been greater availability of open source code. As users and vendors alike discover the benefits to open source – via models direct or indirect – the number of projects has swelled along with the rate of contributions. The database market, for example, has gone from less than half a dozen relevant open source projects to several dozen.
None of which is news even to casual observers of the open source market. What is interesting, however, is how these projects are being developed. In some ways, development today is a return to its roots. Consider the following list of open source projects:
- Cassandra
- Git
- Hadoop
- MongoDB
- Nginx
- Rails
Besides the availability of their source, what do these projects have in common? None were originally authored to be sold. All were built for purpose rather than sale; this is the return of roll-your-own [coverage]. Much as IBM once extracted IMS from an engagement that sent Americans to the moon, each of the above widely used projects was the byproduct of a business problem. Cassandra was written to manage the Facebook Inbox, Git to manage the Linux kernel source tree, Hadoop to power Yahoo’s search indexing, MongoDB to back 10gen’s original Java cloud vision, Nginx to serve pages for Rambler. And as for Rails, it was extracted from Basecamp.
This “extracted” software model is becoming routine. The innovation inherent in these projects, however, is anything but. Extracted software typically exists because a perfect solution does not, which means that in many cases it is introducing new capabilities rather than recreating existing products.
With substantial history behind it, the extracted model seems to be fairly well understood, conceptually. The open question now is about the volume of latent innovation that might emerge from extracted software in the years ahead. Projects like the list above indicate that internal innovation has accelerated over the last decade, driven by trends ranging from the greater availability of open source code to an industry-wide shift towards horizontally scaled-out architectures.
But as concern about the risks of open source thaws and is offset by wider understanding of the benefits, it is probable that waves of new internally developed projects will be released as open source. The majority of which will generate little activity and interest. But from the volume, we might expect the next Git, Hadoop or Rails.
As open source trends go, then, the extracted software model is one to watch.
