Open Source vs Integrated Innovation

Share via Twitter Share via Facebook Share via Linkedin Share via Reddit

Given the fact that we chatted about exactly this subject at OSCON back in August, Jason’s probably not going to be surprised by the fact that I don’t align with his view of open source and integration. Oversimplifying for the sake of brevity, I’d characterize the basic argument as open source = small parts, loosely joined, while closed source = (for lack of a better term) integrated innovation. But those are my words, here’s the argument in his:

I have long said that a key weakness of the OSS model in commercial deployment is founded in one of the fundamental elements of the OSS dev model – loosely coupled components. In an earlier blog entry I posited that the tester becomes a celebrity as OSS continues to be commercialized. SpikeSource is my favorite player in this space. The SpikeSource presentations I have sat through are focused on the difficulty in integration of loosely coupled components and the testing rigor required to bring OSS stack solutions up to par for commercial deployment. Of course, if you are to deploy a well tested solution from them or Red Hat or any other OSS player you will notice that the binary licenses and/or support contracts will specify that there shall be no modification of source code (or else the testing is not worth much).

Is there truth to that argument? You bet. Open source, most notably in the case of Linux [1], has often made a virtue of limited ambitions and focused design – eschewing the path of complex integration. This approach works for some IT staffs, and doesn’t work for others. Certainly it hasn’t been an impediment to sustained growth over a statistically significant period of time. But without a doubt, there are segments of the market that prefer to have everything neatly wrapped up for them – an opportunity that Microsoft perceived and capitalized on with Windows.

Now Jason as I’ve said many times before is a very smart guy, a very well connected guy, and when we spoke he supported his arguments with some very interesting examples. But I just can’t be convinced, however, that this approach is somehow a “weakness” or limitation of open source. In fact, I’m inclined to argue just the opposite: it’s a strength. Here’s why:

  1. My most obvious pushback would be with the alternative, in this case integrated innovation. It’s no secret that I’m skeptical of the benefits of integrated innovation, which in my view increasingly trade customer convenience for the inability to substitute alternative components and product cycle time. The link above has quotes from the likes of Joel Spolsky and Mini-Microsoft on the topic – who presumably have a decent view of the Microsoft internals – and more recently we’ve had the likes of Dare Obasanjo weigh in with the following:

    The primary way this manifests itself is integrated innovation, a buzzword that translates to more dependencies among shipping products, less control of one’s product destiny and longer ship cycles.

    This point isn’t as much about open source as much as its antithesis, of course, but I believe is important for contextual purposes. In short, open source certainly has its limitations, but the same is true of the alternative.

  2. I also think the notion that open source does not foster integration can by itself be proven wrong, or at least prone to exceptions. The plugin model popularized by projects like Eclipse goes a long way to providing many of the benefits of an integration heavy approach, without some of the limitations. The Eclipse community is an excellent example of how numerous independent entities – open source and commercial – can work together to produce a tightly integrated whole.
  3. Further, open source by its nature creates an economic opportunity for third parties interested in addressing the needs of customers demanding tighter integration of the various components, an additional level of comfort, support, etc. Interestingly, the integration is not limited to a single vendor or approach: we have everything from traditional services (Optaros, Virtuas) to on the fly developer integration and configuration (OpenLogic) to heavily tested and certified stacks (SourceLabs, SpikeSource).
  4. If we accept as above that open source can in fact integrate – if not to the extent that Microsoft can – I think it’s interesting to examine whether or not the reverse is true: can Microsoft decouple? If one accepts – as I do – that in the wider market there are customers up and down the spectrum from do-it-yourself to do-it-all-for-me, which is better able to service needs on both ends of the spectrum? Open source, which is generally unintegrated, but has integratable options? Or Microsoft, which generally pitches a more monolithic, integrated stack approach and has a limited ability to decouple?

Ultimately, Jason and I are more or less agreed on the approach that open source (generally) takes with respect to integration, we’re merely divided on the implications of that approach as it pertains to enterprise wants and needs. Which is pretty much how we left it at OSCON 😉

[1] Here’s a quote from Linus: “Nobody should start to undertake a large project. You start with a small _trivial_ project, and you should never expect it to get large. If you do, you’ll just overdesign and generally think it is more important than it likely is at that stage. Or worse, you might be scared away by the sheer size of the work you envision.”


  1. It's important not to take a "loose coupling good, tight coupling bad" mantra too seriously. Before you write this off as MS shilling, consider this: I started to question the "loose coupling == good" dogma a year or so before joining MS, when I bought a Mac Powerbook (in a fit of anti-MS spite after wasting a weekend cleaning spyware off the kids' computer!) and began to feel the power of the Dark Side. All that "it just works" stuff comes because somebody MADE it work without having to be hobbled by concern for implementing standards, open interfaces, keeping 3rd party hardware vendors in the loop, obsession with backwards compatiblity, etc.

    There's a cost to be paid for this. With a Mac, it's a simple dollar cost of having one supplier with no competition for most addons. Ironically Apple itself learned this lesson when IBM couldn't / wouldn't meet its processor requirements. For Microsoft, and I presume Apple, there's also an agility cost — depending on the cool features of some custom component means you're tied to their development schedule, and you slip when they slip. For a big product such as VS or SQL Server 2005, obviously all those little slips added up.

    I'm not sure whether the loose coupling cure is worse than the disease or not. My first impression of MS was that they needed to eat a LOT of that tasty loose coupling dogfood they sell. I'm not so sure now … ultimately the products will be judged by their quality, reliability, performance, etc. not when they shipped or whether their internal interfaces used open standards or not. Tight coupling offers various benefits to offset the drawbacks, including performance (standard interfaces are generally more abstract and the mapping back to the application layer typically consumes cycles), security (tight coupling can reduce the surface area exposed to attack from the outside), and so on.

    An OSS project has fewer alternatives to loose coupling than a big company does, if only because there's nobody in charge to wield the club to force SOME decision to be made. I guess they have to make a virtue of necessity. I don't have much first hand experience with open source projects, but open standards projects almost inevitably tend toward the some common denominator abstraction whenever things get controversial. That's probably a long-term evolutionary strength (keeping things simple and standard), but a short-term competitive weakness if your customers have more tightly integrated alternatives. Think of the relative market success of the X11 loosely coupled GUIs and the Mac and Windows tightly coupled ones. In other words, when asked to "trade customer convenience for the inability to substitute alternative components and product cycle time", customers tend to vote for convenience today rather than alternatives tomorrow.

    So, like almost everything, there's no clear answer IMHO. The penduluum has swung away from tight coupling over the last 10 years, and it should probably keep swinging that way some more inside MS, but I see signs that it is swinging back as people start to complain about the performance and security cost of, for example, the Web / XML interfaces. One little data point – "binary XML" was once considered almost an oxymoron, now W3C is being driven to define a standard for it because of performance concerns. [For the record, MS is on the side of the loosely coupled angels on this particular matter and opposes this effort … :-)]

    The way I suspect it will end up is that code will become increasingly tightly coupled because it is just SO HARD to come up with clean, well-implemented, standard interfaces between independently evolving components. Competitive pressures of one sort or another will reward those who make it work better over those who make it work the same. Data on the other hand will become increasingly loosely coupled and hence usable and reusable across products and platforms, because nobody cares how "good" your data is if they can't use it.

  2. hey stephen -I could have done with a link:

    monkchips on interdependency management http://www.redmonk.com/jgovernor/archives/000945….

    frankly tecosystems should also have linked to itself – your post on interdependency management

  3. Mike: very interesting comments. the point about OS X though, i find very interesting, simply because OS X as you no doubt are aware actually has open source at its core in Darwin. so clearly tight coupling can be accomplished on top of a loosely coupled, open source foundation.

    where we definitely align is that different customers have different needs and priorities. what i'm interested in seeing is, longer term, whose more effective at meeting the variety of needs. should be fun to watch.

    James: short served the both of us link-wise, i guess 🙂 thx for the correction.

    Jason: great post, many thx for the link. Wonder Twin powers indeed 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *