The Future of Development: It’s About Differentiation

“Great inventory! But I miss your point, I guess. I see 2 greenfield companies that built something from scratch with as many free tools as were available. I see opportunism, a lack of strategy and focus, way too much diversity, next to a weighed choice of tools-for-the-job and outsourcing / cloudsourcing where apt

What I see when looking at Facebook and Twitter is a very opportune choice and use of tools that are up-for-grabs. I’d love to see their 5-year roadmap on how all those different languages and tools are going to help support their case, strengthen their competitive position, etc. etc. etc…”

– Martin Linssen, commenting on “Beyond Cassandra: Facebook, Twitter and the Future of Development”

Funny how two people can look at the same data and come to such different conclusions. Martin’s comment typifies one class of objections raised to my piece about the future of software development, centering around a perceived excessive heterogeneity that is the necessary result of a lack of a big picture strategy.

In that respect, I frankly see little to differentiate Facebook or Twitter from their enterprise brethren. The Platonic ideal of an IT environment frequently championed by CIOs, IT managers and other enterprise technology executives is both top down prescriptive and regimented. These languages are permitted, these are our approved suppliers, this is the accepted database approach. And so on.

The only problem with that ideal is that it’s fiction. The CIO is the last to know, remember. Consider the language proliferation alluded to in the comment above – “all those languages.” Effectively zero enterprises today have standardized on a single language. Or two. Shops will have their own tendencies and preferences, of course, but language heterogeneity is not the exception, but the norm. Whether we’re talking about Facebook or Bank of America, the officially approved languages are inevitably a mere subset of the actual languages employed.

And as for the five year roadmap, I would suggest that any enterprise – web or otherwise – that cements its fixed idea of its IT plans for the next five years is doing it wrong. It is not possible for any of us, in any capacity, to anticipate the future direction of infrastructure sufficiently to create a roadmap of that duration with any reasonable degree of confidence. How, for example, will your software development targets evolve in light of programming language and framework developments, advances in Platform-as-a-Service offerings, radically evolving database options, etc (coverage)?

While Facebook and Twitter are clearly distinct from your average Fortune 500 entity then, their approaches with respect to technology planning will have more in common than is commonly realized. Why? Because the new software development models look a lot like the old software development models. The open methodologies favored by the likes of Facebook and Twitter epitomize in many respects the realization that software is frequently non-differentiating, but this idea is anything but new. The market for packaged applications, for example, grew out of the realization that it made little sense for enterprises to roll their own general business applications. What profit is there in building your own CRM system? Or ERP? The answer, almost always, is none.

Ask yourself why Google publishes details on Big Table or Map Reduce, or Amazon Dynamo, for example. Applied to appropriate workloads, these technologies can offer order of magnitude improvements in performance. Traditional software economics would have these assets highly protected. If you can process data more quickly than your competitor, you’ll perform better. This simplistic analysis has ignored the larger economic picture, however. Specifically it obscures the resource cost of closed systems. Secret, proprietary systems are typically known and familiar to their owners or paying customers only. Open systems, on the other hand, are taught in academia and used widely, lowering training and talent acquisition costs. What is the scarcer resource: performant systems, or human resources with a fundamental understanding of these systems?

It seems clear that Amazon, Facebook, Google, Twitter and others have come to the conclusion that it’s frequently beneficial to trade technology for developer familiarity.

Which is not to say that there do not exist opportunities for competitive advantage in software. Google’s search algorithms, for example, are a closely held secret, and likely for good reason. Nor is there incentive for the aforementioned examples like Facebook or Twitter to make it simple for competitors to replicate their entire stack.

When I look at Facebook and Twitter, then, I see something very different than Martin. I don’t see a lack of planning, I see adaptation to a highly dynamic set of requirements. I don’t see too much diversity, I see a recognition that there are different tools for different jobs. Using the wrong library because it’s written in an “approved” language makes little sense to me.

But most of all I see a fundamental understanding that, like general business software in the enterprise, infrastructure software is not likely to be highly differentiating, and thus a competitive advantage. And that being the case, it’s more expensive to author software purely internally than it is to consume or produce open source software.

Which is why I think both companies do in fact have quite a bit to teach us about the way that software development is and will be done. Even if they can’t give us a five year roadmap.