When I was in high school, and it came time to apply to college, my primary goal was as straightforward as it was unambitious: to get in. Obviously I had certain preferences, but like musical chairs I more or less didn’t want to be the one left standing. Most kids, I’m sure, are of the same mindset, which is why they labor through the often tedious application process for so-called “safety schools” that they would prefer not to attend. What differentiated me then wasn’t the fact that I applied to a safety school, but rather the fact that I applied to about a dozen of them.
Recognizing that my relative indifference towards academics and a lamentable lack of real athletic talent did little to distinguish me from my peers, I fell back on the shotgun approach. Apply to enough institutions, my thinking went, and somebody would have to accept me.
Fortunately for everyone – or at least me – Williams rolled the dice and everything worked out nicely. After a positively abysmal freshman year, anyway.
Perhaps because the tactic played a role in the fine education that I received, I find it curious that so few enterprises are willing to embrace it when entering markets with which they are unfamiliar. Many if not most of the vendors we speak with focus, understandably, on the science of community and software development. But while the science has its place, the fact is that the software world is, like the MLB Playoffs, a crapshoot. And success is often far more art than science.
Consider the evidence. Google, for example, built one of the largest software businesses in recent memory more or less under the noses of incumbents like IBM and Microsoft. Were the latter firms somehow stupid, or mismanaged? Hardly. Their condition is instead described as kind of myopia, as characterized by an occasionally slavish devotion to the science of the software business at the expense of the art. Google itself proves the point, as they themselves missed the scope of the social networking opportunity perceived by upstarts like Facebook and MySpace, which compelled them to adapt Cortés’ plan B. The lesson? Even smart, well run software organizations – which I would argue Google, IBM and Microsoft all are – can fail to identify massive market opportunities. There’s an art to it, after all.
The reasons for these misses are many and varied, if cogently summarized for you by Paul Graham here. The part that I’m interested in, however, is this tidbit:
Big companies also lose because they usually only build one of each thing. When you only have one Web browser, you can’t do anything really risky with it. If ten different startups design ten different Web browsers and you take the best, you’ll probably get something better.
The more general version of this problem is that there are too many new ideas for companies to explore them all. There might be 500 startups right now who think they’re making something Microsoft might buy. Even Microsoft probably couldn’t manage 500 development projects in-house.
That diversity and competition breeds more innovation than a lack of both is hardly a revelation. Well, at least if you’re among the now minority in this country that still believes in evolution. In a manner of speaking, evolution is nothing more than the shotgun approach writ large. Really, really large. Both are at their essence manifestations of the power of volume.
Paradoxically, however, evolution in the context of software is inherently misaligned with the science of the software business (yes, I do appreciate the irony of an argument claiming that evolution isn’t “science”). Adherents to the science of software will typically rely on a wealth of market statistics, focus group reports, and more to draw logical conclusions as to how to design, position and sell their product. At the opposite end of the spectrum are those who are cognizant of the limitations of the available data – the challenges of accurately measuring open source adoption for one, or issues with focus groups for another – and would prefer to rely on Darwinian competition to predict what they recognize they cannot.
Admittedly, true software evolution isn’t practical within the context of a single organization. As Graham acknowledges above, even an organization the size of Microsoft can’t manage hundreds of projects effectively. Intelligent software firms, however, have adopted a hybrid approach that borrows some of the best from both worlds. They use software science to manage their own product portfolios – a task for which it is usually well suited, but selectively apply software evolution via the acquisition of startups. To judge from the balance sheets, this dual pronged approach is relatively effective.
The question we continue to try and answer with our clients, however, is whether evolution or the shotgun approach have more of an internal role to play than has generally been assumed. While I’ve never had the theory confirmed for me, it has long been James‘ contention that the decision to open source Cloudscape – now better known as Derby – was just such an act of internal software Darwinism. Rather than try to decide via the science of software (which is all too often influenced by a third factor not to be discussed here, the politics of software) whether or not Cloudscape had a future, why not let evolution make the decision as to whether it will sink or swim?
In spite of such successes, however, science still dominates the business landscape within most of the software houses we speak with. Rather than try multiple markets and channels, looking to fail fast and often, vendors will often put the bulk of their eggs in a few, consensus driven baskets. Which is an approach, frankly, that it’s difficult to argue with given the results.
And yet the unanticipated success of high profile technologies like Linux, Apache, MySQL, PHP, and even Rails suggest that complementing software science with software evolution is likely to yield even more impressive results. By all means, employ science to focus and hone your investments when necessary, but don’t forget that evolution is where true innovation comes from.
Disclosure: Of the firms mentioned, IBM, Microsoft, and MySQL are RedMonk customers, while Google is not.
