The Private Cloud: Opening the Door for Open Source

“In recent years, the term ‘cloud computing’ has emerged to make reference to the idea that from the standpoint of a device, say a laptop, on the Internet, many of the applications appear to be operating somewhere in the network ‘cloud.’ Google, Amazon, Microsoft and others, as well as enterprise operators, are constructing these cloud computing centers. Generally, each cloud knows only about itself and is unaware of the existence of other cloud computing facilities. In some ways, cloud computing is like the networks of the 1960s when my colleagues and I began to think about connecting computers together on networks. Each network was typically proprietary. IBM had Systems Network Architecture; Digital Equipment Corporation had its DECNET; Hewlett-Packard had its Distributed System. These networks were specific to each manufacturer and did not interconnect nor even have a way to express the idea of connecting to another network. The Internet was the solution that Robert Kahn and I developed to allow all such networks to be interconnected in a uniform way.

Cloud computing is at the same stage.” – Vinton Cerf, “Cloud Computing and the Internet”

In other words, the more things change, the more they stay the same. Which, in this case, is a more charitable way of quoting George Santayana, who said “those who cannot remember the past are condemned to repeat it.”

Today’s clouds, more or less without exception, are defined by their abstractions. Amazon abstracts storage and a few other layers. Google and Salesforce, pretty much everything: operating system, database, hardware, you name it. And so on. Whether we’re talking about Infrastructure as a Service or Platform as a Service, the contrast versus the typical platform of yesteryear, where the pieces were hand selected from the hardware on up, is stark and reeks of compromise. Very much like Apple on the consumer level, Google et al demand sacrifices in return for convenience. Perhaps – or make that likely – realizing that businesses will invariably sacrifice the future at the altar of the present. We’ll give you the convenience and time to market now; just don’t expect to leave later.

And it’s hard to blame them for that, honestly. They’ve got jobs to do and kids to feed, and their blind trust in the technology industry to police itself and not lock them in this time as they’ve been locked in so many times before is as Peanuts touching as it is naive. Whether Lucy will yank the football out from under them yet again depends, as far as I can tell, on open source.

Talk to any large enterprise, or those that do business with large enterprises, and you’ll invariably hear about the quote unquote private cloud. Because, of course, they would prefer not to be Charlie Brown to a given public cloud supplier’s Lucy, as Charlie Brown’s tend to get fired when they miss so completely. The theoretical advantage of the private cloud is simple: all the benefits of the public cloud, from fast provisioning to elastic scaling, without the troubling dependencies on an external supplier(s). You and I might poke holes in that, of course – enterprises are already dependent on external IT suppliers, most enterprises can’t scale hardware as well as cloud providers, the private cloud model prohibits the realization of cap-ex and op-ex advantages gained by elastically adding and subtracting hardware, and so on – but none other than the voice of the CIO, Gartner, has been saying that the future of IT is private clouds. Which goes a bit far for my taste, but is not terribly different than what we hear.

These enterprises will no doubt have options with their existing suppliers for replicating cloud-like environments inside their firewalls. VMware, clearly, would like to be the provider of choice. Microsoft will say Azure. IBM will start with CloudBurst. HP, with Assure. And, well, you probably get the point: there will be no shortage of commercial choices for private cloud platforms.

But would these be more compelling than an open source alternative, the LAMP of the clouds, if you will? For a certain class of customer – the one who writes blank checks to a large systems supplier to make its IT problems someone else’s problems – absolutely. But for those that were price senstive, lock-in sensitive, simply wished for broader options of blending private cloud resources with public resources or all of the above, it’s an interesting question.

Consider that the current crop of cloud providers have thus far been very reluctant to open source their respective layers of abstraction, as is their right. And that even if they were to change their mind, the software might not be particularly useful or accessible outside the context of their highly unique datacenters. Then recall that the majority of smaller and medium sized web hosts – the most plausible public cloud alternatives to the big folks – have typically shied away from proprietary platforms (i.e. VMware, or in the past, Java) in favor of readily available open source options. There’s a reason so many hosts run CentOS, Debian, Fedora, or Ubuntu.

All of which is to explain why I think projects like Eucalyptus – who I was fortunate enough to sit down and talk to at OSCON last week – and Hadoop are so important. Open source cloud stacks, after all, would theoretically allow private enterprises to run on top of the exact same stack as alternate public providers. Much as Linux disrupted and commoditized the platform that applications run on, thereby improving competition in the market, so too could open source cloud projects ensure an openness to the platform that host operating systems and attendant cloud fabrics.

Contrary to public opinion, I do not think open source is irrelevant in the cloud; I merely think it will not, by itself, unseat the current or previous generations of technical giants. And while it won’t unseat the Amazon’s and Google’s of the world, open source could compel them to be more open than they would in a market absent such pressure.

Whether open source takes a role front and center, then, remains to be seen, but is certain that it will – as it has to date – have a crucial role in shaping the cloud market to come.