It’s the Data, Stupid. What’s next in Cloud and Apps? IBM, SoftLayer, Joyent, Riak, 10Gen

IBM is acquiring SoftLayer, the hosting and cloud company, for a couple of billion dollars, according to Forbes.

IBM positioned the deal in its announcement call and press release as accelerating efforts in Enterprise Cloud services, which seems reasonable, given the need to drastically accelerate its SmartCloud play to reach critical mass in a market dominated by Amazon. But the pitch from IBM missed something extremely important – namely, that SoftLayer brings IBM an entirely new customer base. SoftLayer was becoming a darling of Web startups for its uncomplicated functionality, not for its enterprise class complexity.
In web development simple wins every time. Amazon wins with simple. IaaS wins over PaaS. “Toy” wins over Enterprise. CAP wins over RAS. Simple means portable.
This is the first time since the late 1990s, the days of the 4 Horsemen of Web 1.0 (Cisco, EMC, Oracle, Sun) with IBM and BEA fighting red and claw to win clients such as eBay, that IBM has been positioned to take a run at startup culture and growth based on public internet infrastructure.

I have been hearing growing chatter from web developers about Softlayer for some time – basically because it’s halfway between IaaS and managed hosting. At first glance it doesn’t have the bells and whistles Amazon AWS has, but it does offer dedicated machines and local storage, increasingly attractive to companies building data-oriented applications. Buy dedicated storage from Amazon and costs swiftly add up. SoftLayer’s play is to, ironically enough, win by offering less, not more, virtualisation.
Of course SoftLayer also sells to enterprises, or IBM wouldn’t be acquiring the firm.

As a I heard from Jared Rossof at 10Gen last year:

“I have one customer running 1200 databases [on SoftLayer]. Most is private, but what isn’t private is on SoftLayer.”

We’re at a really interesting inflection point and IBM’s timing is near impeccable. IBM missed the Compute wave of public cloud, but is now potentially very well set up for the next wave, where we start to see a world defined by key value stores and mass scale distributed file systems, rather than relational databases. NoSQL has far longer legs than its generally given credit for. Amazon is a storage engine, and its erstwhile open source competitor OpenStack is immature in that regard.

So we’re seeing rapid adoption of stuff like RiakCS, a clone of the Amazon storage APIs by hosting players such as SoftLayer itself, but also Tier 3 and Joyent. Joyent also now has its own Manta play, designed to underpin new data-driven workloads. Also see Mongo at Rackspace via ObjectRocket. Check out Inktank (the big financial services firms are).

We’re standing at a junction, where Enterprise wants to look just like the Web when it comes to information management. The NSA is hiring and borrowing from Facebook. When Goldman Sachs calls in storage players its first question is:

“How can you help us do storage like Facebook?”

Hadoop, invented at Yahoo, is blowing up as one of most quickly adopted technologies ever used in the enterprise. Technologies being built by LinkedIn and Twitter are going to define the real time data space (See Kafka and Storm).

Storage is blowing up. You can see why EMC is frantically reorganising everything. Why Pivotal, etc. Why Oracle is suddenly amenable to a deal with Salesforce.

So IBM bought the right company at the right time. But it needs to make the transformation. I remember when IBM acquired Tivoli Systems in 1996. Talking to Frank Moss, Tivoli founder, he commented wryly on the deal:

“Oh IBM didn’t acquire us. It was a reverse takeover”.

Moss was in many respects close to the mark. The deal really was transformational. IBM suddenly had to sell distributed software to people that didn’t give a shit about mainframes. IBM had to start supporting Solaris. The first big battle Tivoli engineering had to fight was with IBM accountants, trying to tell them to use AIX instead.

Its time for another reverse takeover. IBM needs to go easy on the Bluewash this time. It’s not about processing, it’s about storage. It’s not about infinite SKUs but having the right opinion, implemented via automation.

disclosure: Amazon, IBM, Joyent, Basho are all clients.