Skip to content

Salesforce and the New Bucket of Bits. Phoenix, Ashes.


[second update – the strategy and product below was seemingly soft launched at Dreamforce this year, under the name BigObject.]

[update – on reflection i should have started with a use case. Without the ability to store and query billions, then trillions, of events Salesforce will not be able to deliver on its Internet of Things vision, allowing for predictive maintenance, new customer touch points, wearables for healthcare and so on. Salesforce was well set for the age of the system of record, but it needs a new architecture to support systems of engagement.]

When and Oracle announced a renewal of their vows last year I was kind of surprised. Why would Salesforce deepen its reliance on the Oracle database at a time when others across the industry were lessening their reliance on that venerable and not inexpensive platform.

To be fair Oracle database has excellent high availability features, and decades of query optimisation built in. It’s a rock solid database for traditional relational workloads.

But pretty much every business has a data challenge that can’t be easily met with traditional database licensing driving the costs up – for example call data records in telecoms. Oracle’s problem, and that of its customers, isn’t scale so much as cost and inflexibility. What is a system of record if you have to throw away most of your logs in order to keep costs down? I am using Oracle as short hand here- IBM DB2 and even the once low cost alternative Microsoft SQLServer were all designed for an era where software licenses were king. But this is a different era – whether or not you love or hate the term Web Scale, it captures something of the change.

Open source data stores are opening new opportunities for businesses to solve those data management problems they had previously parked, and creating entirely new businesses. Even the Wall Street Journal is writing about data as currency these days. We’re seeing enterprise sales across the board from open source platforms like Amazon Red Shift and SimpleDB, Cloudera, Couchbase, Hortonworks, MapR, MongoDB, Datastax, Data Bricks.

So if everyone else is zigging, why not Salesforce, given its competitive relationship with Oracle? Every customer dollar spent on Oracle is a dollar less on R&D or customer facing operations. Given that Salesforce has a really solid Postgres competence at its Heroku subsidiary that seemed a natural place to invest.

So what would a system of record for every event look like, without requiring a pre specified relational style data model? Increasingly the answer to that question is found in the wider Hadoop ecosystem. The Big Bucket of Bits is Hadoop Distributed File System (HDFS), with a number of technologies designed to take advantage of that data pool. See for example Spark, the new hotness is reading and processing data from Hadoop data sources. Pivotal meanwhile is putting its weight behind Tachyon for “data lakes”.

So let’s get back to Salesforce then, shall we?

Today is a system of record, somewhat constrained by the cost of relational database. Going forward however, Salesforce will increasingly offload storage to a Hadoop store, with SQL query support based on Apache Phoenix, a layer on top of Apache HBase for data that is not based its traditional business object model. A good example is a logging service for compliance, which will take advantage of Phoenix (Salesforce are core committers and project leads). In other words Oh Hai Apache.

Logging and compliance are not an accidental use case- Splunk has created a billion dollar company based on this idea, and needless to say it doesn’t run on Oracle.

Salesforce is not going to replace Oracle at the core any time soon, but it is going to use Hbase and Phoenix at the edges as a pluggable architecture to offer customers. The blob storage will make particular sense for read only data.

Salesforce needs a Big Data play for its customers and ISVs, or they’ll simply go elsewhere, particularly given the prevalence of, and innovation in, data stores, and the ability to spin up IaaS to run them so easily.

It will be interesting to see if the new architecture gets a mention today when Salesforce rolls out its new analytics architecture, codenamed Wave.

Replacing Oracle is hard, but augmenting is not. We can expect to see similar patterns in the enterprise. Just as companies still run mainframes today, Oracle database isn’t going away any time soon. But its period of outright dominance, as the status quo, is now over.

it isn’t saying so explicitly, but for Salesforce Oracle is a legacy technology. Managed decline is the order of the day. As with the mainframe, Oracle capacity will grow, but distributed data is going elsewhere.

Categories: Uncategorized.

Gosling, Canter, Hoodie etc – awesome speakers for IOT at SCALE

The speaker lineup for IOT at SCALE, our conference this week with SAP in Palo Alto is pretty crazy.

James Gosling, the guy behind Oak, the original IoT vision that led to Java will be talking about Liquid Robotics, an amazing environmental monitoring startup.

Marc Canter is CEO of Interface, an authoring tool for IoT. He was one of the founders of MacroMind, which became Macromedia. You might not like Flash, but it was an incredible revolution when it emerged.

Peter Hoodie of Marvell, will be talking up his Kinoma Javascript framework for IoT devices.

Rick Bullotta made a successful exit selling Thingworx, an enterprise integration platform, to PTC. Smart and super opinionated, Bullotta knows developers and business.

Sarah Cooper of M2Mi, developed the first biomedical battery powered by body temperature. Need we say more? She also has a great non-FUD based approach to helping businesses understand IoT opportunities.

We also have a focus on Small Data for IoT, with Raghuram Sudhaakar talking about signal and noise and endpoints in context of the Krikkit Eclipse project, while Abdulkim Daneche from MapR gives us the Big Data story.

All of the speakers are really solid- check out the site for more details. But seriously- Gosling.


Categories: Uncategorized.

IOT at Scale: Disruption, Planes, Cranes, Automobiles

Another teaser for the IOT AT SCALE event next week. I like the way this one came out. You should sign up here.

Categories: Uncategorized.

IoT@Scale: a RedMonk event with SAP

I made this video preview for IoT@Scale is an event we’re running with SAP, in Palo Alto on October 16/17th. We want to bring developers to bear on Industrial scale IoT issues. It’s a chance to enjoy the RedMonk experience on the West Coast. Let’s just say there will be great talks, company and of course tremendous craft beers. Day one will be hacks, day two talks.

As my esteemed colleague Juliane Leary puts it:

“During IoT at Scale, we will get under the skin of the emerging IoT disruption. We are celebrating Web, Developer and Maker culture as their influence grows in the enterprise, delivering better apps and services to users. This is not an event about stuff that won’t ever make it into production, but rather a look at future production systems.

Web scale is going to look small in comparison to IoT scale for automation, analytics, data management, and at every layer of the stack. We’re going to bridge high scale architectures and infrastructures with the kind of business issues that face SAP customers and prospects every day.

Talks will be technical in nature, aimed at developers, architects and digital innovators.”

You should sign up here.

Categories: Uncategorized.

How To Get Girls Into Coding: quite simply the best tweet ever

I am really looking forward to seeing Soledad Penadés, the author of this fantastic tweet, talk at One Shot London this Saturday (some tickets still available here).

“Just turn it into a node module,” and other mantras Edna taught me.

The story of leaving behind a random mix of Python + php + bash + makefile + Scons scripts to totally embrace using Node, modules, standard callbacks, browserify, and friends to build toys that bleep and bloop with MIDI, WebGL and Web Audio.

Categories: Uncategorized.

Integrate All the Things. How Web and Open Source Culture are Eating The World

Integrate All The Things WS02Con from James Governor

A few weeks back I travelled to Barcelona to give a keynote talk at WSO2Conf, a user conference focused on integration middleware. The theme of my talk was that Web and Open Source culture are changing the business of IT, and thus the business of business, as disruption increases in a wide range of markets. With disruption comes fragmentation and the need for new development and integration approaches. Essentially the RedMonk stump pitch, as developers and engineers become increasingly important. But I still have NO IDEA where “Paul Andreesen” came from (watch the talk). See Mark Andreesen’s Software is Eating the World oped here.

WS02 is notable because of how well it represents many of the trends I spoke to in the keynote – a company founded in Sri Lanka, with a development presence in both the UK and the US, successfully develops and packages open source software for customers around the world. Earlier this week I blogged about the ongoing importance of the Apache Software Foundation, and WSO2 is all about offering services and support around Apache stacks. I have written before about how the ASF is contributing to the Sri Lanka economy.

WSO2 offers a full stack of integration middleware, integration bus, business process management and monitoring, registry, ID management etc, based on Apache tools. More recently it’s begun to focus on the emergent Platform as a Service (PaaS) and Internet of Things markets. One area that isn’t fully baked, but is looking very interesting, is the company’s drive to integrate complex event processing as a front end filter for internet of things data streams, integrating its own CEP platform with real time data tooling including Apache Kafka, with batch storage provided by Hadoop. I thought of the approach when I saw this tweet this morning.

Anyway – WSO2 usually moves pretty fast so I look forward to seeing the results of its Big Data work. The company is a client and paid for my T&E to the event. If you’d like to see my keynote, here it is below.

Categories: Uncategorized.

Bluemix, CloudFoundry PaaS and Twilio: It’s all about the APIs. A Workshop next week

Ever since Pulse, when Twilio CEO Jeff Lawson stood up at an IBM event and livecoded in Node.js, deploying to IBM’s Cloudfoundry implementation, called BlueMix, its been pretty clear that popular Web APIs was a great way to make the case for the virtues of platform as a service. In case you don’t know Twilio, it finally cracked the code on turning mobile telecoms services like SMS into easy to code APIs.

Anyway, Shoreditch Works, the event space and coworking business I founded, signed a sponsorship deal with IBM, and we’ve been running a series of events with the company. Next week is a training day and meetup. It should be a great way to learn more about CloudFoundry from the UI or command line perspective, and I am really pleased that IBM persuaded Twilio, now a Shoreditch stalwart to get involved – and I know their will be some live Ruby coding.

So whether you’re building modern Java or Node apps, or Ruby, CloudFoundry is an environment you consider. Perhaps most importantly it is portable – a number of major vendors are going to support CloudFoundry deployment, whether that be GoPivotal itself, which originally built the PaaS, or HP, for example. There may be “cooler” options out there, such as Flynn, but they don’t have the enterprise support.

CloudFoundry is very easy to deploy Web apps too. Indeed- Shoreditch Works now even run a WordPress instance on it.

For IBM customers and partners, it’s a no brainer to consider it. So you should come along next week for this BlueMix day to find out more. Sign up here.

IBM is a customer, HP is a customer, GoPivotal has been a customer.

Categories: Uncategorized.

Apache Software Considered Useful

As the Apache Software Foundation gears up to celebrate 15 years of operations I just wanted to take the time to point out that Java’s renaissance has been underpinned and underwritten by the organisation, playing a huge part in the new and emerging Big Data ecosystems bringing together Enterprise and Web companies.

As a time when Oracle’s stewardship of Java was increasingly challenging, the ASF was a safe harbor for people building cool stuff in Java. Yes that’s right – cool stuff in Java.

Hadoop may now be too successful to be cool exactly, but Apache Giraph is (Facebook is using Giraph alongside Hadoop now). Or how about the real time stuff, contributions from LinkedIn and Twitter – Kafka and Storm?

Or the new coolness from Twitter, AirBnB, Mesosphere etc- Apache Mesos (an OS for the cloud, etc, turning machines into One Big Machine, doing for compute what Hadoop did for batch counting). Meanwhile Google’s Kubernetes platform, which is designed to fulfil similar goals, was also recently open sourced under an Apache license.

ASF is now the go to governance organisation for West Coast code, and the Web Companies there are building a ton of interesting stuff. Apache may seem boring and consensual but those are often virtues when creating standards.

So while it was trendy for a while to decry Java and the ASF, clearly they have plenty of runway ahead of them.

If you think Java is dead, then clearly you haven’t been paying attention.

Here is to another 15 years.

associated links – Happy Birthday Apache (10 years in)

disclosure: the ASF is currently not a client, but we’ve worked with the organisation for many years, and been advocates most of that time.

Categories: Uncategorized.

Red Hat’s entirely rational position on OpenStack

So recently some folks in and around the OpenStack ecosystem got angry when Red Hat said it had no plans to support other vendors’ OpenStack implementations.

OpenStack, in case you hadn’t heard, is an open source set of building blocks for cloud infrastructures – compute, storage etc – which maps roughly to Amazon Web Services (AWS) proprietary stack. Initially founded by NASA and Rackspace, the OpenStack community now includes pretty much every vendor in enterprise IT.

The argument seemed to be that Red Hat as an open source company, should support everyone else’s stacks too.

Ben Kepes, in his post on the story, argued that Red Hat Plays Dirty To Lock Its Customers In – So Much For Caring-Sharing Open Source.

Open source is caring and sharing? Sometimes – but it can also be nasty, brutish and short. Red Hat wasn’t founded on altruism- it was founded on pragmatism, doing the best job of packaging Linux for the enterprise, making it certifiable and above all trusted as a deployment platform for enterprise apps. As such Red Hat supports a range of third party apps – Oracle, SAP etc – that run on Linux, but doesn’t support other Linux distributions per se. Latterly Red Hat has changed positioning somewhat, responding to the rising importance of developers and bottom-up adoption with its embrace of CentOS.

i have written before about OpenStack’s community before: it’s so big and inclusive that it is unwieldy. Different components are evolving at different speeds, and forking is an issue. That combinatorials at Open Stack are hard to manage, and therefore even harder to support.

This statement from Red Hat seems reasonable.

100% of Red Hat’s offerings are open source, meaning users can deploy and run them anywhere they choose. To meet our customers’ stringent mission-critical requirements in a cost-effective manner, and balance the nearly infinite combination of operating systems, hypervisors, cloud platforms, Red Hat fully certifies and supports many specific Red Hat Enterprise Linux footprints on its own and other vendors’ platforms. Where customers have deployed third-party software, drivers and/or uncertified hardware/hypervisors, the longstanding practice of our Global Support Services team has been to work with customers to diagnose the root cause, and when that root cause is the result of an unsupported hardware or software component, we help the customer connect to the provider for support and get back to a working state. Red Hat has sophisticated, proven mechanisms for certifying supportable hardware and software, and we encourage customers to work within supported configurations for optimum customer experience. This is a big part of the subscription value that customers pay Red Hat for, along with automatically distributing patches to Red Hat Enterprise Linux customers. This provides the highest level of assurance that patches work, are secure, and don’t create unwanted side effects. That policy is noted here:

So Red Hat’s move is entirely rational, but it does have significant implications. IBM and HP for example, which both walked Red Hat into their enterprise customers over the years, are both deeply unhappy, giving a shot in the arm to enterprise Ubuntu (which note, now runs on IBM POWER). Mirantis and Canonical have been playing the open card.

Other vendors have been emboldened by Red Hat’s move, but customers are the ones that will decide. It’s not so long ago that Microsoft didn’t support VMware as a production environment. But if customers anoint another OpenStack distro chances are high Red Hat will change its mind.

oh hai disclosure: Canonical, HP, IBM, and Red Hat, are all clients.

Categories: Uncategorized.

Innovating the Wimbledon Fan experience, Bluemix, and a day pass to the AELTC.


In case you missed it IBM is sponsoring the Village Hall, to help me support local communities and startups. Under the terms of the deal they get to host a number of events at the space, and one of them coming up June 11th next week promises to be fun.

IBM has managed the All England Lawn Tennis Club’s IT infrastructure for 25 years, and I am looking forward to hearing from IBM Distinguished Engineer Bill Jinks about the scale challenges, serving mobile and other channels, and all the cool analytics stuff involved. We should have some cool old photos and other good stuff, and some appropriate drinks and food. Strawberries and cream will be part of the mix.

But beyond tennis this is also the first London meetup for IBM’s BlueMix Platform as a Service, which is based on CloudFoundry, the open source platform being driven by GoPivotal. There will be some live coding demos but nothing canned. Not even the beer.

So if you’re interested in next generation IT infrastructure or a behind the scenes look at Wimbledon’s architecture you’re going to enjoy it.

I also have some ground passes to Wimbledon (though sadly not center court tickets), which attendees will be in with a chance of winning. You should sign up to event here.

Categories: Uncategorized.