The coming SMOKEstack: rethinking and retooling “multi-cloud”

Over the past 10 years or so as AWS has continued to grow and dominate new markets we’ve seen a couple of key narratives emerge that might be seen as either realism, or perhaps just an attempt to slow Amazon down. One is hybrid-cloud, and the other is multi-cloud.

Consultants and folks invested in legacy technology stacks find hybrid cloud appealing for obvious reasons. Hybrid cloud means enterprises doing much what they do now, but with a promise of Cloud portability for data and application workloads. Keep doing what you’re doing, but with some public cloud now and more in future. Hybrid cloud is a bit of a punt. One reason Kubernetes is so popular is that it can be made to fit this narrative: you can be cloud native, and on prem, a dessert and a floorwax. With all their legitimate, significant, investments in Kubernetes, meeting genuine customer demand, VMware Tanzu and Red Hat Open Shift are best placed to play the hybrid cloud game.

But what of multi-cloud? One way of looking at multi-cloud is with a similar view of portability. Find a technology that will allow us to build applications to deploy to any public cloud and go from there. Investment protection, transferable skills, write once run anywhere- what’s not to like? Again Kubernetes fits the bill nicely here. Learn and invest in Kubernetes and then an organization should be able to write applications and deploy them to any cloud they choose. Amazon EKS, Google Cloud GKS, Azure AKS. Hey look folks *KS – they’re all the same, right? Of course the devil is always in the detail, in manageability, in slight points of friction and different approaches. Back in the day Unix made similar promises. While the public cloud Kubernetes services share some similarities, there are also plenty of differences – consider for example AWS Fargate – a managed “serverless” Kubernetes platform. Manageability is always a great point of lock in over time. All that said, portability is always a matter of degree, and certainly Kubernetes offers some comfort factors for customers that portability is there if they need it. The Cloud Native Computing Foundation is a risk management organization, and enterprises and cloud companies alike continue to push Cloud vendors to support Kubernetes and related CNCF stack technologies.

While managed services is one form of lock in another powerful lever is egress charges. Once you’re running on AWS you’re not going anywhere else because egress costs are eye-wateringly high. And why would you want to get off AWS? It’s a powerful platform, one of the most incredible engineering organisations in history, with an absurdly rapid cadence of new functionality in all of the most important areas. It may not be be the slickest experiences, but the primitives are incredibly powerful and the documentation does the job.

There is another view of mult-cloud however, which the industry hasn’t really given a name yet. In this view of the world organisations will use different clouds for different jobs. Different parts of the same organisation are going to make different choices. Mergers and acquisition activity are almost certainly going to render any single vendor cloud strategy moot over a reasonable time frame. And different clouds are frankly really good at different things. AWS primitives are excellent, but if you want to bet on a platform for its data services the natural choice is GCP. Spanner, BigTable, BigQuery, PubSub, is a powerful, well integrated platform from Google. Azure continues to make great strides in key areas, and Microsoft is beginning to make the GitHub advantage count. GitHub will become a dominant player in DevOps and CI/CD over time, and Microsoft’s focus on Developer Experience is unmatched by its competitors. Visual Studio Code is everyone’s favourite editor. CodeSpaces is going to make everything easier. Azure has some great IoT tools. Every cloud platform has its strengths.These strengths will be leveraged. Any major organisation is going to be using at least two of the major cloud platforms, but not for portability reasons, just because that’s the nature of tools choices.

And then there are the services. Oh all the lovely highly productive services. If you like the public cloud services just wait til you meet best of breed APIs. Oh my goodness Stripe for payment is just the thing- the docs are dreamy and the APIs are lovely too. Algolia for text search. Auth0 does cross platform authentication management so well, why not use it across your app portfolio? Oh you have image and video assets to manage – Cloudinary jumps out as a highly productive service choice. You want your app to have messaging – Twilio’s got you. Oh managed content management yes you want Contentful for that. Managed customer data with an API and you’re not looking to use Salesforce? See Segment, it’s on point.

• Less Code
• Less Complexity
• Less Reinvention
• High Productivity
• High Scale
• Rich Service Ecosystem
• More Self-Service

Here’s the thing, we’re moving up the stack, to highly productive developer friendly API-driven services. A composition based development model is just never going to be all-in on a single cloud. Have you ever even met software developers?

But what are we going to call this thing? One answer, which comes from a particular perspective is (Javascript, APIs and Markup) JAMstack – which took the ideas of the Static Site Generator generation and added interactive services. APIs and Markup sure, but focusing on Javascript alone is not going to fly for all developers. Have you ever even met software developers?

Read more about Static Site Generators here and emerging deployment models here.

The serverless community has an answer for the best of breed services approach – they call it “serviceful”. The term was coined by Patrick Debois at the first Serverlessconf, held in a sweltering hot warehouse in Brooklyn in 2016. Serviceful has gained some popularity with the serverless folks for sure, but hasn’t crossed over into other tech communities. Special thanks to Bryan LeRoux, Nader Dabit and Joe Emison for pushing me forward here)

In researching the serviceful theme I came across a really interesting suggestion by Mark Hinkle for the new world of composable higher level services – SMOKEstack (Serviceful, Mashable, Open, K(C)composable, Event-driven.). The term may not be perfect, but it’s the best I have come across to describe the possibilities. The explicit calling out of event-driven models is exactly right for the new world of composable services, that touch not only public cloud apps, but also serviceful services and Enterprise SaaS and even on-prem applications. It’s not surprising that Mark Hinkle, who introduced the term to me, is a co-founder of TriggerMesh, a company focusing on these kinds of cross-platform cross cloud event-driven integration scenarios. I wrote a bit about the firm here.

We will need some new underlying technology to underpin the multi-cloud best of breed world. Data integration is a good example. In fact I came across the SMOKEstack term while working on a talk for HasuraCon 2020 this week. Hasura, which has done a great job of making GraphQL more accessible for companies looking to make their data more accessible to modern developers is increasingly positioning itself as a middle-tier for federated serviceful data-driven apps, which could be consumed by JAMstack or SMOKEstack developers.There is plenty of work to be done in delivering SMOKEstack, for example:

• Integrated operating model
• Integrated development model
• Egress costs that won’t kill you
• Persistent Storage
• Deployment flexibility
• Composite Cloud management Layer
• Cross-cloud integration
• Data Federation

I am not sure “SMOKEstack” is perfect. It certainly isn’t the most obvious terminology from a sustainability perspective. A better term may well emerge. But for now, I will use it in thinking about serviceful applications with persistence. It’s kind of catchy. Let me know what you think, of if there is a better industry term that I am missing.

disclosure statement: while this is not commissioned research, Amazon Web Services, Microsoft Azure, Google Cloud Platform, Hasura, Red Hat and VMware are all RedMonk clients,