AWS Lambda and the Spectrum of Compute

TL; DR – Serverless has grown up and found a home in the enterprise. But it is still an awkward teenager.

Perhaps the key message to come out of re:Invent 2017 was not one of any specific direction or guidance, rather that of a spectrum of compute, from bare metal all the way through to Serverless.

Depending on what your viewpoint is you may think that Kubernetes solves everything, Serverless is the one true way, monoliths will make a comeback or we need to go all the way back to bare metal. The reality is different – there are needs for all of the approaches depending on what business requirements you are solving for.

As it was rather succinctly put by a Pivotal customer during the analyst session at SpringOne Platform the following week

"30 years of M&A you get a lot of fun things in your environment" << the enterprise reality that the #cloudnative purists always miss

— Fintan Ryan (@fintanr) December 4, 2017

Amazon fully understand the reality of the compute spectrum, but they are also completely focused on making it easier and easier to begin new development projects on Lambda for a wide variety of scenarios. This makes perfect sense, as we noted previously Serverless is volume compute for a new generation of applications, with significant upside for the providers in usage of adjacent services, and also an efficient disruptor of established processes.

Additionally, by expanding the various entry points to the Serverless paradigm for developers, via routes such as AWS DeepLens, AWS Greengrass and so forth, Amazon are focusing minds on the end product required rather than solving for the underlying operational complexity. Obviously, there is a cost associated with handing the management of everything over to a third party, but as we have noted before packaging, convenience and agility will trump cost for profitable businesses.

It is also interesting to note the emergence of more formally recognised, and documented, design patterns for serverless, encompassing various types of workloads as well as a number of reference architectures.

Go and .net 2.0 Support

I will admit to being surprised at the level of demand on the Go front. While there is no doubt that Go is growing in popularity, it has only ranked as 15th in our last two programming language rankings, and it is very far from being a language that is a mainstay in the enterprise.

On saying that, when we asked for people’s reasons for wanting Go over any of the other choices available within Lambda two specific themes emerged – that it is a strongly typed language and error handling. The popularity of frameworks and tools such as Apex, Gordon and Sparta is also interesting to note, and may be a leading indicator of longer term trends. Apex is particularly interesting due to the involvement of Express creator TJ Holowaychuk. Many of the early adopters of node.js have moved to Go, but it is hard to see how Go will ever have the same level of traction as node has.

In spite of all of that, there is clearly an interest in having Go formally supported and Amazon have decided to accommodate it.

On a more speculative front, and this is very much in the realm of speculation, Amazon may also require Go support for their own internal work – and if that is the case their own volume would be more than sufficient enough to justify support. The continuing growth of Serverless within Amazon is something we will return later in this piece.

.net 2.0 support is an obvious addition for growing enterprise usage. C# continues to be one of the main languages in use in the enterprise, and continuing to provide an easy path into Lambda for C# developers is an eminently sensible thing to do. From Amazons perspective ensuring C# is a first-class citizen is very important in respect to competition with Microsoft.

Serverless Application Registry

The Serverless Application Registry is still in preview, but the idea is sound. As we noted previously the potential for a marketplace of Serverless apps is obvious. We will be watching the development of the Application Registry closely over the coming year.

Serverless Aurora

It is far too early to see what the impact of Serverless Aurora will be, like the app registry it is still in preview, but suffice to say we anticipate it will be very disruptive. My colleague, Stephen O’Grady, has written about the changes to the role of the DBA and on the mismatch in current practices between databases and development.

What do with data has been an issue for many doing Serverless development, and the reality is that developers really do not want to think about databases, particularly when choosing to go down the Serverless route. If Serverless Aurora delivers on its promise, and provides a configuration free experience with full integration into other AWS features such as Cloudtrail, IAM and so forth, we can expect to see a significant and rapid uptake.

Dogfooding Serverless at AWS

We at RedMonk are huge fans of companies who are dogfooding their own offerings. It shows a confidence in their own software which can only be reassuring to customers.

It was very striking to note just how much dogfooding of Serverless and associated technologies that Amazon are doing. Multiple product announcements were made at re:Invent which build on top of the serverless primitives that are provided.  Clearly Amazon are finding huge benefits internally with this approach.

Enterprise Usage

The most interesting aspect of Serverless is the rapid uptake in enterprise. Amazon have published case studies from customers such as Coca-Cola, Thomson Reuters, Expedia and Finra. Organisations like Travelex are talking very publicly about migrating applications to a Serverless architecture.

None of these use cases are trivial – they are at significant scale, and with real revenue attached to them. In our own conversations, we are hearing of more and more Serverless applications being developed, with the vast majority of the workloads going to AWS Lambda, followed by Azure Functions, with Google Cloud Functions running a distant third.

We have also noted an emerging trend, where enterprise developers are skipping over containers and associated paradigms completely and going straight to Serverless architectures for suitable applications. We expect this trend to pick up pace over the coming twelve to eighteen months, but with various teething problems as developers adopt to this new paradigm and reach decisions on what programming languages and frameworks they find to be productive.

Enterprise developers will, however, have to deal with commercial reality and their organisations existing investments. We expect to see more examples like the usage of SAP Hana with Lambda emerging as enterprises adapt to using Serverless approaches. One might say that it is an approach to using technology across a spectrum of compute.

re:Invent

We have published a number of pieces relating to re:Invent 2017. You may also like to read

• The Commodity Container Story – Fintan Ryan
• RedMonk Recap – The RedMonk re:Invent 2017 Recap – Stephen O’Grady, James Governor, Fintan Ryan
• What’s Scary About Amazon Web Services – Stephen O’Grady
• beginning to think this was the most important signal at AWS re:Invent 2017 – James Governor

Disclaimers: Amazon paid for my T&E to re:Invent. Amazon, SAP, Microsoft, Pivotal and Google are current RedMonk clients.