There were so many possible angles of attack in covering the Amazon Web Services re:Invent 2016 conference that I seemingly never got to the point of delivery. RedMonk covered the event with a new recap format, borrowed from fivethirtyeight media, which you can see over here. Fintan has written a couple of posts – one on Elastic GPUs, FPGAs, Deep Learning & Compute and another on Hybrid, Greengrass and the Future of On Premises Hardware.
Greengrass is intriguing, because it illustrates AWS Lambda, a serverless or functions-as-a-service model as an on ramp for hybrid cloud/onprem computing. As with pretty much any new AWS service, adoption is disappointing seemingly right until the point it’s a crushing success. In the couple of months of last year it still seemed that AWS Lambda was struggling to find a place. By the time serverlessconf in May in Williamsburg NY had created a center of gravity to talk about serverless it was apparent things were changing.
And by re:Invent in December I basically didn’t talk to anyone that wasn’t using AWS Lambda to augment their other infrastructure. Many of them were frustrated that they hadn’t really had a chance to properly understand Lambda. We met super smart switched on people from startups, and they were like: “wait, AWS has that already?”, or “oh, we know AWS now has that feature, but we just haven’t re-architected to take advantage of it yet.”
Event-driven computing, with ephemeral managed functions, is a great model for efficient cloud computing. I have written before about how serverless is as much an economic phenomenon as a technical one. Please check out this great Trek10 post for more on that (I totally stole the lovely hand drawn chart above from them). Then check out this post by Fintan on serverless for DevOps.
One obvious issue with Functions as a service is manageability. If you split everything into functions that creates a significant management overhead. Joe Emison is a smart guy and he talks about this a lot. As with micro-services, where every system failure is like a game of Clue, Lambda function approaches create new overheads. Call it the Serverless Premium.
As I said in our roundup at the time:
“I definitely see X-Ray (visibility) being used alongside Lambda Step Functions (new technology for managing Lambda proliferation, with a state machine and visual tooling for service flows).”
By GCP Next a couple of months back I was intrigued by “breaking the glass“, how to maintain visibility and manageability of serverless functions and PaaS. It could be time to resurface message oriented middleware concepts as a way to think about managing apps based on independent functions and microservices. Guaranteed delivery, fan outs, publish and subscribe etc.
Fintan got to the heart of it in terms of AWS:
“then there is step functions with retries, very powerful.”
Step functions with retries, a world made of messages and active points. Think of Kafka, for example, a distributed commit log, used by Salesforce to bridge the gap between Force.com and Heroku services, and now offered as a managed service on Heroku. Kafka isn’t a message bus, but boundaries are fluid. Message buses will definitely be part of the new world. Note Pivotal is getting serious about RabbitMQ, a 10 year old project that deserves some love.
Serverless is going to be huge, but I suspect we’ll see AWS delivering some MoM style functionality to help manage functions and triggers, above and beyond the monitoring and API Gateway functionality so far. Especially as we move into multicloud serverless apps messaging is going to prove its worth. 2 phase commits aren’t even a dirty word any more.
Service Mesh approaches are related, with technologies such as linkerd fast gaining traction. Good timing for bouyant.io to release linkerd 1.0 this week.
The distinctions between categories keep getting more fluid, and serverless is only accelerating the fragmentation. It will be very interesting to see what themes emerge from serverlessconf in Austin this week.
AWS, Salesforce and Pivotal are clients.
Ant Stanley says:
April 26, 2017 at 9:45 pm
The interesting bit is that baked into every FaaS platform is a pub/sub service of some kind routing event payload data to the function that will process that data. If you look at the architecture for OpenWhisk or Kubeless from Bitnami, both have Kafka baked in providing reliable delivery of event payload data. User testing has found that AWS Lambda has a latency of between 20ms to 150ms to deliver payload data, this is due to an unseen pub/sub service routing event payloads to functions.
David Pollak says:
April 27, 2017 at 1:43 pm
Most apps are provisioned at the 95th percentile… so 94% of the time, there’s too much spend and 4% of the time, there’s a slow site
Serverless autoprovisions and makes much better use of resources
Internet of Things News of the Week, April 28 2017 | Stacey on IoT | Internet of Things news and analysis says:
April 28, 2017 at 1:39 pm
[…] This may be a new architecture for IoT: I’ve been wanting to write about functional programming, AWS Lambda and why serverless computing matters for the internet of things for quite some time. The idea is that instead of keeping an Amazon instance constantly running in case of a job, a service like Amazon’s Lambda spins up only when needed. Using a smidgen of compute to send an alert when a motion sensor is activated makes more sense in a severless architecture. For a nice overview of this read the blog by Redmonk’s James Governor. (RedMonk) […]
Internet of Things News of the Week, April 28 2017 | Internet Of Things | IoT says:
April 28, 2017 at 2:25 pm
[…] This may be a new architecture for IoT: I’ve been wanting to write about functional programming, AWS Lambda and why serverless computing matters for the internet of things for quite some time. The idea is that instead of keeping an Amazon instance constantly running in case of a job, a service like Amazon’s Lambda spins up only when needed. Using a smidgen of compute to send an alert when a motion sensor is activated makes more sense in a severless architecture. For a nice overview of this read the blog by Redmonk’s James Governor. (RedMonk) […]
Internet of Things News of the Week, April 28 2017 | Internet Of Things says:
April 28, 2017 at 3:26 pm
[…] This may be a new architecture for IoT: I’ve been wanting to write about functional programming, AWS Lambda and why serverless computing matters for the internet of things for quite some time. The idea is that instead of keeping an Amazon instance constantly running in case of a job, a service like Amazon’s Lambda spins up only when needed. Using a smidgen of compute to send an alert when a motion sensor is activated makes more sense in a severless architecture. For a nice overview of this read the blog by Redmonk’s James Governor. (RedMonk) […]
Internet of Things News of the Week, April 28 2017 - Padtronics says:
April 29, 2017 at 9:41 am
[…] This could also be a brand new structure for IoT: I’ve been wanting to put in writing about useful programming, AWS Lambda and why serverless computing issues for the web of issues for fairly a while. The thought is that as an alternative of holding an Amazon occasion continuously working in case of a job, a service like Amazon’s Lambda spins up solely when wanted. Using a smidgen of compute to ship an alert when a movement sensor is activated makes extra sense in a severless structure. For a pleasant overview of this learn the weblog by Redmonk’s James Governor. (RedMonk) […]
Bitnami comes up from below decks - Enterprise Irregulars says:
May 16, 2017 at 3:36 pm
[…] funcatron, and OpenWhisk. One point of technical interest for me given my recent post about serverless and messaging was the fact it uses Kafka for publish and subscribe. Functions are handled as custom resources, […]