Client side compute and edge compute are converging. Web apps are transitioning where data is processed closer to the user—a move inclusive of not only compute on the device using client side rendering engines like V8, Google’s JavaScript and WebAssembly engine, but also the so-called “edge.” As Stephen O’Grady has so eloquently explained, despite “dozens or perhaps hundreds of technology vendors at present hav[ing] substantial open investments in marketing edge branded products,” the term “edge” is rife with ambiguity. However, beyond exasperating critics if “edge” accomplishes nothing else, it does the important work of signifying servers that are geographically closer to the user than cloud data centers historically have been.
This trend, which AI workloads will only serve to amplify, is moving serverless compute closer to the user for performance reasons—a transition which has repercussions for packaging, runtimes, databases, and abstractions in general. Serverless compute (an idea that is itself fraught with ambiguity, as Rachel Stephens notes) is where all this convergence around the location of data processing, which has migrated away from local data centers (colocation, rackspace), and toward the cloud and the devices themselves (browsers, IoT), whether you want to use the term “edge” or not.
The future of compute will not occur at remote data centers, but will instead take place locally and close to the user. Cloud providers have risen to the challenge of creating a global network of data centers to ensure the workloads they manage are processed close by its customers. US-EAST-1, AWS’s first region and “problem child,” which David Brown, VP for compute and networking services at AWS, recently defended, is far from the only or even the default option for AWS cloud customers who can now take advantage of local zones. Cloudflare’s homepage boasts that 95% of the world’s population is within 50 milliseconds of a Cloudflare Worker, with 80% of them being within 20 milliseconds. This shift goes beyond CDN, a technology that emerged in the 90s to transmit cached data using a distributed network of servers (a history). Vendors including Akamai, Cloudflare, edge.io, Edgio, Fastly, and others realized they could not run functions on CDN, and needed an alternative solution. While strategies and product offerings vary among these players, the race is on to ensure compute occurs as close to the user as possible because today’s workloads increasingly process real time, streaming data. This shift matters because streaming data is finicky. It requires support architectures intended to reduce the heaviness and latency costs associated with moving data into and out of a database, making edge computing, in the sense of data processing occurring at a relative position near the user, absolutely essential.
Frontend Workloads
The shift in where compute happens makes frontend and mobile developers, who work almost exclusively on client side code, heralds of a significant transition in the cloud computing space. In fact, by harnessing the wave of powerful emerging JavaScript tools and technologies that have appeared on the market in recent years, both frontend engineers and the vendors that recognize them as Kingmakers, have positioned themselves as innovators at the tip of the spear.
So, @vercel reverted all edge rendering back to Node.js 😬
Wanted to correct the record here as it's something I've advocated for in the past, and share what I've learned since then.
Also, the "edge" naming has been a bit confusing, so let's clear that up here as well.
What is…
— Lee Robinson (@leeerob) April 17, 2024
The shift toward compute near the user explains much about the brouhaha that erupted following a tweet from Lee Robinson, the VP of Product at Vercel, the self-described “frontend cloud.” In it, Lee not only articulates his “learnings” about edge as a technology, but he also announces Vercel’s decision to revert “all edge rendering back to Node.js.” Evidently, Vercel has discovered that they can’t do server side rendering (SSR), and instead must move their functions to partial pre-rendering (PPR) with Node. This realization likely has much to do with latency and cost added from having Vercel’s AWS infra speak to CloudFlare, as Vercel’s Edge Functions run CloudFlare Workers, which adds egress costs owing to CloudFlare’s Bandwidth Alliance. Vercel’s adoption of Node also suggests an interesting story around V8, as both Node and Cloudflare run V8. But beyond bypassing AWS-Cloudflare friction, and ensconcing technical and partner alignment with Google—all subjects worthy of standalone blog posts—, what interests me about Lee’s post and its reception is what it tells us about the state of serverless.
Where compute is running is deeply significant to frontend engineering teams. Ashley Williams, former Staff Product Manager at Vercel and Systems Engineer at Cloudflare, has noted the importance of “JavaScript’s Journey to the Edge.” She suggests that frontend engineers, even when they could care less about IT networking, benefit from the transition to close compute. Significantly, in most cases frontend workloads are lightweight enough not to mandate the security and latency costs associated with running compute in a data center. While many browser based JavaScript applications are currently needlessly heavy, this is beginning to improve as single page applications (SPAs) move to adopt more HTML-first architectures. For instance, “component islands,” a pattern coined in 2019 by Katie Sylor-Miller, a frontend architect at Etsy, allow for portions of the page to render as static HTML delivered from the server, while interactive portions of the page are delivered as self-contained client side JavaScript. All of these innovations in the space suggest that frontend developers are well aware that the KISS principle applies to serverless. In point of fact, this has long been the argument made by Sunil Pai, founder of the open source real time platform PartyKit and Principal Systems Engineer at Cloudflare. Pai explains in a post on his personal blog:
That’s the kicker: the “next” evolution of serverless isn’t really something “new”, but rather a return to how we would write code if we were running on a single machine. A return to simplicity, if you will. And while traditional serverless (lol) is great for stateless workloads, stateful serverless unlocks using a similar programming model for a much broader set of use cases that we’re already familiar with. So it’s not a replacement, but rather a complement.
When I argued recently that The Future of Frontend is Already Here, I prominently cited PartyKit as evidence for my claim. Today, the SaaS market boasts many new platforms for frontend optimized general compute, but what makes the PartyKit story, and Pai’s philosophy that “The next evolution of serverless is stateful” compelling is what it suggests about compute today, and where workloads will be processed in the future.
PartyKit-Cloudflare Acquisition
On April 5th, PartyKit’s already interesting story got added lift when Cloudflare announced it had acquired Pai’s startup. Beyond technical compatibility (after all, PartyKit wraps Cloudflare Durable Objects), this acquisition is interesting on multiple fronts. As a former Cloudflare employee and rockstar in the developer space, Pai is a desirable acquihire. What is more, PartyKit has high expectations for profitability with 2.5 million in pre-seed funding and a one-person team (following Pai’s decision last November to “[strip] PartyKit way back” by laying off all other full time employees). Finally, Cloudflare has always bet big on performance and network capabilities, and PartyKit advances these goals.
In the announcement, Cloudflare positions the PartyKit acquisition as a product play centered around stateful serverless compute:
With PartyKit now under our roof, we’re doubling down on our commitment to this future — a future where serverless is stateful … PartyKit aims to be a complement to traditional serverless computing, providing a more intuitive and efficient method for developing applications that require stateful behavior, thereby marking the “next evolution” of serverless computing.
This product play is evident to Todd Morey, who acknowledges his preference for a pared down stack owing to his own fatigue with tying together several independent SaaS products. He argues that pre-acquisition PartyKit offered:
a classic example of landing in that space of being a feature vs a product … So Partykit, while interesting tech, may have just struggled as independent infrastructure.
While Cloudflare’s success in the product space is debatable from a primitives versus packaged solutions perspective, their messaging establishes the importance of providing an “intuitive and efficient” developer experience to support real time experiences (important for gaming, collaboration, chatbots, and pub-sub, among other use cases) by maintaining state across multiple clients. In interviews with both Pai and representatives from Cloudflare I learned that there are no plans to change PartyKit’s open source license (MIT), and Pai will continue to build PartyKit in-house. This is noteworthy from a community standpoint as some PartyKit users have elected to build out their own backends rather than use Cloudflare’s Durable Objects.
While early to the CDN market, Cloudflare is a relative latecomer to the cloud functions market, with Workers being announced in 2017 and generally available in 2018. This is four years after AWS’s wildly successful Lambda (2014) went GA, followed by the Apache OpenWhisk project in 2015, which came out of IBM Research and serves as the basis for IBM Cloud Functions (2016). Microsoft Azure Functions also appeared in 2016, and Google Cloud Functions appeared in 2018 as the latest offering in the highly competitive hyperscale cloud functions market. Of course, new offerings continue to appear (DigitalOcean Functions debuted in 2022), but today the functions market has officially transitioned into a packaging exercise. While their marketing copy obfuscates the fact, Vercel’s Edge Functions (2022) are a wrapper running CloudFlare Workers, and its regular serverless functions run AWS. Similarly, although Pai speaks openly on podcasts and in the documentation about PartyKit’s use of Cloudflare’s Durable Objects under the hood, this fact does not appear on the PartyKit homepage. This is not a criticism. PartyKit and Vercel both make the developer experience they offer by means of abstracting away the fiddly parts of running compute primitives the primary selling points for their platforms.
Ralph Squillace, principal PM for open source and container-native developer experience in Microsoft Azure, takes a broader perspective of this acquisition by tying it to this fundamental shift in where compute is happening:
I see PartyKit as a logical step forward in the compute and connectivity of clients section of Cloudflare’s approach to building a full streaming compute platform on circumference compute, outside of origin.
It is this larger movement in the compute space wrestling with where workloads will be processed that makes the PartyKit acquisition worth following. The vast amount of data in the next 10 years will be created on the edge, which for Squillace means not hyperscale, not CDN, but rather compute that happens near the user.
It’s worth defining “user,” as it may make more sense to run compute near the database rather than the end user’s device depending on an app’s architecture. In this case, the “user” is the backend infrastructure. I spoke with Rita Kozlov, VP of Product at Cloudflare (who, incidentally, recorded a RedMonk Conversation with RedMonk’s own confirmed Edge-skeptic on the question of whether “You Need to Care About Edge?”), about how Cloudflare is thinking about the larger transition in serverless to move workloads closer to the user:
We’ve never used “edge” in our messaging. Edge is just kind of a solution in search of a problem. It’s not really clear what it solves.
For this reason, Cloudflare prefers the term smart placement to describe how they ensure Workers run in the most performant locations.
This evolution in compute location deeply concerns the rise in AI workloads. Physical limitations (GPUs, electrical grid) restrict the vast amounts of data necessary to train and use LLMs from being produced in one location and processed in another. Our AI future requires compute that must be moved close to wherever it is consumed. JavaScript developers are merely the earliest adopters of AI’s impending wave. The explosion of client side tools that have enabled frontend developers to leverage compute in the browser tell us much about the future of edge computing. Where data is being created now cannot be shipped back to core for compute work. Shifts in where data is processed will only become more evident as AI workloads expand, but it also reflects the proliferation and maturity of JS frameworks which allow users to accomplish more functionally in the browser. This evolution in serverless compute does much to explain why Pai and Cloudflare decided to bet big on PartyKit.
To wrap up, by acquiring PartyKit, Cloudflare is not only making its streaming functionality easier to consume, particularly by frontend developers, as a feature of its Durable Objects product, it also reflects a larger trend in the serverless compute space. Whether data is cached or else deliberately stored doesn’t matter: what matters is that the cloud of the future places a premium upon workloads that run close to the user.
Disclaimer: AWS, Akamai, Cloudflare, Fastly, Google, and Microsoft are RedMonk clients.
Header image: “A party set in the clouds” created with Dall-E
No Comments