{"id":253,"date":"2024-04-26T16:01:09","date_gmt":"2024-04-26T16:01:09","guid":{"rendered":"https:\/\/redmonk.com\/kholterhoff\/?p=253"},"modified":"2024-04-26T18:54:44","modified_gmt":"2024-04-26T18:54:44","slug":"whither-serverless-compute","status":"publish","type":"post","link":"https:\/\/redmonk.com\/kholterhoff\/2024\/04\/26\/whither-serverless-compute\/","title":{"rendered":"Whither Serverless Compute? or Why the Cloudflare-PartyKit Acquisition Matters"},"content":{"rendered":"<p><img decoding=\"async\" class=\"alignnone size-full wp-image-254\" src=\"http:\/\/redmonk.com\/kholterhoff\/files\/2024\/04\/cloud-party.jpg\" alt=\"\" width=\"100%\" height=\"897\" srcset=\"https:\/\/redmonk.com\/kholterhoff\/files\/2024\/04\/cloud-party.jpg 1716w, https:\/\/redmonk.com\/kholterhoff\/files\/2024\/04\/cloud-party-300x157.jpg 300w, https:\/\/redmonk.com\/kholterhoff\/files\/2024\/04\/cloud-party-1024x535.jpg 1024w, https:\/\/redmonk.com\/kholterhoff\/files\/2024\/04\/cloud-party-768x401.jpg 768w, https:\/\/redmonk.com\/kholterhoff\/files\/2024\/04\/cloud-party-1536x803.jpg 1536w, https:\/\/redmonk.com\/kholterhoff\/files\/2024\/04\/cloud-party-480x251.jpg 480w, https:\/\/redmonk.com\/kholterhoff\/files\/2024\/04\/cloud-party-1200x627.jpg 1200w\" sizes=\"(max-width: 1716px) 100vw, 1716px\" \/><\/p>\n<p>Client side compute and edge compute are converging. Web apps are transitioning where data is processed closer to the user\u2014a move inclusive of not only compute on the device using client side rendering engines like V8, Google&#8217;s JavaScript and WebAssembly engine, but also the so-called \u201cedge.\u201d As Stephen O\u2019Grady has so <a href=\"https:\/\/redmonk.com\/sogrady\/2023\/11\/03\/edge\/\">eloquently explained<\/a>, despite \u201cdozens or perhaps hundreds of technology vendors at present hav[ing] substantial open investments in marketing edge branded products,\u201d the term \u201cedge\u201d is rife with ambiguity. However, beyond exasperating critics if \u201cedge\u201d accomplishes nothing else, it does the important work of signifying servers that are geographically closer to the user than cloud data centers historically have been.<\/p>\n<p>This trend, which AI workloads will only serve to amplify, is moving serverless compute closer to the user for performance reasons\u2014a transition which has repercussions for packaging, runtimes, databases, and abstractions in general. Serverless compute (an idea that is itself <a href=\"https:\/\/redmonk.com\/rstephens\/2018\/12\/14\/serverless-more-than-just-functions\/\">fraught with ambiguity<\/a>, as Rachel Stephens notes) is where all this convergence around the location of data processing, which has migrated away from local data centers (colocation, rackspace), and toward the cloud and the devices themselves (browsers, IoT), whether you want to use the term \u201cedge\u201d or not.<\/p>\n<p>The future of compute will not occur at remote data centers, but will instead take place locally and close to the user. Cloud providers have risen to the challenge of creating a global network of data centers to ensure the workloads they manage are processed close by its customers. US-EAST-1, AWS\u2019s first region and \u201cproblem child,\u201d which <a href=\"https:\/\/www.linkedin.com\/in\/david-brown-aws\/\">David Brown<\/a>, VP for compute and networking services at AWS, <a href=\"https:\/\/www.theregister.com\/2024\/04\/10\/aws_dave_brown_ec2_futures\/\">recently defended<\/a>, is far from the only or even the default option for AWS cloud customers who can now take advantage of <a href=\"https:\/\/aws.amazon.com\/about-aws\/global-infrastructure\/localzones\/locations\/\">local zones<\/a>. Cloudflare\u2019s homepage boasts that 95% of the world&#8217;s population is within <a href=\"https:\/\/www.cloudflare.com\/application-services\/products\/cdn\/\">50 milliseconds<\/a> of a Cloudflare Worker, with 80% of them being within <a href=\"https:\/\/blog.cloudflare.com\/250-cities-is-just-the-start\">20 milliseconds<\/a>. This shift goes beyond CDN, a technology that emerged in the 90s to transmit cached data using a distributed network of servers (<a href=\"https:\/\/redmonk.com\/sogrady\/2020\/06\/10\/convergent-evolution-cdns-cloud\/\">a history<\/a>). Vendors including Akamai, Cloudflare, edge.io, Edgio, Fastly, and others realized they could not run functions on CDN, and needed an alternative solution. While strategies and product offerings vary among these players, the race is on to ensure compute occurs as close to the user as possible because today\u2019s workloads increasingly process real time, streaming data. This shift matters because streaming data is finicky. It requires support architectures intended to reduce the heaviness and latency costs associated with moving data into and out of a database, making edge computing, in the sense of data processing occurring at a relative position near the user, absolutely essential.<\/p>\n<p>&nbsp;<\/p>\n<h2>Frontend Workloads<\/h2>\n<p>The shift in where compute happens makes frontend and mobile developers, who work almost exclusively on client side code, heralds of a significant transition in the cloud computing space. In fact, by harnessing the wave of powerful emerging JavaScript tools and technologies that have appeared on the market in recent years, both frontend engineers and the vendors that recognize them as <a href=\"https:\/\/redmonk.com\/kholterhoff\/2024\/02\/15\/frontend-developers-the-newest-new-kingmakers\/\">Kingmakers<\/a>, have positioned themselves as innovators at the tip of the spear.<\/p>\n<blockquote class=\"twitter-tweet\" data-width=\"500\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">So, <a href=\"https:\/\/twitter.com\/vercel?ref_src=twsrc%5Etfw\">@vercel<\/a> reverted all edge rendering back to Node.js \ud83d\ude2c<\/p>\n<p>Wanted to correct the record here as it&#39;s something I&#39;ve advocated for in the past, and share what I&#39;ve learned since then.<\/p>\n<p>Also, the &quot;edge&quot; naming has been a bit confusing, so let&#39;s clear that up here as well.<\/p>\n<p>What is\u2026<\/p>\n<p>&mdash; Lee Robinson (@leeerob) <a href=\"https:\/\/twitter.com\/leeerob\/status\/1780705942734331983?ref_src=twsrc%5Etfw\">April 17, 2024<\/a><\/p><\/blockquote>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<p>The shift toward compute near the user explains much about the brouhaha that erupted following a tweet from Lee Robinson, the VP of Product at Vercel, the self-described \u201cfrontend cloud.\u201d In it, Lee not only articulates his \u201clearnings\u201d about edge as a technology, but he also announces Vercel\u2019s decision to revert \u201call edge rendering back to Node.js.\u201d Evidently, Vercel has discovered that they can\u2019t do server side rendering (SSR), and instead must move their functions to partial pre-rendering (PPR) with Node. This realization likely has much to do with latency and cost added from having Vercel&#8217;s AWS infra speak to CloudFlare, as Vercel\u2019s Edge Functions run CloudFlare Workers, which adds egress costs owing to CloudFlare&#8217;s <a href=\"https:\/\/www.cloudflare.com\/bandwidth-alliance\/\">Bandwidth Alliance<\/a>. Vercel&#8217;s adoption of Node also suggests an interesting story around V8, as both <a href=\"https:\/\/nodejs.org\/en\/learn\/getting-started\/the-v8-javascript-engine\">Node<\/a> and <a href=\"https:\/\/developers.cloudflare.com\/workers\/reference\/how-workers-works\/\">Cloudflare<\/a> run V8. But beyond bypassing AWS-Cloudflare friction, and ensconcing technical and partner alignment with Google\u2014all subjects worthy of standalone blog posts\u2014, what interests me about Lee\u2019s post and its reception is what it tells us about the state of serverless.<\/p>\n<p>Where compute is running is deeply significant to frontend engineering teams. <a href=\"https:\/\/www.linkedin.com\/in\/ashleygwilliams\/\">Ashley Williams<\/a>, former Staff Product Manager at Vercel and Systems Engineer at Cloudflare, has noted the importance of &#8220;<a href=\"https:\/\/www.youtube.com\/watch?v=MBndZddVQdw\">JavaScript&#8217;s Journey to the Edge.<\/a>&#8221; She suggests that frontend engineers, even when they could care less about IT networking, benefit from the transition to close compute. Significantly, in most cases frontend workloads are lightweight enough not to mandate the security and latency costs associated with running compute in a data center. While many browser based JavaScript applications are currently needlessly heavy, this is beginning to improve as single page applications (SPAs) move to adopt more HTML-first architectures. For instance, \u201ccomponent islands,\u201d a pattern coined in 2019 by <a href=\"https:\/\/www.linkedin.com\/in\/ksylor\/\">Katie Sylor-Miller<\/a>, a frontend architect at Etsy, allow for portions of the page to render as static HTML delivered from the server, while interactive portions of the page are delivered as self-contained client side JavaScript. All of these innovations in the space suggest that frontend developers are well aware that the KISS principle applies to serverless. In point of fact, this has long been the argument made by <a href=\"https:\/\/www.linkedin.com\/in\/sunil-pai-a47732253\/\">Sunil Pai<\/a>, founder of the open source real time platform PartyKit and Principal Systems Engineer at Cloudflare. Pai explains in <a href=\"https:\/\/sunilpai.dev\/posts\/the-future-of-serverless\/\">a post<\/a> on his personal blog:<\/p>\n<blockquote><p>That\u2019s the kicker: the \u201cnext\u201d evolution of serverless isn\u2019t really something \u201cnew\u201d, but rather a return to how we would write code if we were running on a single machine. A return to simplicity, if you will. And while traditional serverless (lol) is great for stateless workloads, stateful serverless unlocks using a similar programming model for a much broader set of use cases that we\u2019re already familiar with. So it\u2019s not a replacement, but rather a complement.<\/p><\/blockquote>\n<p>When I argued recently that <a href=\"https:\/\/redmonk.com\/kholterhoff\/2024\/03\/28\/the-future-of-frontend-is-already-here\/\">The Future of Frontend is Already Here<\/a>, I prominently cited PartyKit as evidence for my claim. Today, the SaaS market boasts many new platforms for frontend optimized general compute, but what makes the PartyKit story, and Pai\u2019s philosophy that \u201cThe next evolution of serverless is stateful\u201d compelling is what it suggests about compute today, and where workloads will be processed in the future.<\/p>\n<p>&nbsp;<\/p>\n<h2>PartyKit-Cloudflare Acquisition<\/h2>\n<p>On April 5th, PartyKit\u2019s already interesting story got added lift when Cloudflare announced it had acquired Pai\u2019s startup. Beyond technical compatibility (after all, PartyKit wraps Cloudflare Durable Objects), this acquisition is interesting on multiple fronts. As a former Cloudflare employee and <a href=\"https:\/\/github.com\/threepointone\/\">rockstar<\/a> in the developer space, Pai is a desirable acquihire. What is more, PartyKit has high expectations for profitability with <a href=\"https:\/\/www.crunchbase.com\/organization\/partykit\">2.5 million in pre-seed funding<\/a> and a one-person team (following Pai\u2019s decision last November to \u201c<a href=\"https:\/\/twitter.com\/threepointone\/status\/1724159621382754738\">[strip] PartyKit way back<\/a>\u201d by laying off all other full time employees). Finally, Cloudflare has always bet big on performance and network capabilities, and PartyKit advances these goals.<\/p>\n<p>In the announcement, Cloudflare <a href=\"https:\/\/blog.cloudflare.com\/cloudflare-acquires-partykit\">positions<\/a> the PartyKit acquisition as a product play centered around stateful serverless compute:<\/p>\n<blockquote><p>With PartyKit now under our roof, we&#8217;re doubling down on our commitment to this future \u2014 a future where serverless is stateful \u2026 PartyKit aims to be a complement to traditional serverless computing, providing a more intuitive and efficient method for developing applications that require stateful behavior, thereby marking the &#8220;next evolution&#8221; of serverless computing.<\/p><\/blockquote>\n<p>This product play is evident to <a href=\"https:\/\/news.ycombinator.com\/user?id=toddmorey\">Todd Morey<\/a>, who acknowledges his preference for a pared down stack owing to his own fatigue with tying together several independent SaaS products. He <a href=\"https:\/\/news.ycombinator.com\/item?id=39946329\">argues<\/a> that pre-acquisition PartyKit offered:<\/p>\n<blockquote><p>a classic example of landing in that space of being a feature vs a product \u2026 So Partykit, while interesting tech, may have just struggled as independent infrastructure.<\/p><\/blockquote>\n<p>While Cloudflare\u2019s success in the product space is debatable from a primitives versus packaged solutions perspective, their messaging establishes the importance of providing an \u201cintuitive and efficient\u201d developer experience to support real time experiences (important for gaming, collaboration, chatbots, and pub-sub, <a href=\"https:\/\/sunilpai.dev\/posts\/the-future-of-serverless\/#what-is-it-good-for\">among other use cases<\/a>) by maintaining state across multiple clients. In interviews with both Pai and representatives from Cloudflare I learned that there are no plans to change PartyKit\u2019s open source license (MIT), and Pai will continue to build PartyKit in-house. This is noteworthy from a community standpoint as some PartyKit users have elected to build out their own backends rather than use Cloudflare\u2019s Durable Objects.<\/p>\n<p>While early to the CDN market, Cloudflare is a relative latecomer to the cloud functions market, with Workers being <a href=\"https:\/\/blog.cloudflare.com\/introducing-cloudflare-workers\">announced<\/a> in 2017 and <a href=\"https:\/\/blog.cloudflare.com\/cloudflare-workers-unleashed\/\">generally available<\/a> in 2018. This is four years after AWS\u2019s wildly successful <a href=\"https:\/\/aws.amazon.com\/about-aws\/whats-new\/2014\/11\/13\/introducing-aws-lambda\/\">Lambda<\/a> (2014) went GA, followed by the <a href=\"https:\/\/medium.com\/@rabbah\/the-state-of-openwhisk-ae8c129e8a48\">Apache OpenWhisk<\/a> project in 2015, which came out of IBM Research and serves as the basis for <a href=\"https:\/\/www.ibm.com\/products\/functions\">IBM Cloud Functions<\/a> (2016). <a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/announcing-general-availability-of-azure-functions\/\">Microsoft Azure Functions<\/a> also appeared in 2016, and <a href=\"https:\/\/www.serverless.com\/blog\/google-cloud-functions-generally-available-what-means-serverless\">Google Cloud Functions<\/a> appeared in 2018 as the latest offering in the highly competitive hyperscale cloud functions market. Of course, new offerings continue to appear (<a href=\"https:\/\/www.infoq.com\/news\/2022\/05\/digitalocean-functions-serverles\/\">DigitalOcean Functions<\/a> debuted in 2022), but today the functions market has officially transitioned into a packaging exercise. While their marketing copy obfuscates the fact, <a href=\"https:\/\/vercel.com\/blog\/edge-functions-generally-available\">Vercel\u2019s Edge Functions<\/a> (2022) are a wrapper running CloudFlare Workers, and its regular serverless functions run AWS. Similarly, although Pai speaks openly on podcasts and in the documentation about PartyKit\u2019s use of Cloudflare\u2019s Durable Objects under the hood, this fact does not appear on the PartyKit homepage. This is not a criticism. PartyKit and Vercel both make the developer experience they offer by means of abstracting away the fiddly parts of running compute primitives the primary selling points for their platforms.<\/p>\n<p><a href=\"https:\/\/github.com\/rasquill\">Ralph Squillace<\/a>, principal PM for open source and container-native developer experience in Microsoft Azure, takes a broader perspective of this acquisition by tying it to this fundamental shift in where compute is happening:<\/p>\n<blockquote><p>I see PartyKit as a logical step forward in the compute and connectivity of clients section of Cloudflare&#8217;s approach to building a full streaming compute platform on circumference compute, outside of origin.<\/p><\/blockquote>\n<p>It is this larger movement in the compute space wrestling with where workloads will be processed that makes the PartyKit acquisition worth following. The vast amount of data in the next 10 years will be created on the edge, which for Squillace means not hyperscale, not CDN, but rather compute that happens near the user.<\/p>\n<p>It\u2019s worth defining \u201cuser,\u201d as it may make more sense to run compute near the database rather than the end user\u2019s device depending on an app\u2019s architecture. In this case, the &#8220;user&#8221; is the backend infrastructure. I spoke with <a href=\"https:\/\/www.linkedin.com\/in\/ritakozlov\/\">Rita Kozlov<\/a>, VP of Product at Cloudflare (who, incidentally, recorded a RedMonk Conversation with RedMonk\u2019s own confirmed Edge-skeptic on the question of whether \u201c<a href=\"https:\/\/redmonk.com\/videos\/a-redmonk-conversation-do-you-need-to-care-about-edge\/\">You Need to Care About Edge?<\/a>\u201d), about how Cloudflare is thinking about the larger transition in serverless to move workloads closer to the user:<\/p>\n<blockquote><p>We&#8217;ve never used \u201cedge\u201d in our messaging. Edge is just kind of a solution in search of a problem. It&#8217;s not really clear what it solves.<\/p><\/blockquote>\n<p>For this reason, Cloudflare prefers the term <a href=\"https:\/\/developers.cloudflare.com\/workers\/configuration\/smart-placement\/\">smart placement<\/a> to describe how they ensure Workers run in the most performant locations.<\/p>\n<p>This evolution in compute location deeply concerns the rise in AI workloads. Physical limitations (GPUs, electrical grid) restrict the vast amounts of data necessary to train and use LLMs from being produced in one location and processed in another. Our AI future requires compute that must be moved close to wherever it is consumed. JavaScript developers are merely the earliest adopters of AI&#8217;s impending wave. The explosion of client side tools that have enabled frontend developers to leverage compute in the browser tell us much about the future of edge computing. Where data is being created now cannot be shipped back to core for compute work. Shifts in where data is processed will only become more evident as AI workloads expand, but it also reflects the proliferation and maturity of JS frameworks which allow users to accomplish more functionally in the browser. This evolution in serverless compute does much to explain why Pai and Cloudflare decided to bet big on PartyKit.<\/p>\n<p>To wrap up, by acquiring PartyKit, Cloudflare is not only making its streaming functionality easier to consume, particularly by frontend developers, as a feature of its Durable Objects product, it also reflects a larger trend in the serverless compute space. Whether data is cached or else deliberately stored doesn\u2019t matter: what matters is that the cloud of the future places a premium upon workloads that run close to the user.<\/p>\n<p><strong>Disclaimer:<\/strong> AWS, Akamai, Cloudflare, Fastly, Google, and Microsoft are RedMonk clients.<\/p>\n<p><sup>Header image: &#8220;A party set in the clouds&#8221; created with Dall-E<\/sup><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Client side compute and edge compute are converging. Web apps are transitioning where data is processed closer to the user\u2014a move inclusive of not only compute on the device using client side rendering engines like V8, Google&#8217;s JavaScript and WebAssembly engine, but also the so-called \u201cedge.\u201d As Stephen O\u2019Grady has so eloquently explained, despite \u201cdozens<\/p>\n","protected":false},"author":50,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[32,36,8,4,37],"tags":[],"class_list":["post-253","post","type-post","status-publish","format-standard","hentry","category-ai","category-edge","category-frontend","category-javascript","category-serverless"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/redmonk.com\/kholterhoff\/wp-json\/wp\/v2\/posts\/253","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/redmonk.com\/kholterhoff\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/redmonk.com\/kholterhoff\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/redmonk.com\/kholterhoff\/wp-json\/wp\/v2\/users\/50"}],"replies":[{"embeddable":true,"href":"https:\/\/redmonk.com\/kholterhoff\/wp-json\/wp\/v2\/comments?post=253"}],"version-history":[{"count":0,"href":"https:\/\/redmonk.com\/kholterhoff\/wp-json\/wp\/v2\/posts\/253\/revisions"}],"wp:attachment":[{"href":"https:\/\/redmonk.com\/kholterhoff\/wp-json\/wp\/v2\/media?parent=253"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/redmonk.com\/kholterhoff\/wp-json\/wp\/v2\/categories?post=253"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/redmonk.com\/kholterhoff\/wp-json\/wp\/v2\/tags?post=253"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}