My colleague Stephen recently wrote an excellent and well-received post about the Developer Experience Gap – the chasm separating modern promises of developer productivity with the reality of stitching environments together by hand. There are so many powerful tools and platforms out there – we keep calling it a Golden Age for developers – but choice and fragmentation, and a lack of focus on the end to end experience, are holding us back as an industry. Pipelines remain a challenge to build and maintain.
What the market is telling developers and their employers alike, effectively, is that the market can provide a system that will shepherd code from its earliest juvenile days in version control through to its adult stage in production. It is telling them that the system can be robust, automated and increasingly intelligent. And it is also telling them that they have to build and maintain it themselves.
and
Fragmentation makes it impossible for vendors to natively supply the requisite components for a fully integrated toolchain. That does not change the reality, however, that developers are forced to borrow time from writing code and redirect it towards managing the issues associated with highly complex, multi-factor developer toolchains held together in places by duct tape and baling wire. This, then, is the developer experience gap. The same market that offers developers any infrastructure primitive they could possibly want is simultaneously telling them that piecing them together is a developer’s problem.”
I couldn’t help being reminded of Stephen’s post when I read this excellent post on Hashicorp’s new Waypoint platform, launched at Hashiconf recently, by Cindy Sridharan.
Sridharan begins by usefully defining what Waypoint isn’t
- a build system (like make, npm, maven etc.)
- a package manager (like helm, pip etc.)
a CI system (like Jenkins, GitHub Actions etc.)- an Artifact registry (like Artifactory, Docker Registry etc.)
- a runtime format (like OCI images, Docker containers, buildpacks, native binaries, archives etc.)
- a cluster orchestrator or platform (like Kubernetes, EC2, EKS etc.)”
Her positive definition follows
I’d personally describe Waypoint as a minimalistic, non-prescriptive and heavily customizable build and deployment tool, that offloads most of the heavy-lifting of the complexity inherent in the “build — test — deploy — release” cycle to existing underlying build/test/deployment tools and services, while offering end-users a uniform and declarative interface to describe the “build — test — deploy — release” workflows in a manner that’s “native” to said underlying tools. Put simply, it’s the best kind of “glue infrastructure”.
“The best kind of glue infrastructure”… In his launch post Hashicorp co-founder Mitchell Hashimoto describes the platform like so:
A consistent developer workflow to build, deploy, and release applications across any platform.
Hashicorp has explicitly made a slight change of tack here, rather than building another new tool for operators, it is addressing the Developer Experience Gap.
Of course Hashicorp isn’t alone in addressing the gap.
There are a host of new platforms and approaches designed to make it easier for developers to maintain flow, with solid automation. New companies have been founded on the basis of opinionated tooling that really understands how modern developers want to work. That’s newcode rather than gluecode.
Vercel is a great example, which collapses distinctions between development and production with highly productive workflows, doing the grunt work of setting things up for the developer. It’s a CDN, but made for frontend developers. Vercel is highly opinionated, and its Next.js framework for React is seeing explosive growth. When Simon Willinson first introduced me to the tool it reminded me of the first time I saw Heroku.
Render and Hostman are both interesting new players, also offering instant continuous deploys from the developer’s repo.
Netlify offers an opinionated approach to development, embracing and extending the Static Site approach (you should read this excellent SSG primer by my colleague Rachel) to content management into interactive services with its JAMstack model. Netlify has done an outstanding job of appreciating the Developer Experience Gap and addressing it. It was already leading the industry in that respect when it hired Sarah Drasner to take it to the next level.
As Stephen puts it:
The combination of the emerging serverless products and the innate capabilities of the various CDNs to host JavaScript, API and Markdown driven applications begins to position CDNs as a target not just for last mile delivery of hashed content but as a first class destination for a sizable and growing subset of enterprise application workloads.
Given the next gen CDN momentum here, Fastly, Cloudflare and Stackpath are of course all going to be in the mix – they’re all investing heavily and effectively in better developer experiences for edge deployment – see Fastly Cloud @Edge and Cloudflare Workers and Stackpath Serverless Scripting.
GitOps, an approach pioneered by Weaveworks, allows for automated workflows and interactions between dev and ops, so developers can write code and make infrastructure changes with pull requests. Pretty much every source code company and developer tools company is now adopting the pattern – see GitLab for example. It’s not just front end developers that want to avoid configuration hassles. Let developers live in Git, their editor and Slack.
In terms of the hyperscale clouds Microsoft is in the best position to pull all of these approaches together given its assets and history: from editor (Visual Studio Code) to source code repo and collaboration (GitHub), with CI/CD (Actions) and deployment (Pages, Azure). That said, Microsoft has a long way to go to fulfill the promise of what it used to call Integrated Innovation and really nail the Developer Experience gap. But GitHub is undoubtedly the key acquisition of the era in addressing it. It’s already proving to be a bargain for Microsoft.
Google Cloud deserves an honourable mention for Cloud Run, a highly productive platform for taking Docker containers into production with a minimum of fuss. It maps really well to modern developer workflows.
AWS is generally not great at integrated Developer Experience – rather it’s a platform of primitives, which do indeed require a lot of stitching together. That said, the AWS Amplify team is doing an excellent job of tracking modern developer trends and needs, and is probably the most developer-focused product group in the company right now.
The fact that the hyperscale clouds aren’t there yet creates opportunities for the likes of Hashicorp, but also folks like Pulumi and Atomist. Pulumi recently took $37.5m in VC funding to build out what it calls its “Cloud Engineering platform” – developers can use their favourite languages to build infrastructure workflows. The founding team came out of Microsoft, and have a solid focus on developer experience and tooling. Atomist, another anti- YAML company, is also building tools to automate development, deployment and operations tasks, with its new Skills platform. Founder Rod Johnson of Spring fame knows a bit about avoiding yak shaving. Atomist, though, is not taking the opinionated approach of “use this new platform for a better developer experience”, rather it’s about providing an overlay for existing tools and platforms.
So in conclusion, Newcode or Gluecode, there is a ton of interesting stuff going on to address The Developer Gap. Addressing this gap will define success in the industry going forward. Companies that makes their developers most successful are going to win in their markets. This post isn’t intended as a comprehensive survey of all the tools players out there, but rather to point to some of the work on, and some folks to watch and try. For obvious reasons this is a fruitful area of research for RedMonk, given our thesis that developers are increasingly the most influential and important group in tech.
Disclosure statement: AWS, Azure, Cloudflare, Fastly, GCP, GitHub, GitLab, Hashicorp, Stackpath and Weaveworks are all RedMonk clients.
No Comments