The two strongest forces in the universe are outrage and confirmation bias. When it comes to highly opinionated marketing driving the outrage button we can’t think of a more resonant recent campaign than Humanitec’s DevOps is Dead, Embrace Platform Engineering management. It really struck a chord even if, or perhaps because, it annoyed a lot of people.
In terms of selling to Platform Engineering teams, Humanitec positions itself as powering internal developer platforms by providing orchestration and automation of configuration as code. Internal developer platforms are having a moment in the sun, with a lot of excitement about software like Spotify Backstage, so it makes a lot of sense to ride this wave.
I recently recorded a video with Drew Oetzel, senior customer success engineer at Humanitec. TLDR – configuration as code is great, but it leads to sprawl. It’s one thing to say we’re going to treat configuration in the same way that we do software engineering, but it’s quite another to deliver on the promise, when we live in world with so many different config platforms and so many different teams doing the work.
According to Oetzel everybody does some platform engineering. At companies like Adobe or Google it can be really slick but at medium sized organisations it might just be a couple of people clicking around in AWS, supporting their dev teams with clicks in the console. As Corey Quinn calls it “ClickOps”
So if you’re at the smaller end of the scale – congratulations! You’re the platform engineering team now. You get to manually set something up as you go through your tickets.
So what even is a platform orchestrator and why would it help? According to Oetzel, it shuffles config files. So it’s shuffling Terraform scripts, it’s shuffling Helm charts, it’s shuffling Kubernetes, and YAML scripts. Automated, it essentially eliminates undifferentiated heavy lifting.
The idea of Humanitec is to automate the tedious interactions between dev and ops. So, to free up your devs from having to understand the intricacies of security on an S3 bucket or the intricacies of this particular Terraform script or something like that. And it also frees up your ops side from having to automatically go and run that Terraform script, for a special test case or to go and make the adjustments that a particular dev needs to run a particular kind of test. So there’s a lot of tedious work. Once you double click on this thing, it’s automated after that, but you still have to double click on it. So you have to go in and make that change or it’s stuck somewhere. And you have to go and figure that out and it turns out its because you accidentally indicated the staging values.yaml instead of the dev values.yaml. There’s so many different config files that are being juggled.
Microservices architectures only exacerbate these problems – so many small files, so similar to each other, flying around. Orchestration provides a means to enable small config changes per environment.
In this video we discuss the current state of the art, but also what kind of tooling might offer a better, more automated experience for platform engineering teams. How we can avoid config file and script sprawl and focus on our jobs, automating the tedious interactions between dev and ops.
Platform Engineering is about standardisation by design, and that happens as part of the process. Golden paths, platform orchestration, and dynamic configuration management. Watch the video to learn more, and tell us what you think.
This was a RedMonk conversation, sponsored by Humanitec.
AWS and Hashicorp are both RedMonk clients.
May 26, 2023 at 7:27 pm
Use sqlite to manage your configs. I’ve started doing this and it’s legitimately amazing. It took me a few days, but I completely undid a rat’s nest of config files. You model environments, roles, servers, all that shit as tables. If something needs to be constrained, you add a foreign key or a key constraint.
Need a specific format? A view and a template generator is easy to write up to spit out what’s needed.
Need some special validation logic? Easy enough to write triggers or computed columns or whatever.
Want to version it? Good news, you can dump it to text and see the diff results easily.
If you don’t actually like config files, good news, you can just load the config db from damn near any language. Just mark the db as read only by the production user.
Biggest problem is you actually have to have a mental model of what your configs actually are instead of just spreading a layer of trash on your services.
James Governor says:
May 30, 2023 at 3:42 pm
thanks Danno. that’s super interesting. folks using sqlite in all kinds of interesting ways