When it comes to highly opinionated marketing, driving the outrage button, we can’t think of a more resonant recent campaign than Humanitec’s DevOps is dead, long live Platform Engineering! In this video though we come to discuss platform orchestration and dynamic configuration management.

So – a conversation between James Governor, co-founder of RedMonk, and Drew Oetzel, senior customer success engineer at Humanitec.

What’s the interplay between platform orchestration and platform engineering? Why should platform engineering care about this? What even is wrong with the current state of the art in configuration management?

The truth is everybody does some platform engineering; everybody has some form of platform engineering. And at companies like Adobe or Google it can be really slick but at medium sized organisations, it might just be a couple of people clicking around in AWS. As Corey Quinn calls it “ClickOps”

Congratulations. You’re the platform engineering solution. You get to manually set something up as you go through tickets. According to Humanitec, at the heart of what a platform orchestrator does is it shuffles config files. So it’s shuffling Terraform scripts, it’s shuffling Helm charts, it’s shuffling Kubernetes, YAMLs. Essentially undifferentiated heavy lifting.

In this video we discuss the current state of the art, but also what kind of tooling might offer a better, more automated experience for platform engineering teams. How we can avoid config file and script sprawl and focus on our jobs, automating the tedious interactions between dev and ops.

Platform Engineering is about standardisation by design, and that happens as part of the process. Golden paths, platform orchestration, and dynamic configuration management. Watch the video to learn more, and tell us what you think.

This was a RedMonk video, sponsored by Humanitec. Disclosure: Google and AWS are also RedMonk clients.

Rather listen to this conversation as a podcast?

 

Transcript

James Governor: Hi, it’s James Governor, co-founder of RedMonk. We’re here for another RedMonk Conversation. Today, I’m lucky enough to have Drew Oetzel with me. He’s from Humanitec. So, Drew, I’ll let you introduce yourself.

Drew Oetzel: Hey, thanks for that. My name is Drew Oetzel, as you said, James. Thanks for having me here. I am a Customer Success Engineer. I think I just got reclassified to Senior Customer Success Engineer. So I think it’s just the gray and the beard.

James: Congratulations!

Drew: Thank you. Thank you. I’ve been working in the startup world since, the early days of the 2000s. I actually lived through the the good old dotcom boom. I interviewed at Pets.com, even! So, you know, I didn’t get the job, though unfortunately.

James: Okay, that’s amazing. Anyway, so Humanitec! Been generating a lot of buzz and attention recently. For some highly opinionated marketing, the death of DevOps. So, yeah, definitely strong opinions in the company. It’s all about platform engineering now. But today we’re actually here to talk about platform orchestration and dynamic configuration management.

Drew: That’s correct.

James: Platform engineering is obviously a big part of this buzz at the moment. And so what’s the interplay between like, platform orchestration, platform engineering? Why should platform engineering care about this? What’s wrong? What is wrong, Drew with the state of the art in configuration management?

Drew: So one of the things that, you know, it’s funny — one of the reasons why I actually took the job at Humanitec is, the competition for Humanitec is not great. Because the truth is: everybody does some platform engineering; everybody has some form of platform engineering. And at someplace like Adobe or Google, you know, it’s going to be really slick and fancy, but at some medium sized places, it might just be you and your buddy clicking around in AWS. Congratulations. You’re the platform engineering solution. You know, by manually setting something up as you go through all your tickets. So that’s the reason why I jumped on to Humanitec because — everybody has it, but nobody’s really looked into professionalizing it and productizing it outside of the really big folks that are here in Silicon Valley that — that’s what they always do. They build it themselves and then it kind of grows out of there. Similar to the way Kubernetes did. But at the heart of what a platform orchestrator does is it shuffles config files. So it’s shuffling TerraForm scripts, it’s shuffling helm charts, it’s shuffling Kubernetes, YAMLs… or, you know, pick your infrastructure as code, pick your platform. But that’s what’s going on. That’s the orchestration, that’s the — the people are playing instruments. They’re playing config files in that orchestra.

James: Okay, and that sucks.

Drew: Yes, it absolutely sucks.

James: To be honest. I mean… some musical instruments sound okay when you’re not particularly good at playing them… a guitar? There’s a lot that sound pretty terrible. So there’s a lot of violins in the orchestra of orchestration tools, in fact.

Drew: Yes.

James: So yeah, I think that’s one of the conversations we’re having in the industry right now. So Kubernetes, to a point, you know, we we’ve expected too much of people. You mentioned mid-sized companies. I mean, trying to get people that fully understand both: potentially the world of app dev, and then, all the YAML and helm charts and everything else. We’re kind of trying at the moment, I think as an industry, to start saying “maybe we shouldn’t let everybody play the violin.”

Drew: Right. Right. And one of the ways I — so, you know, you mentioned our DevOps is dead motto from KubeCon. And there’s no such thing as bad publicity at KubeCon. If you can get people to come up to your booth and interact, even if they come up mad, you can send them away happy. And that’s what we did. And that was the point of that. You know, it was the — you have a small booth, but you be a little provocative and you be a little spicy. And then people come over.

James: It’s hard. And I mean, KubeCon, we got KubeCon Europe I’ll be at soon. And yeah, I mean, just KubeCon is just an environment where there are so many people, it is quite hard to get any kind of attention — in the industry at large — in around the CNCF, Kubernetes infrastructure, all of the associated tooling. I mean I think a little bit of judicious shitposting is probably par for the course.

Drew: Yeah. And you know, I come from Splunk and their first — they had the take the SH out of IT, which I know it seems ridiculous now, but in 2006, 2007 that was spicy.

James: Watching our pulse!

Drew: But people would get mad about that at conferences back then. So it’s a definitely a good marketing ploy. But what I tried to do to turn it around to the people that would come up and go, “I’m a DevOps engineer. Are you saying I’m dead?” And I’m like, “No, I’m saying DevOps is killing us, so maybe we should kill it before it kills us, you know?” Do we all want to be experts in every single aspect of a platform from TerraForm to S3 to Kubernetes to all the little things that run on Kubernetes, you know? There’s just so much, that we might expect a “you build it, you run it” ethos to require. And once we got to talking about it and they realized, first of all, it’s marketing. We’re not — take a deep breath. It’s not — it’s just marketing. But also, this is a genuine problem. This is — you’ve got — suddenly as a developer, you’re also responsible for ten different TerraForm scripts. And then because they need to be run slightly differently in different environments, those ten scripts have 30 override files that go along with them. And so now you went from — you’ve got 40 files that you’re juggling. And then multiply that by all the micro services and everything else that you’re creating. And you’re definitely going to be drowning very quickly in config files or ticket ops or some combination thereof. And that’s, to me, that was the main point. Like kill DevOps before it kills you.

James: Yeah. I mean I think we talk about undifferentiated heavy lifting. And just a different way of putting it to a point. I mean, sneakerware sucked. And automation should be something that we want more of. I mean, is part of the story that you’re saying that there are too many interfaces as well? And that we’re we’re unable to be in a position where we can just work in one place. We’re constantly having to context shift from one system to another. Is that part of the world that you’re trying to automate?

Drew: Yes, yes, absolutely. The idea of Humanitec is to automate the tedious interactions between dev and ops. So, to free up your devs from having to understand the intricacies of AWS security on an S3 bucket or the intricacies of this particular TerraForm script or something like that. And it also frees up your ops side from having to automatically go and run that TerraForm script to set up that RDS in the right — for a special test case or, you know, to go and make the adjustments that a particular dev needs to run a particular kind of test. So there’s a lot of tedious, you know — yes– once you double click on this thing, it’s automated after that, but you have to double click on it. You know, you have to go in and and do that or it’s stuck somewhere. And you have to go and figure out that, oh, well, it’s because you accidentally indicated the staging Values.yaml instead of the dev values.yaml. There’s so many different config files that are being juggled.

Drew: And the reality is the vast majority of these config files are just minor variations on each other. You know, they’re probably 80% the same and then 20% of them change per microservice. And then there’s minor changes per per environment after that. And the other problem that we see is the whole idea of — let’s break down our monoliths and let’s — okay, so now we broke down our monoliths into ten microservices, but we punished ourselves because instead of one monolith with ten config files, we now have ten microservices with 100 config files or 1000 config files. So it becomes, instead of a virtuous cycle, it becomes a vicious cycle of complexity. So something has to give before you know and that’s the evolution into platform engineering and to actually paying attention to your platform orchestration and automating the tedium that’s wearing down your devs and your ops.

James: And a part of this is that thing that you’re saying: hang on a second… all the microservices, even more config that we need to manage. It becomes just combinatorial, it just gets more and more complicated. Part of what you’re saying is you can have these different microservices while centralizing the management and metadata and the scripts required to manage those systems. So it doesn’t have to be somebody on each team that has all of that knowledge. You genuinely are saying, hang on a minute, we can centralize or consolidate regardless of the fact that we’ve got all of these different microservices.

Drew: Absolutely. And it’s that automation of that, you know, I call it the tedium. That is key. So one kind of silly way I like to think of Humanitec is you can kind of think of it as a too detailed oriented, micromanaging robot boss, but not for you. It’s for your CD system. It’s for your CI system, it’s for TerraForm. So it sits in between all of these things and facilitates — all right, TerraForm needs these config files now, Kubernetes needs these config files, and so it sits in between all of them. But it’s also checking up on them. And it’s hearing back from TerraForm and saying, all right, now we’re ready to move on to the next step. So a lot of times those things are achieved today maybe through a CD system. But the config file management is done through git and human interaction for the large part.

James: Okay. And so let’s talk about the dependency problem. Because we are dealing in, constantly, this world of complex dependencies. And so I’m sort of asking about the centralization and standardization there where we can begin to — well, there isn’t such so much of an issue, particularly in these ephemeral environments. Like what are you seeing existing, or what are you seeing prospects do as opposed to customers? Like what are you seeing as the current state of the art? And how can you — what then do you say to those people who say like, hang on a minute, your approach to the dependencies really sucks in terms of your operational environments.

Drew: So what I see a lot are — and I’m going to borrow a term — I see monolith environments. So they’ll have this really well planned out complex TerraForm script that generates their dev environment or that generates their staging environment. And you know, everything is accounted for in that big giant script. And this does work well, it is something that’s automated. After somebody hits start, it’s automated. But the problem is, the same problem you have with monolith workloads is that if you want to change one small part of that, it becomes a much bigger issue. Or if you want to generate an ephemeral version of that, it’s going to be a lot harder because — the entire thing has to be rebuilt. You couldn’t just maybe just change a few things and then run it side by side within the same environment. It’s very hard with these, Again, I’m going to call them monolithic environments to… An average deploy, you updated your workload, you hit it and you push and you’re going to make your test. That works great. But if you update a version of one of those dependencies or if you make a major change or you want to add in a new dependency, suddenly you have to test this entire monolith of an environment. And then being able to rapidly and quickly test just that change without having to redeploy the entire environment makes — that’s one of the aha moments for our customers. Once you’ve automated all of the tedium, then you can start to actually componentize and make ephemeral environments available at a click of a button or, you know, at the end of a CI curl command very easily. So that’s one of the big aha moments we get with our customers.

James: And so that’s managing to get to the good rather than just bad. But then what? I mean, I think we’re trying to — like one of the things that we really see a lot of and a lot of the conversation for enterprises is that they’re trying to learn from like: hang on a minute, we do need to standardize. You know, if we look at the Web companies that we might aspire to, they’re not just randomly doing stuff. The fact is that they have processes in place, approaches to enable golden paths to make life easier for the developer and the operator by introducing some constraints. I mean is that — do you have a policy basis that, in terms of your tooling, enables for that?

Drew: Yes, absolutely. Absolutely. That is — we call that standardization by design, and that happens as part of the process. So we work with two different, well, not just two different, but there’s two kinds of customers that we come in contact with a lot. There’s one that has a decent platform, but they just want to move on to something that they can have tech support for that’s not them. And they have already done the standardization work or never needed to do it because they started centralized. But what we find with a lot of our big customers, they had ten different departments. They all moved to Kubernetes on their own scale. They all made different — one department picked this one brand of CI, another department picked another brand of CI. So that’s the mess they’re in is they have just a massive — so they comment that standardization by design is — a really big point for them is that they can decide, all right, these are the dependencies we want to work with across — these are the ones that we’re going to make it easy for you to use these. We’re going to support these. And then try to bring in the people that are using other things. Because, you know, one thing with developers, it’s a lot easier to attract them than to compel them.

James: For sure, absolutely. I mean, convenience is the killer app. Every time.

Drew: Yeah. So that’s the two different customers that we work with.

James: Yep. And you have to talk to both. I mean, I’ve had this conversation recently quite a lot. You know, you got organisations like, well, we want more control, but yes, we still need to keep developers happy.

Drew: Yeah, yeah. And you know, it’s always like, well, you want to call it a walled garden, you want to call it a golden path. And it’s all like, how do we want to euphemize that we really would like to keep you limited with certain selections of your dependencies. You know, within reason. But that’s why we wind up on the words golden path.

James: Okay, awesome. Well, okay, we’ve got standardization by design. We’ve got golden paths, we’ve got platform orchestration, dynamic configuration management. These are all good things. And on that happy note, I think that we should wrap up, keep it short and sweet. So, Drew, thanks very much for joining us on the show. If you’re watching and you have any thoughts, reckons let me know. Comment, subscribe for more content in and around this area. And yeah, Drew, once again, thanks for joining us. And thanks, everyone, for watching.

Drew: Thanks so much for having me, James. I really appreciate it. Take care, everybody.