Recorded live at Microsoft Build 2025, this RedMonk Conversation features James Governor speaking with Dr. Nicole Forsgren and Gene Kim about the future of DevOps in the era of AI agents and vibe coding. They explore how generative AI is transforming software development, the evolving role of developers, and the importance of maintaining strong processes, culture, and developer experience as code creation accelerates. The conversation also previews their upcoming books on vibe coding and developer experience.

Rather listen to this conversation as a podcast?

 

Transcript

James Governor: Hi, this is James Governor, co-founder of RedMonk, and we’re here for a RedMonk Conversation. And the subject of the day is agentic DevOps.

And the subjects of the interview, these amazing luminaries. And honestly, I mean, we have some amazing, amazing people on the show. We get great opportunity to talk to people that have made a huge impact.

But I think today, it is an embarrassment of riches. I would say that my two guests today need no introduction.

And I’m quite interested to see what their introductions are. And I may have to add some to it because we’ve got an amazing axe thrower in Dr. Nicole Forsgren. And a fantastic author and many other things in the shape of Gene Kim.

Gene Kim: Nicole, why don’t you go first? 

James: So, Nicole. 

Dr. Nicole Forsgren: So, I am Nicole Forsgren. I occasionally do axe throwing, as James pointed out. But I tend to work on research and strategy to help teams and organizations, including Microsoft, figure out how they can improve software outcomes.

Right? Can we be faster? Can we be more stable? Can we be more secure? How can we improve developer experience? 

James: Okay. 

Gene: And I’m Gene Kim. And I’ve been studying high-performing technology organizations for 25 years.

I got to work with Nicole for a decade on the state of DevOps research, which was one of the things I’m most professionally proud of in my career. And these days, I’m a Vibe coder, James.

James: There you go. Well, we’ll talk about that in a minute. Because DevOps and Vibe coding, we’ll see how well they go together. 

Gene: Better than you would think. Because no one should be writing code by hand anymore. 

Okay. Okay.

So, here’s the thing. And seriously, so, if we think about people that literally have written the book in and around DevOps, here we are. And the DevOps research assessment looks like Accelerate.

Yeah. No, there’s just so much track record, it’s ridiculous. So, what is agentic DevOps? 

Gene: Well, I’ll start with agentic DevOps. Yeah, let’s say, I would say it is a significant part of the way we all code now. Because no one should have to type in code by hand anymore. And the person who actually coined that was Dr. Eric Meyer.

So famous in this community for his work on Visual Basic, C Sharp. He went on to develop the hack language at Facebook Meta. And he said, yeah, we are probably the last generation of developers who will write code by hand.

And I just thought that just so spoke to me. And, yeah, so agentic is when you can actually show the LLM agent the output of his work and it can fix it for you.

And, you know, from my perspective, I mean, it’s just utterly transformative. I have an experience where I spent 45 minutes being bossed around by an LLM telling me to type this, type that. And, you know, the aha moment was like asking it to run curl by itself.

Right? And 45 seconds later, it fixed the issue with the Trello API that I was struggling with in 45 seconds. The same thing with Google Docs. I mean, so it’s just like once you see something like that happening, right, it just becomes so obvious, you know, that there’s some things that just shouldn’t be in the loop.

And because the only thing you are now the bottleneck, copying and pasting from one window to another. So, and it’s not just for developers, it’s for everybody. Technologists, for operations, infrastructure, DevOps, all of that. So, I don’t think we want to say that the only people who benefit from this are just developers because as someone famous said, who broke my build is going to be said even more often.

Now that you have, you know, code generation that’s not measured in hundreds of lines per day. But, you know, in the case of Steve Yegge, you know, my co-author on the Vibe Coding Book, he’s doing 13,000 lines of production, high quality code commits per day.

And, you know, to get there, he’s doing 100,000 lines of review. Anyway, so it’s just a wild world we live in. It’s an amazing time to be a developer and in DevOps and in technology. 

James: Okay. So, Nicole, do you have a view on this next frontier for DevOps?

Nicole: I do. I have a slightly different take. Oh, probably just an expansion, although I’m not sure if people aren’t going to be writing code by hand anymore. TV day. I, you know, we’ve really spent a lot of time focusing on writing code and how AI and LLMs and agents, agents can kind of help us bootstrap and amplify and accelerate that.

And I think there’s also a ton of opportunity, not just in the interloop, but all the way through the outer loop. Absolutely. What do we think about, you know, the opportunity for AI?

And especially now we have agents to improve local test and build, to improve our PR and code review process, to improve build and integration, to improve things like release. So, for me, I think that’s, you know, when I think about agentic DevOps, that’s kind of what it is.

Is we know the foundational principles are still really important, right? Especially now that the speed and volume of, you know, creation has sped up so much.

Now the rest of that software development loop is even more important. 

James: I think so. I mean, you know, putting code into production, maintaining it over time, making changes to that code, ensuring that the user experience is good.

Yeah. It’s the initiation of the code that you write is the easy bit. And I think we’re generative. We’re definitely in a world where, yeah, writing code is not the bottleneck.

But we are going to have to be rolling it out in production, managing it. And I guess that’s where DevOps comes in. 

Gene: Yeah. In fact, you know, we were talking about yesterday, you know, a lot of people, when the feedback of the book, when we gave it to early reviewers, was like, man, how can you two supposedly smart people, like, be so stupid?

Right? You know, how can you not know about version control and automated testing? And, you know, in our defense, it’s like, well, what worked at, you know, 100 lines of code a day is wholly insufficient when you’re doing, you know, 10 to 100 times more volume of code.

And so, you know, I think it’s kind of our excuse that we’re going to put in the book, that it was a really great comment. It’s like, what worked for us? 

James: So, by the way, what exactly is this book to which you keep referring? 

Gene: Oh, sorry. 

James: The new book. What is this book? 

Gene: It’s called Vibe Coding. 

James: Vibe Coding. Right. Okay. 

Gene: Building production-grade software. You know, that’s, you know, faster and more ambitious and with more autonomy, fun, optionality.

And, but yeah, I mean, we tell some of the war stories and disasters and surprises that we had. And, you know, I think it’s really kind of like at the frontiers of flight. When we first, you know, had powered flight, you know, there were just a lot of new practices that had to be built.

And I think we’re so lucky in the DevOps space. It’s like, it’s actually the same practices, but you need it like so much more often. You know, you’re going to be committing code probably four to 10 times more often. Right. Because if you’re not saving your game, you’re just going to have like a really bad time.

James: So I’d like to talk a bit about that. One of the concerns I have with this, this Vibe Coding notion is that people will get a little bit too carried away. And we’ll begin to jettison some of the disciplines that we’ve, that have been so hard fought, basically, in terms of decomposing things into smaller pieces, into making smaller changes, into testing things, into actually fully understanding the code that we deploy.

And I think sort of, for me, a big concern is how do we maintain in this new era, those practices of good software engineering that DevOps brought.

And sort of what I hear from you is a little bit worrying. You’re just going to, you know, if it’s all vibes, you know, maybe we’re just huffing on something and we’re going to run into trouble when our customers have an issue, for example.

Gene: And I think some of those fears are genuinely warranted. In fact, over the last many weeks, we’ve been talking about, you know, the door anomaly, right? You know, they found that the more AI you use, it’s worse for throughput and worse for stability. And, you know, I think some of our experiences can, you know, can say, yeah, there’s maybe something to do that.

However, like the really good news, and we were just talking about this earlier this morning, is that it’s the same practices, but you just actually need to be doing a lot more of it. When you have a technology that, you know, is very sensitive to its initial state, and when you can go off the rails quickly, you need a lot of guardrails, and the guardrails, you need more of them.

And so the good news is, we don’t have to invent a lot of new practices, right? They just have to be doing it much more often. So, inner, middle, outer loop. 

Nicole: Yeah. And I, you know, I think, you know, you mentioned the door anomaly.

I think, if anything, that just kind of supports so much of the research that we had found, right? Is that there can be things that happen early in the software development tool chain. And if we don’t have the important technology and process and culture and architecture building blocks, then we do see reduced outcomes later, right?

James: So, by the way, not everybody watching will know about this particular anomaly. So, why don’t you describe the change that you saw in the research, or the change we’ve seen in research on what this anomaly is that you’re describing?

Nicole: So, this is led, the research now is led by the team at Google. And so, I, we weren’t part of the research, but we’ve seen the outcomes, we’ve seen some of the write-ups.

And what they found was increased use of AI was associated with poorer outcomes in terms of stability. But, again, I think this is where it’s important to think about, you know, when we first started talking about DevOps back in the day, we have a handful of building blocks that are important and are actually predictive of speed and stability.

And these building blocks are things like automated testing, CI/CD, good cloud architecture. And so, I don’t think it’s necessarily that as you use more AI, your reliability goes down.

I think what it actually is, is as we’re creating exponentially more code and artifacts through AI and LLMs, we have not yet built up or augmented the rest of those building blocks that are super important, like automated testing that is specific and, you know, attuned to this type of code.

James: Because that was the paradox, I think. One of the paradoxes of DORA was that the fact that organizations were moving faster were actually breaking things less because they had all of those disciplines in place. 

Nicole: If, when they have those disciplines in place.

James: When they have those disciplines in place. 

Nicole: When those disciplines are not in place. 

James: Absolutely. 

Nicole: Then they do see, you know, this mix of outcomes. 

James: Okay. So, software engineering… 

Gene: If I can plant a seed, right?

I think, let’s just say in a couple of months, we’re going to have an opportunity to release some findings that actually say, all right, when you have some additional practices in place, right, you can get the benefit that we have, you know, many of us experienced, right?

Which is like, you know, a many-fold increase in productivity, right? And you can sort of resolve this anomaly, right? And I love that anomalies are often the basis of major scientific breakthroughs, like the precession of Mercury’s orbit in 1911, right?

So, yeah, I think when we resolve the anomaly, I think it will, again, confirm deeply held intuitions, kind of confirm, it will reveal surprising insights. 

James: Well, we’ll see. Confirmation bias is also the sweetest bias.

Nicole: Yeah. It is. Very true. 

James: So, yeah. 

Nicole: …And spurious correlations are the worst correlations. 

James: Absolutely. If we think about where organizations are in this, because, you know, if you’re a bank or a telco or regulated industry, vibe coding may be a little bit scary.

But certainly the idea that you’re going to be using AI and perhaps agent technology in your DevOps, that makes sense. So, on this journey that we’re embarking on, will there be a sort of, if we think about, what are the first steps customers should adopt?

I mean, you have already, you know, in your sort of career done an amazing job of saying, these are the laggards, these are the elite performers. What are elite performers going to look like?

Are those best practices you think already emerging at the research that you do? 

Nicole: Yeah. I think there are a few things there, right? One, you called out earlier, which is user experience, right? So, when we think about the developer experience, I think that’s going to be increasingly important.

Again, because now we’re creating so much more code, so many more different types of artifacts so quickly. I’m working on a book right now with Abi Noda about developer experience and how it’s so important. And so, when we think about, you know, what this means for the highest performers moving forward, they are taking these important and, right now, exploratory steps into figuring out what do these refined and augmented and new infrastructures look like, right?

What does the support need to look like? How can we make sure, and I think this is where I, you know, maybe have a slightly different vision or view. Maybe you’re seeing far ahead of me, but I think, you know, coders will, we will still be creating hand by code.

We will still be reviewing hand by code. And if the rest of our supporting systems, if our build and integration, if our CI/CD, if our monitoring and feedback loops can’t keep up with what we’re doing, that’s going to be the bottleneck.

It really will be supporting that developer experience. And even, like you said, like these mental models, right? How can we reduce cognitive load? How can we improve flow? 

James: Yeah, and I mean, I think from my perspective, one of the things there is, what do the platforms look like?

What do the tools look like? What do the processes look like? What does the culture look like in order to enable all of this? There is an awful lot to work out. We’re certainly not at a clear maturity model yet.

I mean, I think the fact is that the DevOps, here we are so many years into the journey, and a lot of organizations for large swaths of their IT, that’s still an aspirational state. But I think there, so we think about somewhere between your position and Gene’s.

Like, what should organizations be thinking about right now about, so Gene, you’re very confident about vibe coding. What sorts of tools or approaches from an AI perspective should organizations be adopting so that they can get faster?

Gene: Oh, yeah. In fact, I think it’s no surprise that the people who are heads of dev platforms are the people who are leading the charge. And I think there’s kind of two classic places where organizations choose to be.

I think one is when all the hard problems are known, and we standardize. And that’s because we all understand where the idiosyncratic behaviors are, et cetera.

And then the other extreme, like, no one knows. Right? And there you want, everyone’s exploring. And that’s where we allow teams to make their own decisions. Right? And, like, you want to share successes, share lessons learned.

Right? So to elevate the state of the practice. And one of my favorite stories was from Comcast, 8,000 developers. This is Jonathan Moore from Chief Software Officer and Mike Winslow.

And he said, you know what? We want every engineer to be experimenting and innovating, except for a CI/CD. Like, we don’t need 14 CI pipelines. We want two. Maybe one. Right?

And so, you know, we have only so much innovation cycles. Right? You know, we’re going to sort of standardize. First, I think the people who are seeing the most amount of data, seeing what’s working, what’s not working, are the dev platform groups.

Bruno Passos is head of dev productivity at Booking.com, 3,000 developers. And, you know, he said, he found that one of the biggest things that leadership can do is actually train development teams. You know, that’s often a big unlock.

Just because they’re so non-deterministic and, you know, unpredictable without, like, knowing how the system works. And, you know, they’re seeing 70% reductions in code diff sizes, 30% reduction in merge request times.

You know, findings are being repeated at Adidas. So, yeah, just it’s no surprise to me that it’s the dev productivity folks that are leading the charge because they have the most number of feelers out about, you know, what’s working, what’s not working.

James: Yeah, as I say, the integrated platform for me, I think, is so important. Because otherwise, we end up giving more and more overheads, organizational overheads, cognitive overheads, in terms of the tools that you…

I think it’s very hard to drive a platform change at scale if you don’t… To drive a culture change at scale if you don’t have a platform that enables that. I guess from that perspective, yeah, I mean, there are some interesting things.

The collision now of observability with release management and feature flags. So, we’re seeing a lot of consolidation in the industry. This is great.

I mean, I call this stuff progressive delivery. Apparently, like you two, I also have a book which is being published in November. 

Gene: Congratulations. 

James: Thank you so much. I mean, for us, it was quite interesting because progressive delivery is all about, look, there are a set of disciplines.

If we think about everything that we’ve done, some of the CI/CD, in terms of Agile and everything else, but a lot of these things were done before the cloud even existed. So, we just didn’t have the abundance. If you’re going to do something like blue-green deployments, you literally needed to build out two entirely sets of infrastructures.

You need to pay for all of that. You needed to manage all of that. The idea that you could try five of something, you just couldn’t. And so, to bring software engineering into a more… To bring experimentation in, then, of course, you need to be able to observe it.

You need to observe it as a dark launch. So, from my perspective, the AI thing is really interesting because of the fire at lights under progressive delivery. Because you’re like, well, absolutely.

We’re going to have to be able to do a rollback. We’re going to have to… If we can have 15 different flowers, we want to observe how our users are able to enjoy those flowers.

Or perhaps they don’t like them. If it’s fruit, maybe it’s a sour fruit. So, I think the integrated platform story, for me, is just going to be super important. 

Nicole: I mean, one way to think of it is how…

What’s the usefulness of all of this experimentation, all of this innovation, if we can’t see what’s working and what’s not working? 

Gene: In fact, I want to share… When we were talking some weeks ago, I shared the most mind-blowing equation that I…

I mean, it just changed the way I view the world. The importance of those experiments, as told by Dr. Carlos Baldwin, who was the foremost pioneer of modularity, was a student of Dr. Bob Merton, who invented…

We got the Nobel Prize for discovering option pricing with Black and Scholes. NK divided by T and sigma. N is the number of modules that you have.

K is how many experiments can you run in parallel for each one of those modules. T is how fast can you perform an experiment. And then sigma is kind of like a risk-reward payoff uncertainty. And so, you know, the reason why experiments are…

You want lots of modules and lots of parallel experiments is that you massively increase the option value of the system. And so you want to make those options plentiful. You want to make them cheap to perform.

And… 

Nicole: And fast to perform. 

Gene: And fast to perform. Absolutely right. And so, like, AI makes, you know, increases in K, the number of experiments we can do. It reduces the time to perform the experiment.

You know, so that’s why our job as a socio-technical maestro is to make sure that we have a great modular system. Fast feedback loops, right? Which also reduce T. And so when we are living in situations that have high sigma…

If sigma is one… 

James: Oh my God, I have three kids. Please do not use the word sigma. 

Gene: If sigma is one, we know all the answers, right? So there’s actually no value in experimentation because we know all the answers. We just picked the right answer. But when, you know, there’s high uncertainty and the payoff for making the right bet is high, then, you know, there’s so much value.

I mean, it is like exponential. Like how much value, at least quadratic, right? How much value that would create. I mean, so, like, what you’re saying is just so important. 

James: Things that are important.

You sort of very casually are saying, you know, this is the end of hard-coded software. A lot of people watching this might actually be software developers. It’s kind of a scary thing to say.

I mean, part of DevOps, psychological safety, making people feel comfortable and confident so they can do their best work. So they’re not threatened by the arrival of new technology. So they’re not threatened by competition in that sense and making mistakes.

I’m quite interested in some of the human factors. So, Nicole, a lot of the work you’ve done is actually more recently, has focused on developer experience and perhaps the concerns that developers have.

I’d like to talk a bit about that, changing roles. Because it’s one thing, you know, if you hear there’s no more hand coding, then that sounds like there’s no more jobs. Are there still going to be jobs as we have generative AI and agents helping us with DevOps?

Nicole: I mean, the short answer is absolutely, right? There’s been some consternation lately saying, like, are junior developers no longer necessary? I think that developers are going to be more important and more necessary.

Because as we continue to write code, do experimentation, we’ll need more and more people to do that. Because we’re also seeing senior engineers benefit from these technologies, right?

It’s improving their ability to understand the system and design the system and think about architecture. Well, that means that we need a lot more junior engineers to help build and create.

James: I think people that say we don’t need junior engineers have not met any junior engineers.

Because the juniors I meet are phenomenal. Yeah, the range of skills they have, they grew up with a lot of this stuff. For them, the shifting left already happened. They grew up GitHub natives.

They absolutely understand pull requests. They understand this way of writing software. And yeah, the idea that we don’t need juniors anymore, I think, is not helpful.

Nicole: Now, I mean, skill sets are going to change a little bit, right? Like, we’re already seeing that some of the methods and the processes that people use to write code and build and test and deploy are being impacted by LLMs, right?

There’s prompt engineering. There’s deciding which RAG model you’re going to use. So, I think the skilling will increase, right? Or the types of skills that you need are going to grow.

But I think we’re also going to see an entire new category of software development specialists emerge, right? Because we’re dealing with things like data handling, data versioning, model versioning, model selection.

Right now, I mean, it feels like the earliest days of DevOps, right? Because so many teams are building as they go because we’re still discovering. But once we figure out that standardization of kind of the best practice, and some of them are emerging, we need people who are specialists in this to build this for us and to help us deploy these systems and test these systems and deal with the reliability and the safety and security and the validity of the output of these systems.

I think we’re going to see job creation. 

Gene: I think so. 

James: You think we’re going to see job creation. 

Nicole: I think so. 

Gene: Oh, yeah. I think the words that’s bandied about a lot is the notion of Jevon’s paradox, right? It’s like when you reduce the cost of software creation by 10x, right?

You’re going to get 10 times more software jobs because there’s 10 times as much software that can be created. 

James: Anytime you have more efficiency, you do more work. 

Gene: Right. And it’s interesting. We were talking about this yesterday. In the early DevOps days, we were like, okay, what happens to all the network engineering jobs and the database administrators and blah, blah, blah, right?

There are more databases, more networks, more VLANs, more cloud engineers. So, I think it is very easy to get emotionally kind of wrapped around the axle about what does this do to my profession.

And I think our message would be it’s not as hard as you think. It’s more fun than you can imagine. And it’s like imagine life before, you know, Google or Stack Overflow, right?

This is really the next generation of like how we program. And I just, nothing makes me happier to see how like Martin Woodward, you know, VP of DevRel at GitHub, describe how fun it is. Our friend Andrew Flick, he said, I was a programmer.

I was an MVP. I was a WinForm developer. And he was proudly showing off his Git commit history in the last 30 days, right? Full of green, right? As he’s working on the mean stack, he said, I was relegated to the Pew stack, PowerPoint, Excel, and Word.

Look at me now, right? This big grin on his face. And so, I think, you know, there is this element where, I mean, yes, it is like a slot machine. And like a slot machine, it’s like an intermittent reward, which is actually the most addictive, you know, kind of reward cycle there is.

And makes you wildly more productive. You know, you get to have far more fun doing it. You get to do things that would have otherwise been out of reach. I mean, I have to say it is a wholly rewarding thing. And it’s just been so fun to collect so many stories of people who are doing things that never, they never thought possible.

And I think there are some caveats is that as an early adopter, you may be yelled at by open source maintainers by saying, oh, you didn’t disclose that you used an LLM, right? And do things like wipe out your contribution.

You know, they’ll take your commit, but wipe out your name. 

James: To be fair, we spend a lot of time shouting at open source maintainers. And they’re making the world go round. So, you know, a little bit of pushback, I think, is perhaps reasonable. 

Gene: So, it’s amazing this new world we’re entering where these norms are just not known yet.

James: Okay. So, you’re extremely positive about vibe coding and the future. I am. Nicole is talking about the importance of process. 

Nicole: I would say process and tools and culture, right?

Because the things that have been kind of annoying blockers to date will be just deal breakers, right? We have to remove friction. We have to improve trust and transparency. We need to help developers to be able to do these new exciting things better and faster, right?

It’s about… 

Gene: And safely. 

Nicole: And safely. Yeah, absolutely, right? And so, you know, it’s about removing that friction. 

James: Okay. Well, I think we’re going to wrap it up. But I guess my final question would just be, if there was one skill that you think is going to be valuable in the next few years, what is the one skill that people should be learning, adding, whether they’re a junior, whether they’re a senior, what’s the one skill that people should be learning so that they can thrive in this era of agents and DevOps?

Gene: I love that question. I’m a huge fan of Dr. Anders Ericsson, who studied acquired learning. And, you know, basically, he studied how people learn to learn. And he said there’s kind of three critical ingredients.

One is you need to hang out with someone who’s actually better at it than you. Like, have a coach. Two is you need fast feedback, right? Whether you’re learning to play a musical instrument, learning to play a professional sport, like, you need very fast feedback.

And then you have to work on things that are just slightly out of your reach, right? So things that are slightly out of your ability. And those three things, right? I had the ability to do that with Steve Yegge in August, and it changed my life. I wrote something in 47 minutes that would have taken me days if I had ever bothered to start.

And so I think finding someone who can be that person for you, I would find that person and ask them to help you achieve, you know, get better at this thing that we’re calling Vibe Coding.

James: Love that. 

Nicole: Mine would probably be communication, right? We, as things move faster and faster, and as we get an increased ability to innovate, we need to understand and be able to communicate what we’re trying, what our experiments are, why we’re trying them.

And, you know, even related to that, can we write better specs? Can we write better diagrams? Can we write better design docs so that we can have the models help us do what we want? 

James: Okay. So there we go.

That’s the conversation for today. I mean, there are a couple of things there. I think the book’s going to be a lot of fun. Certainly, Steve Yegge, he wrote, he’s famous, well, famous for his engineering, but he has written a couple of rants about the cultures at Google and Amazon that are extremely interesting in terms of where the bodies are buried.

You might want to check those out. So, yes, I look forward to that. Certainly, any work that Nicole does, always worth following. So the next book, that also sounds…

Nicole: Thank you. Frictionless. And Abi Noda is another expert in the space, so I’m really excited to have that come out. We’re getting some really fantastic notes from reviewers. 

James: Developer experience, so important in all of this.

And, I mean, you know, I think Microsoft continues to invest heavily in developer experience. I think that’s why they are where they are today. That’s kept them going. That’s going to be super important.

And, yeah, that is another RedMonk Conversation. Thanks for joining us.