In this MonkCast with the Monks episode, RedMonk analysts Kate Holterhoff, Rachel Stephens, and James Governor discuss the current ‘grumpy era’ of AI, where developers are questioning the ROI of AI tools and their effectiveness. They check in on AI code assistants, vibe coding, agentic IDEs, AI spec-driven development, and the AI Engineer.

Transcript

Kate Holterhoff (00:12)
Hello and welcome to this MonkCast with the Monks. My name is Kate Holterhoff, Senior Analyst with RedMonk, and with me today I have Rachel Stephens, Research Director at RedMonk and James Governor Co-founder of RedMonk. Rachel and James, thanks so much for stopping by to chat with me today a little bit about AI.

Rachel Stephens (00:27)
So delighted to be here!

James Governor (00:27)
A little bit about AI?

Kate Holterhoff (00:31)
What about AI? These are the questions that I feel like we have been dwelling on in particular. So We have published several blog posts on the subject. I wanted to invite my colleagues here to discuss what we’re seeing. So James, I’m gonna ask you to start, kick us off here. What would you say are some of the most important trends that you’re following around AI code assistants and the AI space more generally today?

James Governor (00:54)
I think we have entered the grumpy era of AI and codersystems. And some of that grumpiness is the people that have always hated AI tooling, the people that are on Bluesky. Now, those folks do not like AI. But then even the people that really like AI are a little bit like,

Kate Holterhoff (00:58)
Hahaha

James Governor (01:21)
And then the people were like, we’re going to give it a try. They’re now like three or four months in, even people were like, I’m not. And now they’re beginning to be like, hmm, why did it hallucinate that library? Why doesn’t it just do what I want? Why am I spending so much time having to clean up? And it’s not even cleaning up after myself. It’s cleaning up after the AI. You know, I think we talked about like sort of AI as a, an intern. Well, it’s not like an intern, it’s like a kid.

It’s leaving its trash everywhere. So messy. It’s unbelievable. So we’ve entered the grumpy era of AI with these agents, these little agents running around. They’re like kids. They’re throwing their clothes all around the room and, you know, just generally making a mess. So, yeah, I mean, the good news is that clearly there are some possibilities to break through productivity.

It can be great to find a bug, you’re looking through some horrific YAML file, it will do that for you. Definitely can make developers more productive in their daily life, but the whole sort of vibe coding, YOLO, know, just hit tab, I think we’re beginning to ask some questions about that. So I think the, yeah, the grumpy era of AI is probably where we’re at. mean, there’s a lot of boosterism still to happen. I think it’s interesting, you know, you got…

Open AI is like, they’re to be a hardware company, they’re going to be an application company, and they’re going to be a human, what are they going to be? They’re going to be like a job board company, a new LinkedIn. That’s Max of a severe lack of focus to me. But yeah, so you see, I’m grumpy too. And that’s, think, where we’re at, but lots of new tools. Development continues apace. And we’re just trying to, I mean, for me,

Rachel Stephens (02:50)
Like a job board company.

James Governor (03:14)
If I had a sort of investment thesis, or my research thesis at the moment, I think is how do we take non-deterministic things and actually make them a little bit more deterministic? So how do we take these things that are generating all sorts of stuff that might hallucinate anything else and actually make it something that we can rely on? And that’s what I’m interested in seeing from here on in.

Kate Holterhoff (03:38)
All right, Rachel, I would love to hear your two cents now. Would you agree we are in the grumpy era?

Rachel Stephens (03:42)
Yeah. would also say we’re in like a little bit of an ROI questioning era. So I feel like we have a lot of companies that kind of dove in in a FOMO style saying like, have to have an AI strategy. And all of a sudden, every single tool in their stack has started to incorporate AI in different ways. And I think it’s not entirely clear, like.

where we should be making these investments, where the investments are actually going to be having an outsized impact and a return on investment. And I think also just the token cost and everything just feels deliberately obfuscated right now. And if you look at just like, even just trying to parse the valuation of these AI companies themselves, like it’s it’s a little bit, it’s like, fuck.

James Governor (04:30)
tokens, tokens, just R &D, it’s just R &D

don’t worry about that, don’t worry about that and cost a good salt

Rachel Stephens (04:36)
So I, it’s fine.

Kate Holterhoff (04:36)
You

Rachel Stephens (04:39)
So I think a lot of companies are.

If I’m going to be making this level of investment and it’s looking like it’s going to be a sizable investment, especially if we think about like VC money going away at some point, like if I’m going to be moving into this world and this method of working, how do I do it effectively and efficiently? And so I think people are starting to ask those questions and the answers that we’re getting are very unsatisfactory. Like if anyone can talk about AI economics satisfactorily, they can already retire because it’s everybody wants that answer, everyone will pay for that answer.

James Governor (05:15)
Well, I love that recent essay by, Steve Yegge, amazing communicator, amazing technologist, you know, sort of history, at AWS, at Google, now at Sourcegraph.

And he’s talking about the sorts of economics that we might expect. And I think it was a little bit in the spirit of sort of a modest proposal, like, wait, are you serious? But he was sort of saying that to be, if you were really serious about software development, as we began to add all of this up, that you should be budgeting for $50,000 per year, per engineer, on AI tooling. I mean, I understood how it got to the math.

And there are certainly people out there bragging about how they’re spending, you know, $200 a day. Look, I’m such a great engineer and spending a lot of money. It’s quite performative. But I don’t know many enterprises that are…

I mean, you Rachel talked about ROI and with a finance background, Rachel, how many folks do you all know that work in finance that like hearing about an extra 50 K per employee for tooling? that, is that a fun conversation with finance?

Rachel Stephens (06:31)
So I think one of the fundamental disconnects is that the way we’re selling AI is that it’s enabling all of these new work streams, new value streams, things that we couldn’t do before are coming together. Great.

The fact of the matter is for most people, their IT slash engineering organizations are cost centers, not profit centers. And so if you’re going to be adding in all of this additional cost, there’s gotta be something that’s offsetting some of our costs coming back out. Are we able to like actually generate new revenue streams from what we’re building? And I think that there’s a lot of organizations that are not prepared to have, it’s a really fundamental shift in your business to say that what I’m building is something that is now generating profit. Like that’s a really big change. So I think a lot of companies are now in the place of like what am I actually investing in and what is this going to do?

Kate Holterhoff (07:25)
Yeah, all of my podcasts were full of episodes about the AI bubble. And then when I saw what the actual developers were saying about these articles, which seemed a little bit inflammatory on purpose, right? They were saying, well, I’m still using these tools, right? They’re still important to my workflow. And I think those are the stories that I’m more interested in rather than trying to sort through the froth, if that’s a thing

Rachel Stephens (07:49)
Yeah, I think that’s the other disconnect is that I think that being individually productive and being organizationally productive are not always the same thing. And so if I, as a developer, can do XYZ task faster, does that actually push anything through my pipeline any faster? Like, can the rest of the organization keep up with it? Am I working on, like, once I go faster on one thing, is there more valuable work waiting for me in the queue later? Like, all of these, I think, are open questions.

Yes, we want people to be productive and I very much believe that there is developer productivity gains to be had from these tools, but I think there’s also just systemic questions around how we should be incorporating these tools.

James Governor (08:32)
I mean, it’s not my job to, you know, tidy up my kids’ bedroom and I try and make it their job, but I seem to spend an awful lot of my time picking stuff up. you know, and that’s, yeah, like I wish I could get paid for that time. And I wish I could do that with tokens. Cause let me tell you, it’s a big mess out there. I think we do need some help cleaning things up, disciplining the children.

We need some carrots and sticks, big sticks. Have I taken this too far, As a mother of two young boys, I mean, you know what I’m talking about. Autonomy is great, except…

Rachel Stephens (09:07)
I love a tortured metaphor.

Kate Holterhoff (09:08)
No, I love a tortured metaphor, the just keep going, just, yeah.

Yes, I do indeed.

Rachel Stephens (09:19)
I’m

This is fair. Kate, want to, like, so, like, we kind of talked about the grumpiness side of the James intro. Let’s talk about your hot vibe code summer piece, though. Like, because, like, there are some sunshine and rainbows in here. Although I believe your direct quote was, it’s not all sunshine and Megan Thee Stallion hits, tell me the both sides that you’re seeing.

Kate Holterhoff (09:32)
Absolutely.

Yes, I know and I think.

Well, I think it’s just important to give voice to this particular view because I am in no way a booster. I am not ignoring all the dark aspects of this. But I think what folks like Simon Willison have pointed out is that there is a sort of fundamental disconnect right now about

James Governor (10:07)
Wait, wait, wait a second, Kate. I’m just adding you to the AI Apologist block list on Bluesky. Okay? It’s now 186,001.

Rachel Stephens (10:11)
Yeah

Kate Holterhoff (10:13)
Mea culpa. It’s me.

So what I’m hearing is that, developers are using these as part of their workflow. We are seeing these tools move left in terms of spec-driven development trends. We’ve talked a lot about Kiro already at RedMonk. But I think what’s exciting about vibe coding is that it gets to that tinkerer spirit that really drew me.

to this entire domain, to working at RedMonk, to talking with developers. And that’s where I’m seeing, that energy flow right now. And so, yeah, we can talk about AI Winters and the grumpy folks and the fact that there’s a lot of shenanigans going on around VC spend and the Windsurf acquisition. mean, all of these things that have very complex stories. And I don’t know that we’re going to see beyond

what the press releases tell us. But what vibe coding does is shows the potential of large language models to enable a whole new group of developers to participate in coding. And so, it’s worth talking about that. it’s worth, I focus.

Rachel Stephens (11:23)
Yeah, there’s no disagreement

there for me. I think you’re absolutely correct there.

Kate Holterhoff (11:28)
Yeah, I focused on designers a fair bit. I mean, because I’ve seen that in practice, how designers have in the past been stopped from actually, deploying projects. And now they have that ability.

Rachel Stephens (11:38)
I think you’re absolutely right that it does enable it. Like when I’m saying it enables new workflows, like I mean that legitimately and I have seen this myself, like it has enabled workflows for me and like I would never, I would, I would never want to.

denigrate somebody who’s, especially someone who is now able to do things that they weren’t able to do before. Like that’s huge and that’s powerful. And it does feel a little bit like gatekeeping to say like, you’re not.

coding the right way, you’re not learning the right way, you’re not entering this industry the right way. And so I don’t want to be that way at all. I think the question is really just how are organizations set up to accommodate all of this? I think that’s where the disconnect for me is.

Kate Holterhoff (12:21)
Right.

Yeah, yeah, I agree. And there’s a book coming Addy Osmani’s Beyond Vibe Coding, because I think that’s kind of where we’re at. yeah, in the post I just published, I include many, many links to folks calling vibe coding as a word cringe. I mean, even Andrej Karpathy has moved beyond it. So nobody likes the term, which I think is kind of hilarious, so I’m going to lean into it extra hard.

in order to torture my own metaphors. But I think that the essence, the energy, the tinker spirit that it accesses is worth talking about because that, I think, is going to be the sticky part of this. That’s what is going to not be going anywhere as we move into new seasons and new ways of characterizing the type of work that we’re doing.

James Governor (13:08)
Well, I mean, as somebody that is called it the grumpy era of generative AI, I got to admit, look, I mean, here’s the thing. So there was a conversation that Anil Dash was having, again, mentioning Bluesky. And he was talking about the fact that he was on this block list. there are people that are literally like, no, AI has no value.

It does nothing and it can never make anyone more productive. And of course, as Kate, you sort of half mentioned, you know, like the things environmental theft of people’s IP, you know, not being a shit about people having jobs, all of these sorts of things, very real concerns. But the thing about that conversation that really strikes me, the thing about vibe coding is that it is fun.

There’s so much dopamine involved. why did Midjourney explode? Because people enjoyed creating images that they felt in some way it was reflected what they imagined, and they found it fun.

Rachel Stephens (14:26)
Can I pause this really quickly to tell you that last night my daughter recreated K-pop Demon Hunters as capybaras using AI It’s fun to make ridiculous images.

James Governor (14:38)
So, you know,

Kate Holterhoff (14:39)
Yes.

James Governor (14:41)
the Capybara revolution is in full swing. And, you know, I think that I generally don’t bet against stuff that people enjoy. so, yeah, judging, you like, this thing that people, things that have fun, yeah, that could definitely lead to a revolution in behaviors.

and there’s so much, so much of what we do in terms of like, don’t do this, don’t do that of course they’re gonna do it so yeah, I mean think that whether or not vibe coding is sort bad and cringe and I mean as a father of three I know a lot about cringe so, but your kids are not calling you cringe yet, Kate

Kate Holterhoff (15:25)
You and me both.

Not yet, not yet, but I’m getting ready, like mentally preparing myself.

James Governor (15:34)
They’ll have a new word, it won’t be great. Ah yeah, there you go. So yeah, that’s the thing I think that a lot of people, the sort of the total naysayers, you’re betting against fun. And sorry, that is absolutely the, I mean, you we talk a lot about, of Luddites and so on, but I think it’s a bit…

Rachel Stephens (15:35)
don’t know what you’re talking about, I’m so cool.

James Governor (16:00)
The proof point here is it’s King Canute. You’re standing there and the argument is actually he knew the tide wasn’t coming. He didn’t think the tide was going to in. The tide has come in. The feet are wet. The fun has happened. It’s too late. People are to use these tools. so I think that’s, overall, that was what really I noticed in conversations with developers and using the tools. But just seeing, just people having…

The number of people that are software developers of years that have always loved the craft and educated and all this and they’re still saying, actually, this is the most fun I’ve had in years. And so, yeah, maybe it’s not the grumpy era, but I said it at the beginning, I’m sticking with it. Maybe it’s the grumpy fun era.

Kate Holterhoff (16:49)
The grumpy fun era. I think it comes down to RedMonk’s thesis. I mean, right? It’s developer led. The developers are gonna do what they think is, I don’t know, yeah, the best, the tools that make the most sense to them. And a lot of them, a lot of them have rallied around not just becoming AI engineers, but actually vibe coding. Like, yeah, mean, Karpathy was the one who coined it, you couldn’t ask for somebody who’s a more seasoned developer to be not even checking their code, right? To just be, letting the AI steer the ship.

Do it.

James Governor (17:19)
rant on a little bit. you mentioned, you know, as we talk about vibe coding and of course the new one now is context engineering. But what is, you mentioned specs and you mentioned Kiro. What are you on about? What is specs? How does specs, how are they gonna retain the fun but make us less grumpy?

Kate Holterhoff (17:26)
yeah. I’m gonna let Rachel take this one.

James Governor (17:39)
Or obliquely.

Rachel Stephens (17:40)
Now, now I’m gonna uncool myself, because I’m gonna tell you about how much I love Markdown. So man, I’m gonna have to backtrack on my cool statement like three minutes later. I love Markdown. I think Markdown is excellent.

Kate Holterhoff (17:45)
Yes.

Rachel Stephens (17:53)
for me in terms of how I organize my thoughts consistently. It’s just the way my brain works. And so go in and instead of trying to prompt my way through something, I can sit down and organize a document of this is what I’m trying to do and then have the AI actually give me feedback on that document, work from that document and then from their build based off of those requirements and designs. For me, it worked really well, but…

The caveat is that I am like by no means a real like I do this for hobbyist stuff or for like creating charts and graphs. It’s like none of my code ever goes to production. Like this is not, this is not something that I have the same concerns and that’s how I feel about vibe coding too. like I can enjoy it because I have absolutely none of the risk and downside. And so some of the teams that are trying to do this in a more professional sense, like probably have different feelings, but for me it really works.

Kate Holterhoff (18:49)
Yeah.

James Governor (18:50)
You’re able to say nobody died at all times in the process.

Rachel Stephens (18:54)
This is true. As soon as I take my children to school, I’m in a nobody died zone. I it like I have to keep people alive from the afternoon to the morning and then I’m scot free.

Kate Holterhoff (18:59)
and Rachel.

James Governor (19:05)
Amazing,

there you go.

Kate Holterhoff (19:06)
Someone else’s job. Rachel, let me run this by it. You have this amazing quote in one of your pieces where you say, vibe coding and spec driven development aren’t opposing camps. They’re different tools for different stages of the creative process. And Chef’s Kiss, I love that phrasing. I’m interested in how you’re defining creative process there. Are you thinking of the SDLC or are we getting creative in the sense of wild experiments?

Rachel Stephens (19:31)
Like for me, I think vibe coding in particular is great for early stage prototyping. Like what do we, what is it I even want to build? Or if you’re talking about different personas. So like if I am a designer or a PM and I want to hop in and instead of just like create, as I just talked about creating a requirements doc, instead of like creating a static document, I can create a prototype that I want to share with my team. It’s like, this is what I’m thinking. Like how can we make this real? And instead of just having kind of these nebulous abstract ideas that we’re talking over, we can actually talk over an actual prototype.

Which I think is awesome. And then you can kind of go in and flesh it out and make it real in a more structured way and a more maintainable way in a way that you can like somebody else can actually, nobody’s gonna go back through and say like follow your like prompt history. it’s like, well, what were they building here? Like let me read through screens after screens of how they prompted this versus I can go through and read a requirements doc and be like, like this is what we were going for. Like we have documented what our intent was.

was we can see what we were trying to do. We can see what the agent’s built from. And so for me, it’s like, it’s, I feel like prototyping and like vibe coding is really like single player mode and spec driven and development is when you start to get something that like, need this to be understandable and not just between me and the AI, like peering together, but like the rest of my human peers also need to be able to understand what it is that I was doing.

Kate Holterhoff (20:54)
All right, so we are almost out of time. I would like to end with us all doing a little bit of thinking ahead. What are we going to be following in terms of AI into the fall and the winter? What are you looking for in the next six months? Give it to us, you know, in a sentence or less. Rachel, will you go first?

Rachel Stephens (21:11)
Is this like slightly AI tangential? I think the thing that I’m interested in is whether API access stays open in a world of MCP. So if MCP has it so that we can all access our data across all these things and people are trying to also build competitive modes by building AI tooling into their own tools, like are they going to shut down API access so that we can’t pull all of our data across systems or really curtail how that happens. I’m interested to see how that happens.

Kate Holterhoff (21:43)
That’s a good one.

Yes, love it. All right, James.

James Governor (21:47)
Yeah, grumpiness and making things a bit more deterministic and understanding behaviors and less hallucination. I think the companies, there is going to be some nuclear winter happening at some time quite soon. It may not be this winter, it may be next winter, but you just cannot have the absurd valuations and money pouring into as many companies as it is.

So I think the companies that come out of the other side are going to find ways to accommodate consistency. And I think that that’s what I’m interested in. That’s why the spec thing is interesting to me. It’ll be interesting to see how Kiro continues to do from AWS. You got Tessl, they’re going to be in the mix there. Specification, I think, is a really interesting idea.

So yeah, watching what developers do in terms of, as Rachel called it, multiplayer but also yeah living in a world where Frankly, we already have enterprise data, right? And yet these machines are just like popping out URLs that like, they don’t exist. So we need to, there are lots of things we need to improve and that’s good because, know, tough problems to solve. That’s what we’re gonna help with. And that’s what I’m gonna be interested in seeing what companies can help with that.

Kate Holterhoff (23:10)
All right, and I’ll go last, keep it quick. I think that the spec-driven development idea is going to become bi-directional. So instead of it just saying, like, here’s the spec, you know, create the code from it, we’re going to see it more as a sort of interplay so that we revise the spec and it becomes more like documentation. I think that that’s really the missing part of that whole sphere of how is it that we’re not only documenting the code that we have, but also writing the new raw information that we’re using for these applications. So, all right.

James Governor (23:43)
Yeah, final thing, IDEs are gonna look super weird. Like I know it’s all agents all the time now and we’ve moved away from it, but we’re gonna have some weird editing tools that really think about this spec. We’re gonna have spec driven dev tools that do not look like the editors that we’re used to. So we’re gonna see some weird innovation in those front ends. It may just be a big tab button, but we’ll see.

Kate Holterhoff (24:04)
love it. I love it. love it. So James got in two, hey, that works. All right. Really enjoyed speaking with James and Rachel today on this MonkCast with the Monks. Again, my name is Kate Holterhoff, senior analyst at RedMonk. If you enjoyed this conversation, please like, subscribe, and review the MonkCast on your podcast platform of choice. If you’re watching us on RedMonk’s YouTube channel, please like, subscribe, and engage with us in the comments.

MonkCast with the Monks: AI’s Grumpy-Fun Era

Links

Transcript

No Comments

Leave a Reply Cancel reply