---
title: "Why ChatGPT Recommends Your Competitor: LLMs & the Future of DevTool Marketing with Adam DuVander"
date: 2026-09-01T08:00:57Z
modified: 2026-08-24T11:45:27Z
permalink: "https://redmonk.com/videos/adam-duvander/"
type: video
status: publish
excerpt: ""
wpid: 4995
categories:
  - Podcast
series:
  - Conversations
channel:
  - RedMonk Video
featured_image: "https://redmonk.com/wp-content/uploads/2026/07/Adam-DuVander.jpg"
timestamp: 2026-08-24T11:45:27Z
tags:
  - Podcast
  - Conversations
  - RedMonk Video
---

Why does ChatGPT recommend one DevTool over another, and what can founders do about it? In this RedMonk Conversation, Kate Holterhoff sits down with Adam DuVander, Principal Consultant at EveryDeveloper and author of _Developer Marketing Does Not Exist,_ to explore how AI assistants are transforming software discovery. They discuss Adam’s LLM Rank research, which tracks how models like ChatGPT, Claude, and Gemini recommend developer tools, and why those recommendations often differ. The conversation covers the practical steps DevTool companies can take to improve AI discoverability, from documentation and developer experience to use-case positioning and contextual content. Along the way, Kate and Adam examine the expanding definition of “developer,” the rise of AI-assisted builders, and why developer marketing is evolving rather than disappearing. Whether you’re building a developer product or rethinking your go-to-market strategy, this episode offers a practical framework for succeeding in an LLM-first world.

## Links

- [LinkedIn: Adam DuVander](https://www.linkedin.com/in/duvander/)
- [llmrank.fyi](http://llmrank.fyi)

# Transcript

**Kate Holterhoff (00:04)**
Hello there, Kate Holterhoff here for another MonkCast, and I’m excited to have on as a guest today Adam DuVander. He’s the principal consultant at EveryDeveloper and the author of Developer Marketing Does Not Exist, my favorite book on that subject. So, you know, totally jazzed to have you on the the show here and to talk a little bit about where where we’re at in developer marketing these days.

**Adam DuVander (00:27)**
Awesome. Thanks so much, Kate. Yeah.

**Kate Holterhoff (00:30)**
I want to begin with simultaneously the most difficult question I’ve been asked recently and potentially the easiest. So and I’ve had this I had this pulled on me just recently. So, Adam, what are you telling folks who ask you to define what a developer is these days?

**Adam DuVander (00:50)**
Yeah, wow. I I mean I ha I have been saying that that definition is expanding. And I mean it’s been least more at least more than a year that I’ve been saying that. So I guess if it’s continuing to expand, I’m not sure it’s hit everybody yet, but but but quite a few are. I think there are still distinctions we can make to kind of

**Kate Holterhoff (00:51)**
Ha ha ha.

Okay.

**Adam DuVander (01:17)**
write code or at least ship code for a job versus those who are maybe doing it as a hobby or to solve an initial problem, right? I think there are distinctions there to be to be made still. but maybe as that expands, that distinction will become less important. But but I think you have still your kind of traditional used to write the code and now

**Kate Holterhoff (01:30)**
Yeah.

**Adam DuVander (01:46)**
write the code alongside an agent, likely developer and and then others. And I’m happy to talk about either of those. I think the approach that a that a company that has a product for one of those, I think that approach changes a little bit depending on on who that audience is for sure.

**Kate Holterhoff (02:07)**
Right, right, right. Yeah. So just in terms of the workplace, whereas, you know, a company that does use a lot of technologies, maybe not necessarily like a SaaS company or or exclusively like a software development company, but like i you know, at i in the workplace, who now is self-identifying as a developer in your experience?

**Adam DuVander (02:30)**
Yeah, and that’s it’s funny, that’s actually a slightly different question than the first one, right? That self-identification. Because because I do think that a lot of a lot of the folks who maybe haven’t traditionally shipped code wouldn’t give themselves that label because they still feel the the

**Kate Holterhoff (02:34)**
Right. I know. We’re gonna build on this. Yeah.

Okay.

**Adam DuVander (02:54)**
Responsibility. I was gonna say baggage. It could be that, but the responsibility that comes alongside that label, right? Of of like knowing the software development lifecycle and feeling confident that they’ve found edge cases, all of the things that that someone who is just looking to solve a problem might not feel as confident about, right? so yeah, and I and I’ve started to see companies.

**Kate Holterhoff (03:17)**
Yeah, right.

**Adam DuVander (03:23)**
center on this sort of builder label that is I think most of them mean it to be kind of a the expanded definition, right? That developers still have a place inside there. in some cases, some might truly mean this this person who is now doing stuff with code that does not have that, has not had that as a job before. So

**Kate Holterhoff (03:36)**
Mm-hmm.

**Adam DuVander (03:51)**
So I’m not sure everyone is using it the same way to mean this sort of expansion that includes traditional software engineers. but but in some cases they they are. And it’s sort of okay, let’s call it this so that then those that are developers can find our maybe more technical avenues toward this, jump into the docs themselves, and then those who are more on the builder side will will use the tools that’s

**Kate Holterhoff (03:59)**
Mm-hmm.

**Adam DuVander (04:21)**
will kind of do it for them and and help them along.

**Kate Holterhoff (04:25)**
Okay. Okay. Well this is gonna help me to segue into trying to understand what it is that EveryDeveloper does. So the definition of developer is expanding here. It seems like we might even get to everybody. Everybody is a developer. I don’t know. Talk to me about what EveryDeveloper is and does and and how it fits into this conversation.

**Adam DuVander (04:37)**
Yeah. Right.

Yeah, so we work with DevTool companies to help them figure out why developers traditionally would choose their products and what problems they’re looking to solve with it, which is kind of that first step that you need to understand to be able to to have them choose your product. the name itself has a was never meant to mean everybody as a developer or or that you want to reach EveryDeveloper.

**Kate Holterhoff (04:55)**
Yeah.

**Adam DuVander (05:16)**
it it came out of so my background is I was the editor of Programmable Web, the first directory of APIs, watched that sort of if you’ve ever seen that hockey stick chart that used to be in every conference presentation. you know, I and my team found those APIs to be able to generate that that chart. And so one of the things that that made that difficult was

**Kate Holterhoff (05:37)**
Wow.

**Adam DuVander (05:45)**
We were always in search of these new APIs, but it felt like there’s some amount that really EveryDeveloper needs to know. And so that’s so when I left there, I thought, you know, what if I just zeroed in on those and we identified the things like great developer experience that they have in common and and could start to see like what are what are the things that people would would look at and choose. And so that’s where so

**Kate Holterhoff (06:06)**
Yeah.

**Adam DuVander (06:15)**
The APIs EveryDeveloper needs to know was sort of the, like let’s let’s have this at everydeveloper.com. So that’s where the name comes from originally. Obviously what I do is different than that, but also very related in that we still care about what are the things that that those devs want from a developer experience standpoint. And yeah, and what are the best the best examples of that doing?

**Kate Holterhoff (06:45)**
Yeah. Well I you know, I feel like you’ve alighted on a very good title for a company, for the sake that it is kind of like ambiguously vague in a way like allows for lots of potential meetings here. So this is good. It feels inclusive. I like, you know.

**Adam DuVander (06:57)**
Yeah.

**Kate Holterhoff (07:04)**
So we’re setting up the question of like what is a developer. To put some of my cards on the table here, I’ve talked to folks like Sean Wang about, what it means to be an AI engineer and the fact that he’s now including like the agents themselves as developers. More folks are doing that. I’ve written a little bit about the

these agentic teammates or, AI coworkers and and all this sort of terminology around that. And so I think that we’ve we’re at this interesting nexus of trying to figure out what the heck a developer is. what sort of identity is still gonna be authentic to this label as everything else is getting smooshy and and you know, we’ve got PMs that are shipping things now. And you know, it’s just it it’s it’s it’s moving quickly.

**Adam DuVander (07:41)**
Yeah.

**Kate Holterhoff (07:49)**
And you know, so we’re trying to keep up and I I feel like you’re you’re very much at the forefront of this. so I guess to to yeah, well let me let me first get your reaction to that. Can you know, a computer be a developer now?

**Adam DuVander (08:04)**
Yeah. I I can’t say I’ve personally got gone that far yet because I still feel like they someone someone has to has to tell them what’s important, right? And and it feels like if you if you walk that back and you know maybe it’s the agents, agents, agents, and then finally you finally you get to that human who is who is there who is who’s saying this is the direction we want to go.

**Kate Holterhoff (08:10)**
Okay, that’s that’s the Rubicon.

Okay.

**Adam DuVander (08:33)**
That to me is still the still the developer, but at some at some point it’s it’s semantics, right? It’s like that agent does need to find things like a developer would need to find things, and they’re going to need things perhaps slightly differently than than that human would. and I think some some might say drastically differently. I think that’s another area we could we could dive into.

**Kate Holterhoff (08:44)**
Mm-hmm.

yeah. yeah. okay. Well yeah, as you know, but let’s let’s do it. I mean my f my first sort of thought on this is like how do you market to this new breed of developers? Would that be a fair place for you to you to sort of share some of this wisdom you’ve been accruing?

**Adam DuVander (09:17)**
We’re w are we talking about the humans here? in the definition? Yeah.

**Kate Holterhoff (09:20)**
Well, I I’m thinking of like,

if Developer Marketing Does Not Exist, who who what’s the developer anymore? Sh do you need to rewrite this book? where does that sit? So but no, I I a as we sort of hone in on this and and talk a little bit about some of the projects you’re working on now, yeah, I I would be happy to to take it both either direction. So what what is appealing to you?

**Adam DuVander (09:30)**
Yeah.

Yeah. Yeah, and on on on that title, which also s sits with me the same way EveryDeveloper does, right? I’m I’m stuck with it. I can’t use that phrase developer marketing ever, right? Because I’ve already said it doesn’t exist. But the the idea there was that’s that developers don’t want to be marketed to. They don’t want to see that marketing, they don’t want like they they want to be

**Kate Holterhoff (09:47)**
Yeah.

Mm.

**Adam DuVander (10:07)**
educated and inspired, right? And I’ve had many people say to me, you know, by the way, nobody likes marketing. Nobody wants to be mar you know, like ever everyone could be that. So, you know, we we might yeah, we might we might be arriving at that same destination where it is it is the you know, maybe it is the core the core piece of marketing to this person who wants to build something.

**Kate Holterhoff (10:19)**
Well actually Adam

**Adam DuVander (10:36)**
with all of the many tools we have now that yeah that they they don’t want to be marketed to. They want to find the right tool at the right time and know feel confident that it can help them and and so I think there are I mean there are a lot of tools now that are that are helping them to be able to do that. A lot of times it is asking the agent

That’s you know where a lot of my attention has been recently is on LLM Rank, which is looking at when these developers ask these questions, what are the products that come back? And and how does that how’s that different based on the model that they’re that they’re asking too? And so I mean I’ve been for most of this year asking kind of the top three, so Gemini.

OpenAI and Claude and asking them about these popular developer categories and watching as it changes and shifts. and then the the real fun stuff is seeing how does that change as the way you ask changes and the the way you ask. I mean, I think that’s maybe one of the big differences between if we just take very broadly the traditional developer and the builder.

that we’re saying now, the new developer, the way they ask questions is likely going to be different also, or the things that they want. And yeah, and and we’re in a in a time now where a little tweak of language can drastically change what you get back.

**Kate Holterhoff (12:02)**
Mm-hmm.

This is stressing me out. I yes, absolutely. you feel like a a like talk about a wizard, right? You’re like, I gotta get the incantation right or else it’s just not gonna work. okay, but to take a step back, so your the project you’ve been working on, LLM I will provide a link in the show notes. but it’s llmrank.fyi, correct?

**Adam DuVander (12:20)**
Yeah.

Dot FYI, yeah. Yep.

**Kate Holterhoff (12:39)**
Okay. And so are you considering this like a benchmark? Like what do you what do you call like what is what is the nature of this project? Is this like sponsored? Like what yeah, what it what is it? Is it like your research? Okay.

**Adam DuVander (12:48)**
It’s not not at all sponsored. Yeah, yeah. So

it’s it’s research and it’s actually a lot like the to be full circle back to to the starting of EveryDeveloper of looking at what are these tools that that people want to use. And yeah, and it’s I I mean I I share it with folks and I get back, you know, really we should be number one, obviously, is the is the first thing people say. Like

**Kate Holterhoff (13:00)**
Yes.

**Adam DuVander (13:17)**
th their product should be number one or, you know, I think this this and that is is wrong and like it might be, but I kinda have to say, like this is like there’s some editorial judgment in there, some, certainly in how the question is asked, but really I’m trying to keep it at what are these models saying and then what can we learn from from their responses? And then how can how can you use that to take that that next step and

Yeah, and so it’s it’s yeah, fully I’m I’m running them and I’m sharing this data and I’m I’m sharing what I learn as I go too.

**Kate Holterhoff (13:58)**
Yeah. Well, I have had a very good time checking out this site here. We love a little data, RedMonk, although skeptical, right? metrics with an MEH, you know. we want you know, but I think what is very cool about this is we’re living in this world of like new benchmarks. So I’m thinking of like Simon Willison’s he’s got the pelican riding the bicycle thing. I maybe you’re familiar. and that’s like his way of try trying out a model.

And it’s just like a silly, goofy way of of like testing the model. I feel like we had yeah, w the the analyst at RedMonk should come up with with our own little goofy trial here. But but yeah, for for like a concrete example here, I’m looking at LLM Rank and you’ve got CDN. So if developers ask AI, which CDN should I use, it always ranks Cloudflare at the top, and then we’ve got Amazon Cloud Front and Akamai, and then Google Cloud CDN, and then it the list goes on.

**Adam DuVander (14:26)**
Okay, yes, yeah. Yeah.

**Kate Holterhoff (14:54)**
But what’s interesting is it’s got like you know, the rankings go up and down. And then you you you break down which models ranked how. And so that’s very fun. And you and so I’m sure folks will want to dig into the methods. I don’t know that need to go through everything, but basically you’re asking the model ten times about this and you don’t share that your exact question because you don’t want folks to game it. Am I leaving out anything important for folks to know to to kind of get it?

**Adam DuVander (15:16)**
Right, yeah, yeah.

no, that’s it. Yeah, and asking the newest model. So so making sure so there have been so the most recent numbers, July is the first time that we haven’t seen one of those models change. So so the the changes that you’re seeing there are either just the natural variability that you would see, or like it seems like it’s enough that there might be some

**Kate Holterhoff (15:21)**
Okay.

**Adam DuVander (15:47)**
some subtle sort of tuning that is happening behind the scenes. I mean there’s not enough for us to really know whether that’s the case. But they were fairly consistent between those months. but but yeah being asking those questions and then the the sort of next layer is is that kind of

What are the differences? What are the differences between specific use cases and things like that, which are not quite out there yet? These are numbers. I mean, this is what we’ve been doing with clients is looking at what are the different use cases and how does that change. and then that’s to me the excitement. Yes, it’s fun to see where the ranks change, but the real excitement is like when you ask for the open source tool and the entire top 10 just drops out and it becomes.

you know, the bottom of the of the the ranks might move up because they were open source and the what were looked at as the category leaders are not. And then you have actual open source tools that are now part of the conversation that they weren’t before when it was just this broader sort of sort of topic. And open source is the one that I’ve looked at the most on there and and yeah it’s

I mean in in a huge number of cases, I’m looking here, yeah, the I mean in the so in identity management as an example, you have that situation where the top ten disappears and five from the lower come up and then five net new products that are that are a lot of those going to be like true like link to GitHub repo open source tools that might not have

have been the thing that that the LLMs had decided the developer wanted when they were asking for a way to solve this because the quickest way sometimes is, hey, here’s this service that off the shelf does it. But if you’re asking for open source, then there’s a different set of things that you want to achieve here.

**Kate Holterhoff (18:07)**
Huh. this is fascinating. Okay. And so you’re working with companies to help their ranking in terms of this sort of AI assisted discovery.

**Adam DuVander (18:19)**
So that’s on the EveryDeveloper side. So I mean we’re not directly saying, hey, work with us and we will change where things are on LLM Rank. But but but yeah, looking at what if you look at the delta between these, then you can learn. I mean, maybe you’re okay that you don’t show up on the open source because your product is not open source. And so you’re okay.

**Kate Holterhoff (18:23)**
Yeah.

Right. But you personally are doing a little bit of that. That’s a little bit of the consulting you’re doing.

**Adam DuVander (18:48)**
being essentially invisible to the developer who’s looking for that. but what we help help DevTools with is understanding what where do they want to be visible? What do they think their positioning is, and do the LLMs agree is essentially the essentially the question there. And so yeah, when someone comes and says we think we should be number one, well, right, I mean, right now the LLMs don’t

**Kate Holterhoff (18:53)**
Mm.

Mm.

Amazing.

**Adam DuVander (19:17)**
Don’t agree. Let’s figure out what that means and what does it even should you be attempting to be number one in this category when there are all of these contextual ways to look at it where you might already be number 1.

**Kate Holterhoff (19:33)**
Yeah, right, right, right. I mean this is so fascinating to me as someone who admires your book, which came out in what, twenty twenty-one, is that right? Okay. because in that you talk documentation and the importance of documentation to developer experience. and now and and sort of all the writing that goes around that. Blog posts, you know, social media, having

**Adam DuVander (19:42)**
Right, yeah.

Yeah.

**Kate Holterhoff (19:58)**
You know, the all all the the sort of developer watering holes where you would expect developers to be looking for these answers. which have been completely upended, of course. You know, Stack Overflow now is, you know, it’s it’s not being used in the way that it has. I don’t want to say it’s dead or anything, but so what’s interesting to me then is what you how how things have shifted. Is documentation still now the avenue to help folks who want to improve?

**Adam DuVander (20:11)**
Right.

**Kate Holterhoff (20:27)**
how these models are finding these DevTool companies.

**Adam DuVander (20:31)**
Right.

Yeah, and I I I mean I don’t wanna I don’t wanna go on a on a definition tangents, but but you know, I think so thinking about documentation, developer experience and what is that if you just think about sort of the how it looks, like a developer wants it to be dark mode and have the syntax highlighting and those sorts of things, right? Then

**Kate Holterhoff (20:40)**
Sure. Okay.

Yeah.

**Adam DuVander (21:00)**
That stuff is not at all important to an agent who can’t see, right? but I think I think it goes much deeper than just the how it looks to like what matters. And so if you go back to the I mean the book starts out with a developer experience chapter, like to set that foundation. Like this is where it starts. And of course, written at a time that where

**Kate Holterhoff (21:05)**
Mm.

**Adam DuVander (21:28)**
Agents were not the way that we were developing. But yeah, if you go back to those 13 criteria that I lay out in the book, I mean almost all of them is pretty directly translatable to things that an agent would need. And I’ll so I’ll give you like like three big ones off the top. Like so accurate, up-to-date reference documentation. Right?

**Kate Holterhoff (21:31)**
Yes.

Wow.

**Adam DuVander (21:56)**
I mean that was something that a dev needs, and I think you’d be in a lot of trouble if you don’t have that for your for your agents in in some way, right? They they ha need to have at least trained on that somewhere. Some accurate up-to-date documentation must live. And just based on the training time frames, you would want to make sure that it isn’t just counting on on that sort of

**Kate Holterhoff (22:06)**
Yeah.

**Adam DuVander (22:25)**
training data. But then so in addition to that, you’ve got the the the actual context of the developer, whether that’s whether that’s the agent or the or the human developer. And and that’s like are there samples or something that that f frames the a use case? Like like is there some way to show this agent that the thing they want to get done?

can get done with this product. So some kind of framing around use cases like in in the in the olden DX, we would say sample apps, right? and then the other important piece of context is what’s the language or stack? And like that a lot of times now is coming, I mean, the agent has that context in the code base that they’re building in.

And that is going having something that is framed around whatever that stack is or the particular language is still important. And I think having a having an SDK can still help that agent be able to integrate quicker than than not. And and that is still, I mean that’s straight out of

The I mean the example I’ve given for years for framing around programming languages is to look at the Heroku Dev Center and you see those beautiful logos of programming languages. And if you are one of those, if one of those languages is the stack you use, you’re going to see that. You’re going to say, This is for me. You’re going to click on it and you’re going to get the things you need, the documentation framed around that that language. And

Even though an agent is multilingual, doesn’t have a a specific one that’s their favorite, the code base they’re in is going to have selected that at some point. And so you still want to be able to have that context. So those those are three of the 13, and I think they’re they’re still very valuable, and you could argue more valuable than than they were.

before. I mean that first the updated and and accurate. I I mean, I don’t know. Maybe people listening to this will say that’s obvious, but I think anyone who has has integrated with anything will say y yeah, maybe obvious, but also really dang important.

**Kate Holterhoff (25:03)**
Yeah. This is super helpful. okay. So the book still relevant, we’re the the user potentially has changed. so i if when more like when when these DevTool founders come to you and they say, Hey, ChatGPT is recommending our competitor over us, the first thing that you’re telling them to check then? How are they prioritizing this list of things that are are so important now that the agents are doing so much of this recommending?

**Adam DuVander (25:31)**
Right, yeah. and a lot of the talk now also is can that happen without the human at all? I mean, even more important, right, at that point, if if these agent loops are all making these decisions, then we don’t even get a chance to say, we wanted you to look at something else. yeah, s so for sure. So so coming with that with that single model question, I mean that says, and I think this is this is the I mean it’s the

**Kate Holterhoff (25:40)**
Wow. Yeah.

**Adam DuVander (26:01)**
classic developer way to be, right? To you sort of have your frame of reference and you stick in it. And so so those founders might not realize that while they aren’t being recommended by Chat GPT, they might be the number one choice of Claude or Gemini or another, right? Or both or all, right? And and that’s I mean one of the things that’s

**Kate Holterhoff (26:25)**
Mm-hmm.

**Adam DuVander (26:31)**
That I’ve seen is just the so it’s a about half of the time in the LLM Rank, so the 33 categories that that I’ve looked at, about half of the time, all three models agree on a top three at least. Maybe not the order, but the the top three they agree, which sounds good, but that means that when someone is looking for your tool, DevTool Founder, that there’s a

50% chance that if they asked a second model, they’d get a different recommendation. Right? That’s yeah, yeah. So so being able to look, so I think it’s it’s a different message to the devs and and to the dev founders, to the devs, like adding a little bit of of robust search in here to, you know, right? Like

**Kate Holterhoff (27:08)**
That’s big. Yeah.

**Adam DuVander (27:28)**
Maybe there’s a skill, so to put it in agent terms, there’s a skill to choosing one of these tools. And it should be not just taking the first recommendation. It should be not just looking for a single model, the answer from a single model. but looking for some consensus, making some comparison, certainly give it as much context of what you want to achieve as possible.

**Kate Holterhoff (27:39)**
Mm.

**Adam DuVander (27:56)**
And then, yeah, to the DevTool founder, it’s yeah, look look at that, but also make sure you understand where else devs are are looking. And and interestingly, Claude is often the outlier of of the three. It’s not you know more than the other two are. and in my experience and talking to devs.

Claude would also be the number one choice of model. So so there’s a good chance as a dev you are asking something that might disagree with another. And yeah, and as a as a DevTool, if you’re asking, you you should get the full picture. and then one other bit to the DevTool founder is to we often we talked about how the language can make a big difference.

Like we often are asking the question we wish someone would ask. And so we are asking it in a way that might that maybe you would copy paste from your H1 on your on your homepage and giving giving a little bit more thought into how that’s asked. And some of that could be having multiple ways of phrasing it. I I heard a story from someone going to a

**Kate Holterhoff (29:11)**
Yeah.

**Adam DuVander (29:25)**
going to a company retreat. And they they all wondered, where do we where do we sit when we when people when devs are asking about this? So they all open their laptop, they all go to their own favorite model, they all ask their own separate question, and then are surprised when everybody in in the circle there has essentially a different answer. and that’s

That’s because it’s going to matter how you how you how you ask that, right? And which model you ask. And so trying to trying to make sure that you take stock of which models are the ones that your audience is most likely to be using and try to expand that beyond beyond one, and then really think about what are the questions devs would

actually be asking for. So a lot of times that’s that’s more the problem and less the solution. It’s the, you know, the classic dev marketer problem is kind of leading with that solution with that feature and not the thing that’s in the dev’s mind before they’ve discovered you, right? Before they know that you you are the feature fet flag solution.

Before they know that you’re the feature flag solution, you want to be saying, How can you experiment? How can I make a change with some of my users? That kind of approach. And and yeah, and so bringing that into the same way you’d bring that into your messaging, you want to bring that into how you’re asking these LLMs as well. Otherwise, some of the data you get is going to be.

fully biased, probably toward you. Maybe it will make you feel better, but it won’t be what devs are really experiencing.

**Kate Holterhoff (31:27)**
And so if a DevTool company discovers that they are not ranking very high, you’ve got your thirteen things that you suggested for marketers to do historically. is it just trying to like write a blog post that uses that sort of phrasing, create a little content, a YouTube video that then does satisfy that? Because the the models take a while to refresh, right?

as opposed to like, you know, Google search or something. I i it feels like maybe the payoff isn’t as quick the turnaround.

**Adam DuVander (31:55)**
Yeah. It’s

and it’s funny, SEO also is a long game, or historically was a long game. From what I’ve seen, it’s a longer game, this this the LLM ranking for sure. not only the the how infrequently the training is the retrains, so the training cutoffs,

**Kate Holterhoff (32:09)**
Wow. Yeah.

**Adam DuVander (32:25)**
But also it seems like incumbents stick around, like they they stick around longer than, and this is this is just sort of a like developer knowledge gut check of like, so I I can’t say for sure, right? But it like as an example, Bootstrap was still kind of in the running. I mean it still is, but I mean it was until very recently was kind of like

**Kate Holterhoff (32:38)**
Mm-hmm.

**Adam DuVander (32:54)**
up top and even higher than tailwind, which in in my developer circles, that feels like, that’s like two years ago. But I think that a lot of it is is just there’s so much that they’ve trained on bootstrap for so long. People have talked about it. So that’s which is going to be the case with every incumbent, right? That if it’s already the or if it has been the de facto number one, then

**Kate Holterhoff (33:02)**
Right. Yeah.

**Adam DuVander (33:22)**
There’s just gonna be more to latch on to. And w one other piece could be that the models have not weighted newness higher in a way that that Google, I mean, this is maybe five years ago, 10 years ago, right, said like this is important that you know you should go and update your your post and don’t just put a new date on it, like actually change it to to be able to still be relevant.

**Kate Holterhoff (33:48)**
Mm.

**Adam DuVander (33:53)**
I’m sure there’s some of that weightedness in in the LLMs, but it seems like it’s not heavy enough for what we all would expect. And so incumbents are sticking around, which which yeah, just means means longer longer timelines. I think that is still where the sort of thinking about it in terms of context versus category. You’re not owning owning a f don’t you know the

The category leader will take a long time, but the context leader could be much sooner, as soon as the next training data. also most of these most of these LLMs will pull in recent search results, like they will factor that into some of the results. Intentionally, LLM Rank does not use that right now because it feels like that could could kind of

Taint what we’re seeing with the the actual training data. And the reality is the training data is what a model’s going to fall back on if some of the the anything new isn’t the newer context is not able to help. And so it’s sort of like, how do we see this sort of foundationally? but but if you look at at sort of the the contexts, which would be things like use cases.

Then you start to see, okay, we’re already we’re at least performing better in the in this one. Or if we think that we should be the top Python tool for X, then and we’re we’re number three, and we know that we have better tooling than some of the others, then it’s taking stock of, okay, are there places that we haven’t said this? we don’t actually have a

Python landing page where someone where a dev or an agent can go and find all of the resources that they would want from a Python stand standpoint. maybe that’s maybe that’s a change to make. Yes, you’re it you’ll you’ll have to wait potentially to see that come out across, you know, rankings, but I think you can see and and we’ve definitely seen at the very least with a

Developer who has discovered you, if you provide them a page like that, then they’re going to have a much better chance of being able to use Python in this case with your product. And yeah, so there are there are moments along the way where you don’t have to wait that long time. But recognizing if you’re if you’re on LLM Rank right now and looking and saying, we need to be higher, we need to be higher immediately, that’s that’s going to take a longer time.

**Kate Holterhoff (36:44)**
Yeah, I think many vibe coders have had the experience of saying, Hey, to try out this new tool, I’m thinking of like bundlers or something. You know, you say, Hey, I want to try out Rspack or Vite or something, and it’s like, no, no, we’re using Webpack. And then it’ll just even like silently switch it back to Webpack because there’s just so much documentation written around that. And this is happening so often. I’ve also heard the experience that some are having where, you know, if they’ve had a major change.

**Adam DuVander (37:02)**
Yeah. Yeah.

**Kate Holterhoff (37:12)**
with that you know with that product, the old documentation is showing up and giving bad advice then. is have you found a way to like overhaul that? Like is is the answer to like delete it or or is I don’t I don’t know, to like make it clearer that this is outdated, to just like so somehow tell the robots you know we create more robots files. I don’t know. Yeah, what what are you what are you telling folks who are getting bad information from the agents?

**Adam DuVander (37:27)**
Right. I be Yeah.

So

So I think correcting correcting the record is definitely the first step, right? And and including that wherever you can. I think one who who I’ve seen having this issue is Stripe, who is usually way ahead of the game in all all things DX, right? And you would think, they have solved this, they’ve got

**Kate Holterhoff (37:48)**
Mm-hmm.

yeah.

**Adam DuVander (38:07)**
They’ve got developers, you know, they’ve got the agents using their the right version of their APIs. Not the case, though they’re still making sure to to l have it be visible. And it’s one one thing. They’re the only one I saw in an LLMs.txt and I went through this goes back to end of last year, went through for 30 some, I think the there were.

At that time, 20 of the 30 that I looked at that had that file. And so went through Stripe was the only one who kind of spoke directly to the LLM at kind of a section that said, if you are an LLM agent, please use the latest version of an API. Look at deprecation headers. You know, it gave sort of a a like

**Kate Holterhoff (39:03)**
Yeah.

**Adam DuVander (39:05)**
This is what to do. There’s still questions about whether whether the LLMs use that LLMs.txt, but that’s one place to put it. And and absolutely, I think that’s a great example to follow to have that. Then I think you’re seeing a lot of dev tools who are kind of throwing a lot of different things. They’re creating skills, they’re MCP servers for sure. I think.

**Kate Holterhoff (39:07)**
Wow.

Right. MCP

absolutely, yeah.

**Adam DuVander (39:34)**
I think these

are these are all around answering exactly that question, which is the stuff that comes after what we’ve talked about. The actual discovery, right? Like that’s yes, it’s important, but then can they actually be successful? Can the agents be successful when they’ve found your tool? And that’s where there are are a lot of these tools that primarily are human developer.

**Kate Holterhoff (39:54)**
Yeah.

**Adam DuVander (40:02)**
Here is a thing, please give this to your agent. and the the sort of maybe not spoken but but kind of message underneath is your agents are making the wrong choices, choosing the wrong tools. Please give them this thing that will help them make the right choices. And so I think that’s why we’re seeing a lot of those. And so yeah, you should

**Kate Holterhoff (40:27)**
Mm.

**Adam DuVander (40:32)**
Make sure that documentation can be easily copy-pasteable. I think especially for that builder audience who if their agent isn’t able to do it without that, they maybe will give it that sort of copy-paste. And then after that, they’re probably moving on to another tool on the list. Do you know anyone who is not this one that you told me that I can’t get to work, right? So

so anything you can do to reduce that friction. And so that could be something as simple as copy pastes in these docs. but but yeah, might be MCP server skills, things that’s that will be able to orient hopefully the the documentation you already have that humans can get as well. Think things like here’s the everything you know to get started, here are

sample apps so you can see how we build and what we what calls we make. Yeah. So I mean that’s whether that whether that makes a difference, I think, you know, is still still to be determined, right? And whether that makes it into training the next time so that these issues don’t happen. And we we also know that the there’ll be changes in the future that

**Kate Holterhoff (41:50)**
Yeah.

**Adam DuVander (42:00)**
are also going to be after this this cutoff. So the more that we can we can provide these tools, yeah, the better. But right now it’s still the human has to in many cases point the agent at this tool that then will help them be able to use it better.

**Kate Holterhoff (42:21)**
and I’m glad that you articulated that distinction and between like, making sure that developers, builders are successful once they have chosen the tool, and then the discoverability, you know, th these are separate concerns, you know, there’s some overlap, of course, but these are these are things that both need to be addressed by founders. okay, so I want to play devil’s advocate here. So one could argue that on the the discovery side.

**Adam DuVander (42:34)**
Yes, yep, yep.

**Kate Holterhoff (42:49)**
that any enterprise is not going to use Claude or Chat GPT to actually discover products. That they’ve got their seasoned senior developers, their engineering leadership, whoever, who are gonna be choosing. And so that isn’t this is just hobbyists. Like who cares, right? This is not important. And potentially maybe it’ll never be co important. May I could imagine someone arguing that. So, do you respond to that? What

Why do you think discoverability is important for from a purchasing side? Like why why it why is this part of a GTM strategy like PLG? You w t talk to me about who why should I care?

**Adam DuVander (43:29)**
Yeah, so I I mean let’s let’s first of all just assume that maybe they maybe they are right about tooling in internally. That will be the case. but we have seen a long line of developers who are working on that side project who happen to also have you know senior architect role at at large corporations. So there are plenty of

**Kate Holterhoff (43:56)**
Absolutely.

**Adam DuVander (43:57)**
plenty of examples of

that happening over the years. now whether that whether that continues, I mean, I I think probably the trend is likely too. We can also look back at at bringing your own device into work. And I mean 15 years ago that it was mostly like, no, we’re gonna lock everything down. And now there’s a realization that at some level you know

**Kate Holterhoff (44:01)**
For sure.

**Adam DuVander (44:25)**
devices are allowing us, at least even post-COVID, right? Like we we learned, right, we need to actually be able to to to get our work done distr in a distributed way, right? So I think similar things we’re we’re seeing here. Now, if the question is should there be other models that you that you look at and see how how those are discovered there.

**Kate Holterhoff (44:39)**
Mm.

**Adam DuVander (44:54)**
‘Cause I think even that that senior team is going to be using these tools. They’re not going to be saying saying, All right, we’re only going to go and do our own non-agentic research on on the tools that we use. They’re going to be using them too. So yeah, it should there be other models, you know, maybe. it’s it’s a lot to keep in my brain at least for three right now. But but as others

**Kate Holterhoff (45:01)**
Mm-hmm.

**Adam DuVander (45:23)**
arrive for sure that’s a that’s a possibility. yeah but then I think it gets to also that audience and so if someone says these these independent builders they’re not important that’s we’ve been seeing that in in developer circles since way before ai. And that is it’s a it’s a natural tendency for for

**Kate Holterhoff (45:33)**
Yeah.

Yeah.

**Adam DuVander (45:52)**
Companies that started on what what people will will call developer first or bottom up to move toward this more enterprise sales, top-down, you know, maybe developer last if we want to be real mean about the label approach. and yeah, so that’s a natural progression. And I think if you’re saying that, then

You kind of do understand where that audience is, but it means you are cutting off this rapidly growing definition of developer. And then at the same time you don’t have access to that senior person who is working on that side project who’s going to then bring that tool, which could be the best tool for for the job, into the workplace. So it does seem short-sighted, but I think.

At least understanding who that audience is, and you can’t appeal to anyone who would possibly use any tool ever. So you do have to sort of pick where those areas are that you’re focused. And and so doing that not sort of haphazardly and sort of waving off, but doing that intentionally, I think, is is really the real answer there.

and knowing who that is you wanna reach and then making sure that you’re in the places where those developers or builders are are working and and playing.

**Kate Holterhoff (47:27)**
So as we wind down this conversation, I am interested in your predictions for where this is all going. So as the foremost dev marketing expert here, d do you see developer marketing as as maybe I don’t know, it we’re we it’s definitely gonna evolve, but do you see that it maybe is gonna decline, become less important?

Or do you see that this is, sure it’ll reinvent itself, but in fact it will become even more important because, you know, now everyone’s a developer, or because, you know, hey, we we have to try harder, even SEO is even on the longer time side. All all the challenges that we’ve kind of addressed here. Where do you see it going?

**Adam DuVander (48:02)**
Right, yeah.

Yeah.

Yeah, so I you know, I think if you if you look at w at the things that I say are important in Developer Marketing Does Not Exist, it’s really pointing toward engaging developers. I mean the classic kind of dev rel tools, right? And so from what we’ve even seen just in the last few years, there was there was a little bit of devrel, that’s less important now.

oops, it’s actually the most important thing and our top thing that we should hire for. Like that’s the trend that we’ve noticed in this these last couple of years. And so it seems like that that trend would continue. Whether we call that role developer marketing, whether the that is the audience that you’re even marketing toward. I mean there are there are certainly other audiences that that you could pick in there whose problem you solve.

**Kate Holterhoff (48:43)**
Ha.

**Adam DuVander (49:11)**
But if it is a core developer problem that you’re solving, then yeah, I think that developer engagement is going to matter. And so yeah, what we call that, I don’t know. but but that that is important even in in this agentic world that we’re that we’re moving into. Because again, there’s someone who’s saying the things that are important who identifies those problems to solve, and then

There might be a whole team of agents that goes and needs the information and as we talked about, most of that list is still important today and maybe even more important to the agents than it was to the devs.

**Kate Holterhoff (49:59)**
Right, Adam. Well you have convinced me. so before we we sign off here, for folks who want to hear more, where are you directing them? What are your socials and what what sort of links can you share?

**Adam DuVander (50:11)**
Yeah, you you can find me at at LinkedIn for sure if you wanna chat with me. but then I think probably the best place to go and start to noodle around is llmrank.fyi.

**Kate Holterhoff (50:22)**
Fantastic. All right. Well, again, my guest today has been Adam DuVander. My name is Kate Holterhoff. I’m an analyst at RedMonk. If you’ve enjoyed this conversation, please like, subscribe, and review the MonkCast on your podcast platform of choice. If you’re watching us on RedMonk’s YouTube channel, please like, subscribe, and engage with us in the comments.