Charity Majors on AI, Determinism, Instrumentation, & Eating Your Broccoli

Share via Twitter Share via Facebook Share via Linkedin Share via Reddit

Get more video from Redmonk, Subscribe!

James Governor talks with Charity Majors, Co-founder and CTO of Honeycomb, about what AI is doing to engineering practices. Majors argues that non-determinism takes more discipline, not less, and every CEO who wants their AI cookies has to eat their broccoli first. Agents in production are finally forcing teams to do the instrumentation work they should have done years ago. They get into Chad Fowler’s Phoenix architectures and the idea that the system, not the code, holds the truth about how software behaves. Also: which problems AI has genuinely solved and which it hasn’t, why bragging about token spend is like driving a Hummer, and the last year of AI hype has been aimed almost entirely at individual productivity when the hard useful work is always about software development as a team sport. They discuss Honeycomb’s Canvas, the company’s AI-guided Observabiolity workspace. And finally they discuss the difference between performative outrage and actually artists in the AI era – and make some recommendations accordingly!
Honeycomb is a RedMonk client, but this video is free an independent content.

Links

Transcript

James Governor (00:04)
hey this is James Governor from RedMonk i am excited to be here because i have a buddy Charity Majors from honeycomb and Charity always has lots of great ideas many of which he even puts into production because test in prod or live a lie

But yeah, you have been writing a lot. You’ve obviously been thinking a lot about the impact of AI. And in fact, so much so Charity that we’ll sort of get to this. But I had a thought today in another conversation, I was like, you know what? Charity may, well, we’re not gonna get to it. I’m gonna do it right now. Thinking about the nature of the harness and where…

one lives sort of in the stack. I kind of thinking that Honeycomb, arguably is going to be moving up the stack and we’ll have sort of like an operational excellence harness rather than just being an observability platform. Because if we think about all of one thing that you and your organization have done, I think over the years, he’s just done such a good job of like codifying engineering, knowledge and culture.

take something like testing in production. That is something that you have written about, have made very clear, have made as a set of virtues and values that frankly could end up being something that a machine could read and use as the basis of inputs and outputs and run your systems in production.

Charity Majors (01:51)
I like where you’re going with this. I think we have increasingly come to the realization that people’s change budgets are so low because it’s all the change budget is being sucked up by AI. And so most people have just decided observability is a solved problem. Yeah, we know our solution is shitty, we don’t have any cycles to try and make it better. It’s three pillars, it’s metrics, logs, and traces. That’s what we’re doing. that’s so…

As you know, we have never really thought that that’s the right way to make sense of your systems. But you can only, you you need to go where people are willing to accept change. And I think that the next great frontier is knitting together determinism and non determinism. What does even mean? What is it? You know, what I love about AI is that it is

finally forcing everyone to do all the things that they needed to do all along but couldn’t get the investment time or what like instrumentation with traces the trace is the source of truth the trace is part of the product which is something we’ve been saying for How long is that this needs to be a product decision not an infrastructure decision? Yeah, and I really feel like I feel like agents in production are forcing the point on so many of these. It’s so exciting.

James Governor (03:15)
That is exactly right. think so your argument about production excellence. I one of the things is there are lots of dimensions to this. And certainly when I’ve spoken to you about progressive delivery, it’s been on the notion of like, there are so many things that you need to do. There’s actually a set of associated disciplines being rather than just one thing. And they are possibly multiple entry points, but there are so many things you need to do. But there is no doubt that AI is a forcing factor where

Look, if you’ve got shitty processes, poor documentation, all the knowledge is in people’s heads, and then you… And it’s very hard to get the benefits if you haven’t done all of that groundwork. And this is groundwork that there are many reasons that we should have done this. there are every few years, I think a forcing fact comes along, but it’s pretty clear.

Charity Majors (03:53)
to your AI if it’s all locked.

James Governor (04:11)
that you’re not going to be able to build, deploy, manage and iterate on software as effectively as your peers if they are doing a better job.

Charity Majors (04:25)
Actually, I don’t know if you saw the piece that I published at like 11 p.m. last night, but it was about… So a week or two ago, I wrote a piece about AI enthusiasts versus skeptics and got a lot of really great responses. And so there were two buckets of responses, technical and ethical. So I took the technical ones first because that’s easiest. basically, just arguing that non-determinism is going to take more discipline from us, not less.

going to take up a lot more engineering discipline. so all of these seats, which is not the direction, I think you’ll agree that the industry has been running over the past year. But my hope is that this is the year where we start to go, okay, every CEO wants their shiny, they want their fancy AI cookies. Well, discipline comes first, then you get cookies. Because that is like, this is what Dora says, this is what space, this is what all of the big engineering, you know, leaders and brands are saying is that

you need to bring discipline to get predictable results out and value is backed by durability. So my hope is that, I mean, this is an opportunity, whether we take it or not is kind of up to us.

James Governor (05:37)
Well, my friend Thomas Otter always said, you have to eat your broccoli if you want to get your ice cream. And that’s sort of where we are now, that people really need to start eating the broccoli. I mean, Lord knows they’ve been drinking the Kool-Aid. So.

Charity Majors (05:44)
Exactly!

Peace.

James Governor (05:59)
Yeah, okay, so good.

Charity Majors (06:01)
What the other thing I will say about that is…

This is the biggest opening we’ve had in my lifetime to bring our engineering values to the mainstream. They’ve all been out of reach of the median engineer because it’s been because code has been so hard to write and so time consuming and it’s taken up all of the oxygen. There are so many ways in which we can leverage this technology to bring it to the median engineering team because because the economics of code have just been upended. don’t know. you been following the stuff that Chad Fowler has been writing about with the Phoenix? Phoenix architectures.

James Governor (06:37)
I am gonna admit ignorance, which I hate doing. god, you’ve really, you’ve really got me there, Charity. I’m so sorry. And I’m certainly, you know, more than familiar with this work. So I’m that I have not been following this.

Charity Majors (06:54)
your reader for your listeners sake, I’ll catch them up on what he’s been. Chad Feller is the guy who coined the term immutable infrastructure back in 2013. Now I am just barely old enough that my first job when I was a teenager, my job title was system administrator. So I lived through the, you lived through the whole handcrafted servers to cattle, right?

of what was going on at the time, I was just a kid, but like I lived through it and I remember how grumpy the Sysadmans were and how it seemed like the world was ending. And I feel like there’s so much resonance to what’s happening right now with the handcrafted code because Chad has been writing about how

There’s so much bound up in lines of code. It’s the only record we have of what bugs have been fixed, what users expect, but the code is not the source of truth. The system is the source of truth. And that’s why every single API rewrite and legacy code, why we literally have to find these edges and these behaviors and these contracts by breaking them. It’s the only way. It’s miserable. And now with the economics of code being upended the way they are,

Now code is cheap. Now we should be able to have different artifacts. We should be able to generate the code from the architecture. should be able, there are all of the, listen, anyone who’s ever done a really gnarly, hairy database migration should be a little bit dubious of how far we can actually go with extrapolating and writing down users’ expectations and then developing the spec. But I believe so strongly that any movement we can take in that direction will be so good for the industry.

And it’s all about bringing discipline to our software. This is still an engineering problem, James.

James Governor (08:47)
Well, I mean, an engineering problem, I think people sometimes forget the engineering problems are of course people problems.

So, How far do you think, and it’s interesting because you’ve caveated there, you said anyone that’s done this knows that maybe it’s harder than it sounds. Because I think we are at the moment, there is a bit of a sort of, there were those that would literally be like, oh no, they’re all solved problems now.

Charity Majors (09:16)
I that’s adorable.

James Governor (09:17)
any problems to do with code are solved problems, right?

Charity Majors (09:22)
I look forward to seeing how their lives go.

James Governor (09:24)
Right. Yeah. mean, I’ve got to say, um, right. I don’t, don’t have to say, but I’m going to say, I do think that like, one of the shittiest things in life is maintaining software. Yes. Like it is not fun. And if, if, if you’re like, there is a view now, I think that we’ll use agents to build the software and then we’ll just use agents to maintain the software and therefore we’ll suffer no pain. I’m kind

Charity Majors (09:56)
I, extraordinary claims require extraordinary evidence. I, so I do think that there’s a part of why it’s so hard to talk about this is because there’s a sort of fracturing of reality, right? I think that there are some categories of software where that’s absolutely true. I think that the type, think that the software, the personal apps, right? Solved problem.

Absolutely. Personal productivity, toy apps and everything. I love this for us. But like Slack, anything that a billion users use, anything that has high expectations. So the interesting edges for me are wherever the world of bits means the world of atoms or cells or molecules or whatever. And so on the one hand, you’ve got persistence problems, famously difficult.

Those are not going to be vibe coded away anytime soon. other side, have user interfaces, right? And the last thing in the world that I want is to wake up every morning and see my buttons move around a little bit in my Slack app. just, don’t want that. are, trust is built through friction for the most part. And we want our friction to be stable, right? So I think that there are lots of hard problems that remain. I just think that

The last year has unleashed enough change that it will take us a generation or two to digest it, even if the AI developments stop today.

James Governor (11:30)
100%. Tell me about the balance between, and this is kind of interesting, I sort of wish that I had Kate here, because this is an ongoing sort of internal RedMonk debate. I’m very interested in your thoughts on what should be deterministic and like what should not and how that plays out. I thought there was a really brilliant talk last week at Fastly’s show where

was Accelerate London and they were talking about the fact that at least with their web application firewall, when they were developing, it’s like first pass use AI, but then use that use AI to generate deterministic code so that it is repeatable and performant, right?

Charity Majors (12:17)
Yes, that is the best of both worlds. Exactly. There’s, I mean, LLMs are brilliant for exploration, for prototyping, for experimenting, for like, I think one of the best things that has happened to us with the Advent of A, maybe even I’ll just say the best thing is it has forced us to ask ourselves again and again and again, what does good look like? What would better look like? What would great look like?

And you notice like the first generation of that debate was all about fast. Can we do it faster? I mean, sure, who wouldn’t want to wave their wand and do all their work in an hour instead of it? Sure, but it’s ultimately pretty boring, right? Because it’s the same work. What if it’s better? What if it’s fast? What if fast? Fast is often part of it, but it’s rarely all of it, right? And we’re not used to even asking ourselves that question. But I think that’s exactly right. The fast part is figuring out the fast part.

That has always been the slow part, right? We can do that much faster now. We can do it much better now. We can try more things. We can talk to more people. We can be more curious. We can try more prototypes. But when it comes down to something you want to rely on, you’re back to determinism.

James Governor (13:29)
I you know, like I’ve been saying that like, the best token is the one that you don’t have to burn. You know, it, just, it’s living in this world. I’m, I’m struggling a bit with this, the notion of just shoveling all of the money into Nvidia and OpenAI and Anthropic. mean,

Charity Majors (13:36)
Bye.

feel like I would respect these VibeCoders token maxxers more if they were bragging about how much profit they were making, not how profit they were shoveling into the maws of these giant companies.

James Governor (14:04)
Yeah, I mean, that would sort of make sense, right? Like this is what I have generated as opposed to… I don’t know. It’s so… And honestly, it is weird to me. I mean, I had a good conversation again. So I was talking to some folks at UK AI company, pre-LLM company called TurinTech and…

Charity Majors (14:10)
Is time for tokenminning?

James Governor (14:33)
I think that really interesting view that actually token maxxing should be about maximizing efficiency. This bragging about what we’ve spent is so freaking weird.

Charity Majors (14:43)
It’s like driving a Hummer down the street.

James Governor (14:45)
It is a bit like that, or just wearing the most expensive sort of label, gaudy thing that you could possibly have. I don’t, it’s a bit of a, yeah, I found that whole thing to be quite strange.

Charity Majors (15:01)
Yeah, quite gross.

James Governor (15:03)
And just, you know, especially when you make that a corporate standard. And we’ve seen recently sort of, you Microsoft saying, wait, we’re spending how much? And Amazon saying, yeah, maybe these leaderboards of how many tokens you’re burning are not a good idea.

Charity Majors (15:19)
I’ve been going nuts with the sticker machine and I’ve got one that says, Goodhart was here.

James Governor (15:26)
Sorry, I didn’t even get that.

Charity Majors (15:28)
I’ve been going nuts with my little AI ChatGPT sticker machine and I have a sticker that says Goodhart was here.

James Governor (15:34)
Oh, okay,

there you go. So, ah, there’s parts of me worried that we’re not gonna follow a line on this conversation, but I think that was always gonna be the case. So here’s the thing, talking about the difference between…

I think part of the problem that we’ve just sort of described is to do with the fact that so much of what LLMs unleashed

has been about individual productivity. It has been, here is a thing that I can make. this makes me, I mean, whether or not the AI is a God, it makes us feel God-like. And that, I mean, for people that have, know, a tendency to post this madness or something, some very interesting things are happening. But it’s just like, you know, like having superpowers.

And in some cases, you’ll begin to see this open source where, know, Mitch Wohashimoto recently said, why would you have other people in your project, right? You can build the whole thing yourself. Like, I don’t need pull requests. don’t. So, or LadyBirds, they said, well, a smaller team, just don’t need outside contributions anymore. But I think, and you you’ve spoken about what does good look like.

We think about the whole point of observability is the feedback loops and understanding the behavior of a system, understanding what it means for the user. I think that the, token maxxing comes from a point of view, which is actually about an appearance of individual productivity. Whereas really what we need to be focusing on, and I think where we as an industry need to go and, you said, you know, what would be great this year is if, is if we began to focus on engineering as a team sport and what is the role of AI there?

Charity Majors (17:45)
Yeah, I…

Yeah, it’s interesting. Part of me wonders if this is just all it. I think about the early programmers and I think they felt very similar to the way that we feel with AI. Just like, my God, it’s a magic world. I have superpowers. And there were some less than savory aspects of that, but I think it’s also just very natural. But it is sort of weird that we don’t really see people

doing work in public about how they’re cooperating, do we?

James Governor (18:24)
No, that’s, yeah, at least I’m not seeing it. This is how AI made me much better at collaborating.

Charity Majors (18:27)
Yeah.

If I’m able to do everything myself, why wouldn’t I want to? Well, I can think of a bunch of reasons. You might die. Do you want your work to live on? You might want to go on vacation. Do you want your work to be relied on by other people? You know, I mean, I think this sounds like kindergarten logic, but you know, we are a cooperative species and it is our superpower. This madness must come to an end, right?

James Governor (19:00)
I I kind of think so. One of the most exciting projects that I’ve seen in recent times is, and it’s largely because of the social aspects, is called npmx So it’s like a search front end on top of npm And it has lots of nice bells and whistles. It’s the kind of experience you would want for the front end of a package manager. Their aim is not to replace npm, but…

but certainly to help people to get more value out of it. And so we look at Mateo and Dan Rowe, who sort of initially led the project, the degree to which they leaned into collaboration was just really awesome. it was sort of interesting and weird, because it’s also the first, like, I mean, it’s the first great sort of

open source project of Bluesky. not, it’s not, you know, it happens to use atproto, but it’s not an atproto app in itself. It’s, it, isn’t, but they use that network to build a community. And so, and seeing like the number of posts I’ve seen from first time open source contributors going and being like, we’re part of this like npmx world now.

What I find fascinating about is this is the opposite of the direction of travel I’m seeing a lot of the industry moving. But I think there has to be room for learning and community and designing great experiences together.

Charity Majors (20:41)
Yeah, totally. I mean, have you seen Honeycomb’s canvas?

James Governor (20:44)
I have seen Canvas.

Charity Majors (20:46)
It’s basically somebody called it Google Docs plus, I don’t remember what the visual thing was, but like for collaborating on an incident, like I’m seeing this and you just drop it in, you drop a little graph and people are collaborating. Even in a world of agents, there are things that you want other people for.

James Governor (21:08)
So lean into that a little bit more, I think, given we’ve got you here, you do know a little bit about Honeycomb. And I have said that this AI thing and certainly engineering has to be a team sport. And we’ve taken probably 23 minutes to get to you actually talking about one of your products. So how did Canvas emerge? What was the specific thinking there? Tell us a bit about that.

I think the honeycomb AI journey has been like super fascinating because, know, I think for many of us, you look initially and one’s a bit skeptical. Then you’re like, this could be good. And then you lean into that. And I think now you’re at a new place in terms of you’re thinking about where it can take you as a company and where you can take it. So yeah, talk a bit about that. Like the sort of the team, the team sport of observability.

Charity Majors (22:07)
It actually feels like a real culmination of something that I’ve always wanted. I always wanted social debugging. was part of our earliest design discussions was how can we help people collaborate in middle of an incident? You’re all huddled. It’s 2 a.m. It’s dark out. Everyone’s asleep. And you and two or three of your colleagues are up trying to save the world.

Right? And it just, it’s so essentially multiplayer. But we were never able to, I mean, it’s not an easy thing to build and nobody was asking for it.

James Governor (22:48)
If I send enough Slack messages to enough people in the team and tell everyone else to fuck off, then hopefully good things might happen. mean, it’s, it’s, know, hey, I’ve seen, you know, like, yeah.

Charity Majors (23:03)
So I had nothing to do with coming up with it, which is why I’m like doubly, triply just like tickled because it fulfills so it’s this it’s so much better than I ever hoped it would be or ever could have been given the technology of 10 years ago. But I love because I think that it speaks to something that’s so real and essential about collaborative understanding. You know, I was having a conversation with Jessitron. Right after OllyCon, which I’m so sad you missed it was such a good event.

It was, it was, you know, I have always the events that I enjoy going to that I enjoy fighting. I don’t want to go sit in a, I can sit at home and watch talks. Talks are great. I’m always grateful that people give great talks, but like it’s about showing up to a place, meeting people, talking to them, meeting new people, hearing their stories. And I just felt like the whole thing had a buzz. Anyway, Jessitron and I were talking about.

in an era of agents, human contact is just so, I’m reconsidering, we’re an all remote team. I know I’m all over the place here. We’re an all remote team. And I hate that talking to my coworkers so often feels very non-distinguishable from talking to agents. And I’m looking for ways to really, I want talking to people to feel like people, right?

I have these foundational experiences at Lynden Lab, at PARSS, people that I spent all this time with. I don’t have the same experience of the people that I work with at Honeycomb. And Jessatron was reminding me of that old definition of consciousness. There is something that it is like to be a person. so when you’re talking about collaboration, I think where my mind is going is bringing the human back to collaborating like it shouldn’t feel.

Like it feels like when you’re collaborating with an agent. And this is what I love about Canvas. It does not feel like you’re collaborating with agents who are just like helpfully put. You’re collaborating with people who are people. There is something it is like to be them, right? And you have a mental model of each other and you can see where each other is coming from. This is hard to describe because it’s all very visual, but is that making any sense?

James Governor (25:19)
Yeah, I think it is. just wondering, I mean, I’m assuming you have some nice screen casted explanations that we can include in the comments below or in the description so that we can point people at some documentation about…

Charity Majors (25:41)
I’ll give you some videos, but I guess in summary, think teams should be looking right now for things to help them collaborate with each other that feel different from collaborating with agents because otherwise, what’s the point?

James Governor (25:58)
So we’ve got, got, we’ve got where does non-deterministic meet deterministic. And obviously you were doing that before LLMs, but, and then this, sort of interesting, almost counterintuitive. And I think it talks a bit to like, you talked about the points of friction.

Humans actually are points of friction. That’s sort of the point. And yeah, it’s kind of what we do. And for better for worse, it be incredibly infuriating. also incredibly powerful.

Charity Majors (26:22)
yeah.

what we do. And in fact, I’m working right now on a doc that we hope to share externally by the end of the month, but it’s AI norms and values. What are our working agreements? And when you say it that way, it sounds like, the subject of the sentence is AI, AI norms and values, but it’s actually us, right? The subject is us. And how do our norms and values need to adjust or adapt in an era of AI? Because technology always, it resets the bar, right? It changes what is expected of us. It changes what great means.

And to your point about communication, I’ve been trying to figure out how to write down. Yes, other people are friction. Yes. how do we, what expectation do we set about what we put in front of them? Like it’s our job to guard our own minds and attention. It’s also our job to protect each other’s minds and attention. So for example, if you’re dumping a clot doc at them, is that okay? Is it not? When is it? Should it be? I’m curious what you think.

James Governor (27:42)
Well, it is interesting that you should ask me because I too have been asking some questions about AI norms and values because

I RedMonk, I mean, I think we’ve been described in this way and I’m proud of it as sort of authentic. You know, we have a voice. I think our individual RICs have voices that we celebrate. We don’t expect everyone to sound the same. We’ve never, we’ve always actively not wanted to polish off.

the voices and make them sound like a voice. We don’t have any aspiration to have a single voice.

Charity Majors (28:37)
with your market.

James Governor (28:39)
And I, yeah, I don’t think that helps credibility. so, know, AI does tend to rather sandpaper things off. And so I think, you know, we’ve got some, as a firm, some really important questions here because authenticity, trust, credibility, are the things that make us valuable. Now,

Are there some things that AI, LLM is profoundly good at? Yes, absolutely. So my colleague, Kate Holterhoff is like, let’s look at the current open source policies and license changes across a number of projects. You could literally spend weeks going and reading every single policy in order to extract.

any diffs there have been or what the current state of the art is. And she was able to do that work supported by AI very quickly and effectively. Stephen O’Grady, my business partner, I don’t know, I mean at the moment he is on some, I don’t know, he’s had his Wheaties every single day. I mean, he’s writing up a storm.

He’s using AI a lot for data-driven analysis, but I know that Stephen hand writes every single line on his blog. you know, well, he sees himself as a writer. I see myself as a writer. We think that’s really important, writing is thinking. And so outsourcing your thinking can be suboptimal. And I think the point about friction here,

is that we should be willing and able. I mean, look, it is great the idea that we will write something so polished that it could not have been any better. I mean, there are great stylists of language and you’re like, you know, my God, you know, if only I could write like Hemingway or, but actually sometimes, and this is something I have seen repeatedly in our years in this business is sometimes it’s the throwaway post. that generates the most discussion.

Charity Majors (31:05)
Yeah, it’s impossible to predict.

James Governor (31:07)
You can spend all of your time trying to create the most polished thing, but then there’ll be one stupid thing you said or you thought it was stupid and that will be the thing that people are like, that’s the really good stuff. So I think the points of friction is giving like

Honestly, the reason that we first, Stephen and I began to write blog posts was that we were wasting too much time editing each other’s work. We would go back in this weird dance. was like, dude, and at the time it was like, are two of us. We just, we don’t have time for this. And eventually we said, you know what? We will publish and be damned and let the internet come to us. And that will be part of like, people can comment.

And obviously the golden era of RSS, RIP, my God, good times. But our view was, yeah, that actually it’s those gnarly bits, those threads that people can pluck at and pull at. That’s where things become more interesting. That’s where you learn more. That’s where you’re collaborating and designing collaboratively. That to me, I think is worth celebrating, fighting for, enabling, finding ways to continue doing in the era of AI.

Charity Majors (32:25)
I think it’s fascinating how fast and how decisively the tables have turned against anything that sounds like it was generated by AI. It’s like for a few months there, people were intrigued by it. that sounds snappy. that sounds sharp. that heightened contrast. And then all of a sudden, just like, I hate you. Like this really emotional, you’re telling you to go fuck yourself. You’re like, you would show, you would put AI at me, bitch.

It’s been really violent and satisfying. I am not immune. I too am feeling that. But I think the lens that I am looking at this through is, so here’s my working theory. Here’s how it’s currently in my head. I haven’t written this down yet, but I think that when you’re at work, there are a couple different lanes that you’re in when you’re communicating with other people. There is, you’re doing the work.

and the work needs to be high quality. Not perfect, but you stand behind it. There are times when higher quality is called for, like if you are doing an architecture decision record, or if you are proposing a product thing and it’s a doc and it’s like, this is kind of like, it represents you professionally. You should stand behind every single comma, regardless of how it was generated, whether you typed it yourself or whether Claude did it, there’s no room for, that was Claude, not me. There is zero room for that, right?

And, and, but people should have to deal with the document, whether or not they like that it sounded like Claude wrote it or not, right? Because it’s about the work. It’s not about in the same way that like someone’s spelling errors might make you kind of, but it’s about the work, right? It’s functional. And then there is relational communication. Sometimes with a power dynamic, sometimes with a peer, um, giving a performance review or when you are giving someone hard feedback or something. And in relationship.

communication lane there is no tolerance for it looking like someone pushed a button and in fact I think an acid test of the communication should be if the person receiving it knew exactly how you created it would you be proud? Because I think that what’s the difference between your manager telling you giving you some feedback versus you reading it in a book it’s that they know you.

They know you and they’re hopefully invested in you and they want you to succeed. And they also have power over whether or not you get a raise or not. Right? So like it is appropriate. There are ways to for managers to use AI to in lots of really great ways. But when it comes down to the communicate, the relationship can’t be outsourced.

James Governor (35:06)
Yeah, no, I think that’s exactly right. I mean, my God, I mean, obviously,

For me, Charity, I’m like, that phrase, the bitter lesson, you know, just about just throw more compute at it and it will be better than any of the other things that you tried in essence. Over time, compute will win. No, the real bitter lesson are all these fucked up bitter lessons that we’re currently facing. Like, the bitter lesson is your…

you know, your frontier model bill at the end of the month, the bitter lesson is that actually, much though we might think we don’t need to communicate, we do. so I think, you know, we’re sort of, yeah, the bitter lesson is there are a lot of bitter lessons and I don’t think they’re all about compute. I think that we’re, as you say that, I mean,

uh yeah just the bitter taste of shitty prose and and it’s interesting because the first one it was visual imagery and i have to say so my daughter she is like super anti-generative ai she is uh you know going to be 16 in a few days she is amazing um you know she is creative she is smart she is committed she’s like amazing and and just

hates generative AI. She has caught me. She has caught me once using it was for a talk on prompt engineering and she caught me. I was like, well, I’m going to prompt an image for the she was super mad at me. She said, dad, I will, I will always draw you slides if you ever. But the yet like you talked about the revulsion. And it just was so it’s it for whatever reason it was so it’s so clear with visual imagery. It just…

Charity Majors (37:13)
I’ve been going to town on I have been learning visual concepts and I’ve always wanted to learn art stuff and I okay

James Governor (37:23)
Are you gonna shoot all of that down and be like, my stickers are now better than they’ve ever been?

Charity Majors (37:27)
They are!

James Governor (37:30)
I’m

shit out of them! Yeah, no, there is-

Charity Majors (37:33)
typography and visual languages and like color balances and all these things and I’m and and like

James Governor (37:40)
But Charity, thing is, most people are not doing that. They’re just generating slop.

Charity Majors (37:44)
Well, that’s always an option with…

James Governor (37:47)
I

know, and of course Slop

Charity Majors (37:50)
here before AI. I have been really on it. So for some reason, it’s a lot easier for me to go on this journey with art and stickers than it is with code. Like it’s just too close to home, right? So I’m chewing over, I want to write a piece about this, about how I learned to love the machine and, also like, look, if you support artists, support artists, you know, like, I think it’s a little bit ironic that this debate has gotten distilled into,

the big bad AI companies versus the tiny little artists when it’s actually more like big bad AI companies versus big bad media companies. I’m like there are no virtuous actors in this.

James Governor (38:27)
I mean, you know, yeah.

Charity Majors (38:29)
If you support artists, fucking support them.

James Governor (38:32)
Yeah, absolutely. mean, know, yeah, ASOS and H&M have been stealing designs.

Charity Majors (38:40)
Yeah, so this is the thing with AI. There are lots of good reasons to hate AI, but I mostly see people hating AI for bad reasons. Reasons that are so clearly retroactively justified. They hate AI, that’s the starting point, and then they come up with the reasons why. And the reason it’s so obvious is because AI is not the only offender in any of these categories. It’s the largest offender in these categories like data centers and communities or art threats or whatever.

So like, feel like it delegitimizes the real pointy critiques of AI when we’re so-

James Governor (39:16)
Who is a small, who is an independent artist that Charity Majors thinks people should go and look at their work and possibly buy a sticker roll? Name a book.

Charity Majors (39:28)
first name that comes to mind is, it’s not visual, it’s a book, but I’ve been meaning to write. Kara Voorhees Reynolds wrote this amazing book called Priestess and then a sequel, Illuminator and Pilgrimess.

James Governor (39:43)
That

Kara Voorhees Reynolds Okay. this is fiction.

Charity Majors (39:45)
good.

fiction, it’s fantasy, and it’s got a, it’s got like middle-aged female characters who are recovering from sexual abuse. I’m not making it sound amazing, but it’s like poetic and beautiful and funny and this is what this is why I’m not in marketing, but they’re so good.

James Governor (40:06)
I mean, mean, you know, I, the very awful truth is that there are not many women that are middle aged that have not faced, mean, I don’t know what, kind of survivorship bias you have if you’ve emerged into middle age and you have never

Charity Majors (40:30)
Right, Or just how oblivious you really are.

James Governor (40:33)
views in some ways so…

Charity Majors (40:36)
I’m making it sound like an after-school special but but that’s the point is it’s not an after school It’s just part of life. It’s just one of many things that you deal with and it’s lovely

James Governor (40:45)
And will I enjoy these books?

Charity Majors (40:50)
I think you would love them. Anyone who loves fantasy adventure books.

James Governor (40:53)
love them. Okay, okay. So that’s a good one, supporting the artist. I’ve got a personal… So I’ve got a friend that has done the iconography for my last couple of MonkiGras conferences. She’s called Gail Myerscough. She’s got a really particular way of seeing the world, of like 1960s, very bright primary colors, loves the color orange. She has some really nice postcards of like…

Brutalist Architecture also likes to do drawings of like record players or you know old tape machines or stuff like that and she’s awesome so you know if you’re listening go and you know maybe check out Kara’s work, check out Gail’s work. Gail’s amazing!

Gail does like prints and everything, but she will do corporate commissions and everything else. She’s amazing.

Charity Majors (41:58)
Gail, Gail, what’s her name?

James Governor (41:59)
Meyers-Koff. M-Y-E-R-S-C-O-U-G-H. She’s just got a very particular sensibility.

Charity Majors (42:06)
Thank you.

Love it.

James Governor (42:12)
And so, yeah, I’ve been trying to do a better job of that Charity, partly because, you know, because when my kids were saying how shitty AI was generated, generative AI, and I’m like, well, I’m dealing with it in my domain. But one of the things was, but and to a point, capitalism, you know, it’s gonna, but I was gonna be like, but you’ve all got ad blockers on.

which means that artists are not being paid for their work and they were like, dad, that’s Google, fuck off. yeah, trying to make sure that I buy a print or I recently bought my daughter a print, you know, just something on an ongoing basis to your point. If you to support artists, then support artists.

Charity Majors (42:59)
I think that affirmative action is always better than attempting for purity by not doing it.

James Governor (43:05)
things.

Yep. And also, my god, see we’re well off the track here, but one thing I do think that is little to your point, is the performative anger.

Charity Majors (43:19)
my god!

James Governor (43:20)
If we’re, we’re, use Bluesky a lot. So I know about this. It’s when we’re, more, we’re more, we spend a lot of time like articulating our rage about instead of just doing the thing. you know, and this is, think, well, yeah, it’s not, it’s not super great. The, the

Charity Majors (43:34)
Doning our friends.

I can’t stand Bluesky. It’s too much of the I’m angry about this today and I’m angry about this person who is too miliseconds to like just slightly off from my position, but I’m

James Governor (43:59)
I think that’s a problem for the network. But it’s super interesting, actually. think the atproto thing and seeing the alternative Bluesky, there’s a Euro sky, there’s black sky, which is like, they moderate as they want for the black community. And it has a lot of weird and interesting, I’m sticking with it, but I know that being scolded is not a particularly great feeling.

And just being performatively outraged. I’m trying to, I’ve always, yeah, I mean, look, there’s a reason I do what I do as a researcher. I’ve always wanted to shine the light on work that is good. know, complaining about stuff has its place. But also, you know.

Charity Majors (44:46)
vinegar in the sauce, you know, but you don’t want a vinegar sauce.

James Governor (44:49)
Absolutely. Okay, so I think we should, what, you’ve got some further publishing coming up. I’ve got this like mad vision of you, of Honeycomb creating, as I say, a harness for operational excellence going beyond observability. So that’s just, we’ll see what happens, but I’m just imagining, you know, all of the ideas you’ve got about the linkages between observability and everything else.

are codifiable and can begin to…

Charity Majors (45:26)
Harness is the cool word of the day.

James Governor (45:28)
It abscise, I don’t know what else to call it, a knowledge base. Knowledge management is back. Knowledge management is 100 % back. Oh yeah.

Charity Majors (45:36)
Do you think that Salesforce just bought Fin?

James Governor (45:39)
I Everyone is, mean, management, ontology is the word, you know.

Charity Majors (45:46)
Everything is ontologies.

James Governor (45:48)
So, you know, I don’t see why we can’t have like a honeycomb operational exon ontology, but I’ll banging that drum. You’ve got a couple of other publications coming, but what other than Canvas? Yeah, like what’s super exciting about, or what is exciting you about where you are, a honeycomb right now, and like for the, if any readers, watchers, whatever, manage to put up with us for this nearly an hour of Monkchips and Charity Show. Tell us about some Honeycomb stuff to finish up.

Charity Majors (46:27)
I don’t know how it so one of the things that I’m most excited about is the timeline analysis because the transaction is no longer the source of truth and you need to be able to Decide I want to start my session here and end it there wait, that’s not where it ends It ends over here and you need to be able to trace any amount of stuff so one of the things that honeycomb really shines that is and this is the thing we’ve never learned how to really market, but it’s the

going from different altitudes, being able to go very high level all the way down to very individual requests at any point in between. Like we’re the only tool that can do that. And timeline analysis extends that horizontally in time as well. And it is super cool. We’ve also got some really exciting auto instrumentation stuff. that I feel like for the first time ever, it is true that you can use instrumentation to write faster, not slower.

OTel, the common patterns are all trained by the LLM stuff. we’ve got Claude skills. You just hook into Claude skill and as you’re developing all of the honeycomb traces, we’ve got some really interesting agentic stuff. But I really think the canvas and timeline analysis are, they just, make, let people’s eyes light up.

James Governor (47:52)
Awesome, love that for you. you if you say that the timeline stuff is something that you, you know, have never so far marketed effectively, I’m, you know, I’m a big fan of your, well, relatively new CMO, certainly Julie Neumann, amazing. So I reckon you’ll fix that. High hope for you on that.

Great, okay, so those two specific things. And the thing you’re right, harness is sort of a bullshit term, but the harness to me, one of my definitions is like, harness is all the things where you say no, not like that. So you’re the model and then the harness is all the things you’re like, no, not like that. It’s actually supposed to be like this. So it’s like, it’s just that all of that sort of, I don’t know what to call it, ligature or whatever.

Charity Majors (48:51)
Ligature is a good word.

James Governor (48:52)
But no, not like that. It’s just like to hold it all together. Once you go out of alignment, you snap back in. And that’s, I think, and discipline, and that’s what comes out of doing the work, scraping your knee, bashing yourself on the thumb, and all of that sort of stuff.

Charity Majors (48:59)
discipline.

You’re

a great teacher.

James Governor (49:17)
That is, yes it is. And that’s where, that’s what I look forward to and I think will begin to scale. your culture even further going forward. And that was why I used the term harness.

Charity Majors (49:37)
I think it’s absolutely, it’s very of the moment, but it’s a good word.

James Governor (49:42)
I mean, you know, love language, happy to see it keep on turning over. As I say, although we’re, yeah, knowledge management, was a lot, that was, I think that was before you were born, Charity. That was a long time ago that the industry was on that thing, but definitely back. Okay, so that was a MonkCast conversation, RedMonk talking to Charity Majors.

of Honeycomb and yeah, hope you all enjoyed it. If you have, then like, subscribe, do all of that stuff, share it with your friends, check out Canvas, think about timelines. Of course, it being Charity Majors, it’s always about those traces and spans. And on that happy note, thanks so much for joining us, Charity, and I’ll hopefully see you in person soon because it’s been too long, my friend.

Charity Majors (50:38)
It has been far too long.

More in this series

Conversations (135)