RedMonk’s Kate Holterhoff sits down with Chris DeMars, Senior Developer Advocate at TuxCare, for a conversation about patching in the AI era. Chris started writing code in Q Basic in the mid-90s and now spends his time at conferences explaining the JavaScript supply chain to people who’d rather not think about it: typosquatting, the Shai-Hulud worm, and getting locked out of his own VS Code editor at Vueconf. They get into why a Dockerfile that Claude wrote him pulled in an insecure version of Node, why most enterprise customers are nowhere near migrating off end-of-life software regardless of the modernization story being sold to them, and what rebootless live patching actually looks like once a CVE drops.

Links

• LinkedIn: Chris DeMars
• X/ Twitter: @saltnburnem
• Bluesky: @chrisdemars.net
• Chrisdemars.net

Transcript

Kate Holterhoff (00:04)
Kate Holterhoff here, and with me I have a very exciting guest. We’ve got Chris DeMars, he’s a senior developer advocate at TuxCare. Chris, how ya doin?

Chris DeMars (00:14)
we are six feet above ground and ten toes down, so can’t complain.

Kate Holterhoff (00:20)
Living the life. I know it’s almost the weekend too. Are you going to be fishing?

Chris DeMars (00:21)
Yeah.

I am. I have a derby tomorrow. I have a lot of prep I have to do for it. so my my club I fish is a co angler, so I don’t use my boat and I had to take my boat in anyway. So like there’s that. That’s a three week turnaround. But yeah, first club event of the season is tomorrow. So you know, it depends. ‘Cause like so we’re getting massages today after work. So like I got that to look forward to.

Kate Holterhoff (00:42)
How you feeling?

Chris DeMars (00:51)
But I still have to like get things together and prep all my stuff. And then I gotta wake up at four in the morning to go to this lake that’s an hour away. So I’m not looking forward to that because I’m not a morning person. And the other day I pulled something in my neck and my shoulder and like it just it’s killing me all through my neck, through my shoulder, all through here. Like I’m in so much pain. So hopefully that works out and maybe the massage therapist will help help me out. But besides that, we’re good. It’s beautiful day here in Michigan and can’t complain.

Kate Holterhoff (01:01)
No

Yeah.

Yeah.

Okay. All right. Well, it does seem like fishing is shoulder intensive work, so I hope that they get you sorted for your big day tomorrow. okay, well, very cool. all right, well, I have invited Chris on here to continue my, I guess, long-running series on CVEs and where we’re at with security, and AI. So Chris and I both attended Vueconf here in Atlanta. got to talking about some of the stuff that you guys are doing over at TuxCare. So

Chris DeMars (01:25)
Yeah.

Kate Holterhoff (01:49)
thought I’d have him on to to chat a little bit about this. So let’s begin maybe with your background. So introduce yourself or folks who who who don’t know who you are and what you do.

Chris DeMars (01:59)
Yeah, so I I’ve been a front end developer for thirty plus years. I started off writing I think I started off writing code in like ninety-four, ninety-five. So yeah, a long, long time. Started off with Q Basic, then moved into HTML, and then kind of did that off and on for so many years. And then I I got my first gig in two thousand eight as a contractor, an independent contractor for a local company. And then fast forward to like the twenty teens, I got an official official job.

Kate Holterhoff (02:08)
Wow.

Chris DeMars (02:29)
with benefits and kind of just been going at it ever since. I’ve sort of been in the DevRel role, officially, unofficially, probably since the twenty teens. I got started off in the Michigan or the Detroit tech community. We had meetup we used to have meetups all the time. And kind of always did the speaking thing and then officially got a title of developer advocate s six years ago. So then I finally got started getting paid to do the job. So and now here we are a senior dev advocate at Tuxcare.

Kate Holterhoff (02:59)
Amazing. and you’re involved in a lot of conferences as well, right? as an organizer.

Chris DeMars (03:03)
yeah.

Yep. Yep. So I help to what do I do? I I run a meetup here in Detroit and then I’ve ran helped to run other conferences and meetups in Michigan. I I sort of I say I work for K C D C I’m on their speaker selection committee. I used to work for Codemash on their speaker selection committee. So I kinda keep my hand in all things community.

Kate Holterhoff (03:26)
Okay. Yeah. And you know, it’s a it’s a a worrying time in developer advocacy. a lot of DevRel folks are are looking for work right now. It seems like a story that I keep hearing from folks like you in the industry is that they are just deeply involved in community in this organic way. sounds like that’s kinda how you moved into this role. are are you hearing that as well? Like what’s your sense?

Chris DeMars (03:48)
Yeah, I think the people who move into DevRel, they’ve always just had a community knack to them, right? Whether that was writing or creating their own video work or just, you know, speaking at meetups, loving a product and just being super technical with it, if it’s a super technical product. I think it just it just kind of makes sense to move into that. And especially if you’re an extrovert. Like I know there are some DAs out there that are introverts and they put on like an extrovert facade when they have to.

Kate Holterhoff (03:55)
Yeah.

Chris DeMars (04:16)
But being an extrovert really helps too. And if you like just being around people and I love being around people and I love to teach and mentor and learn and it just makes sense. Yeah. A lot of travel.

Kate Holterhoff (04:24)
And travel too, right?

That’s big part of it. okay, interesting. All right, so let’s dig into this So I was interested in the talk that you gave at Vueconf. it was horror themed, as is right for you, Chris. You know, you always have sort of a a good th that that’s sort of your thing. but what was interesting to me about your your abstract is that you talk about typosquatting, dependency confusion, supply chain attacks.

Chris DeMars (04:38)
Mm-hmm.

Yeah.

Kate Holterhoff (04:55)
And that they’re haunting the JavaScript ecosystem specifically. So, your front-end chops certainly makes that be the case. you know, I when I’ve talked to other folks about the CVE issues, they haven’t talked about JavaScript specifically. So where does JavaScript fit into all of these new horrors, if you will, around security?

Chris DeMars (05:03)
Mm-hmm.

Yeah.

So it w we see it kind of all over the board in all different stacks and languages and libraries and frameworks. Like that’s what Tuxcare does. We provide patching s patching services for anything open source when we talk about endless lifecycle support. So that’s libraries, frameworks, applications. So stuff in Java, Spring, we do PHP, Python, we have a couple a couple solutions for dot net, and then a lot of

That too is heavy on the JavaScript side. So we support all different versions of Angular, AngularJS, pretty much everything in AngularJS and everything in Angular up to version nineteen. We support a couple versions. I think we support one or two versions of Vue. And then we just dropped support for Nuxt 3 the other day. we support stuff with MagoDB and then the database side of things, even Postgres. So it kind of just goes all over the board, but

Kate Holterhoff (05:52)
Mm.

Chris DeMars (06:08)
a lot of the supply chain attacks that we’re seeing recently, they’re all stemming from NPM. And NPM, for anybody that’s not aware of what NPM is, it’s a node package manager, right? So it’s the way to install program not programs, but projects and utilities and tools inside your applications on the front end, which is mostly all JavaScript. And then you got your HTML and your CSS pieces too, but your JavaScript is the one that does kind of all the cool different things and in the function and stuff like that. So

You see a lot of those NPM attacks stemming from people inserting malicious software into already existing projects or people creating typosquatting is a great example. creating application or creating projects, putting them on NPM, they’re not vetted out, and then someone mistakenly downloading or installing the wrong thing, and then you get your NPM attack.

Kate Holterhoff (07:01)
it. Okay. Yeah. I mean I and and so TuxCare specifically, I tend to think of them as dealing more with the Linux kernel, but they’re doing a lot with the JavaScript ecosystem. Is that kind of a new thing or is that you know, has that just always been part of it?

Chris DeMars (07:13)
So I think they’ve

been doing ELS for a few years now, ’cause the one main big product that we do is kernel care, which is library bootless patching for Linux distributions. And AlmaLinux was a product of ours and AlmaLinux kind of split off to do their own thing. But we offer enterprise support for AlmaLinux. So it’s a heavy Linux background, yes, but moving into the patching of all different open source systems.

Kate Holterhoff (07:17)
Okay.

Yeah.

Right, right, right. Okay. And so what’s it like working at a company that does so much with Linux as a front-end engineer? Do they like you have they, you know, taken you under their wing and been like, it’s okay, Chris, you belong here.

Chris DeMars (07:49)
I I stay out of that. I stay out of that area.

Like I I I know how to use Linux. I mean Linux is under the hood of a Mac is is you you know it runs Linux. But like I used to run Ubuntu and Lubuntu back in the day and just various distributions. So like I know my way around a Linux distribution and you know, a Linux you know, the the terminal and and running Linux commands and stuff like that. But the whole Linux side of things, we have extreme professionals that understand

Kate Holterhoff (08:08)
Okay.

Chris DeMars (08:18)
like that infrastructure side of things. So I let them handle it.

Kate Holterhoff (08:22)
All right, fair enough. Okay. All right. Well, I mean when it comes to what we hear in the news, couldn’t agree with you more. NPM is is everywhere, Shai-Hulud all of these social engineering attacks. I mean this is big stuff. So it makes sense that you folks are f kind of honed in on the problem of, JavaScript and security.

So I guess I’m interested in where AI comes into this. So what are you seeing in terms of like how AI is accelerating this? can you get into any more like granular detail? I think everyone knows like arms race, but like yeah, what what’s actually happening?

Chris DeMars (08:54)
Yeah.

So I think using AI, if you’re not if you’re just throwing AI at a project to help you build it out, it’s gonna come along with some issues. Like there are a handful of people out there that are talking about the problems with security in AI, not AI creating secure applications. I wrote a blog post last year on what was it on? I wrote a blog post on creating a Docker file or I don’t remember. It was something. So it was a security blog post that I wrote.

From a personal blog. And I I don’t have a background using Docker Kubernetes and like this the server or like the engineering side of that that piece. So I had Claude write me a Docker file because I was playing around with Docker and I wanted to kind of set up some environments. So Claude wrote out a Docker file for me. And come to find out, the version of node that it was pulling in was unsecure. Now this was this was straight from Claude. Claude gave me this Docker file to use.

And so I opened up in VS Code and there’s a squiggly line under it. I’m like, what the hell is this? So I hovered over it and it said that, hey, there are known vulnerabilities for this specific version of Node. Click this link to go learn more. So I clicked the link in VS Code. It opens up Node’s website with their like CVE dashboard. And it was talking about that specific version of Node, how it’s not secure. And if you’re using it, you need to upgrade or just replace it with this other version. I did that in my code, it fixed it.

changed it and then there was no no vulnerabilities with that specific Docker file. So I mean you can you can still get into issues like that, especially if you don’t know what you’re reading or you’re not staying kind of ahead of of what’s going on in the NPM world and the JavaScript security world, you might not know, right? Or if your your code editor isn’t set up to catch things like that, if you’re not running NPM audit or yarn audit, you’re just gonna not you’re not gonna be able to see these things. So there is a big piece to that. And I

I’m still trying to catch up on the whole Mythos Glasswing thing, right? I think it’s all through anthropic and how like the identified, like their AI tooling identified like really old CVEs and vulnerabilities that are out there for projects that like nobody has even identified or discovered. And it’s causing a lot of issues across the board, right? So AI, I always look at it like Skynet, right? Do we really want Skynet to happen? So

Kate Holterhoff (10:55)
Mm. Yeah.

Mm.

No don’t think so. jeez.

Chris DeMars (11:19)
No, exactly. And we could be going that way. I mean, I don’t know. I was at

a conference in San Jose and I they had robots there and I’m like, this is like seeing it live in person is is totally different than watching like a YouTube short or something of like those little robots and like the little dogs that run around. It’s it’s a little trippy to see it in real life. No.

Kate Holterhoff (11:38)
Yeah.

I’m not ready.

yeah. okay. Well tell me this. So there you know, there’s been jokes that developers don’t care about security, where it’s like don’t want to think about it. And and it’s not till you get like a real enterprise job, right? You know, to wear a suit that suddenly developers are like, geez, you know, I’m gonna do stuff with compliance and governance and all this. Do do you think I mean you can disagree with me if you if that’s not what you’ve if if that’s not your your supposition, but

Chris DeMars (12:07)
Mm-hmm.

Kate Holterhoff (12:08)
Is AI changing that? Are developers suddenly like, Whoa, security. Well that matters.

Chris DeMars (12:13)
I th I I mean I think so. But it’s surprising how many people that I’ve I’ve spoken to and that we’ve spoken to at conferences that are unaware they’re running end of life software or that they have vulnerabilities in it or like they’re using Claude or Codex or whatever to vide code their systems, yet they’re not being cognizant of what is getting pulled in. And if they’re running like, hey, yeah, we’re running this old version of this or old versions of that and we don’t we’re not familiar with end of life or what ELS stands for.

Kate Holterhoff (12:15)
Yeah, okay.

Yeah.

Chris DeMars (12:42)
There is a big push, I think, now to start more start worrying more about security because AI. I I also say too, I talk about this with friends. Back in the day we used to read books. Like you I’m you know this. You’re a doctor. We used to read books, you know. I have a huge library of web development books. All AI is, it’s just grabbing old reference material.

Kate Holterhoff (13:01)
Sure. Yeah, read a few.

Yeah.

Chris DeMars (13:10)
from digitized books and blog posts and y in in and tutorial sites and stuff, right? It’s grabbing that kind of stuff and just throwing it into your code base. So you gotta really like, if it’s gonna pull something that’s a couple years old, you know, now that it’s end of life, you really have to be specific with how you’re using the AI and prompting it and and being extremely, extremely specific.

Kate Holterhoff (13:10)
Mm.

Chris DeMars (13:36)
At least from what I’ve heard and what I’ve been learning, try and use it for some like little side projects.

Kate Holterhoff (13:41)
Okay. Well that’s interesting, ’cause that’s like a whole culture shift then. But yeah, that makes sense about the the fact that developers maybe but it’s almost becoming an education issue. It’s like maybe you know, they might care, but they don’t know that this is even a problem. And frankly, all of us have whiplash. Like nobody can keep up on all the different things going on. As I’ve been writing about CVE’s, I feel like I learned about two extra for every one that I’m trying to investigate. I’m like, well Jesus, I didn’t even know about all that.

Chris DeMars (13:53)
Yeah. Right. Yep.

No.

Kate Holterhoff (14:10)
so yeah, I mean i even folks who ostensibly are are experts in this are are are still trying to like play catch up. So that all makes sense. So so there’s an education aspect. there also seems like you know, creating what? making the right thing to do, the easy thing to do. I’m thinking of like Vercel. I’ve got some little hobby projects on there. And then all of a sudden they like, no, no, we can’t update this because you’re on the wrong node version. And then they made me update stuff.

Chris DeMars (14:17)
Mm-hmm.

Mm-hmm.

Kate Holterhoff (14:37)
So I feel like there’s

Chris DeMars (14:37)
Yeah.

Kate Holterhoff (14:37)
ways that we’ve got like golden paths being integrated. I don’t know, what what’s what’s your sense there? Is it like do you have to like hold the hand of these developers or are you doing it more as like a gentle like a carrot situation rather than a stick of like here here’s some resources.

Chris DeMars (14:50)
Yeah, I mean I always I kinda resort to you you can lead a horse to water but you can’t make you know make make it drink. You could talk about it all day long. You can hold hands with people, but unless they take the time and effort to to upgrade their systems or, you know, use a company like ours or even just investigate things. They’re just they’re not gonna do it. And why is that? Because it eats up time. And like I said in my talk, like we don’t care and I’m I’ll throw myself into that too, we don’t care about

Kate Holterhoff (14:56)
Yeah, sure.

Mm. Yeah.

Chris DeMars (15:18)
performance or accessibility or security until shit hits the fan. We just don’t. And there’s multiple reasons for that. You know, it’s it’s resources or it’s or it’s time and money and Jira sprints and and deadlines and and all these various little factors that come into play. We’re we’re not always thinking about that, right? Like, we got to get this feature done in two weeks. Well we got to worry about we have to worry about security, performance, and accessibility of this feature.

Kate Holterhoff (15:21)
Mm. Yeah.

Chris DeMars (15:46)
we don’t have time for that. You know, we we gotta just get it out the door. So that’s when you get into the whole cutting corners thing and I’m not a fan of cutting corners, so

Kate Holterhoff (15:54)
Well, CVEs are new. These have been around for a while. so, when you say that patching is more important than ever, with the mythos and all of that, what specifically has changed would you say? Is it just the acceleration or is there anything else that you’re you’re you’ve detected as as being like part of this this big

What seismic shift in the industry?

Chris DeMars (16:18)
Yeah, I mean I think I think there’s really more awareness around it. now more than ever. ‘Cause it seems like every day we wake up there’s a new NPM attack somewhere that is now just coming to fruition. When I really didn’t see a whole pattern of that prior to kind of the for Mythos and and Glasswing and and all these other shifts in security. I give you what asked me last year.

Kate Holterhoff (16:22)
Okay. Awareness, huh?

Yeah, yes.

Chris DeMars (16:46)
what what what pattern do you see with NPM attacks? I’m like, well, I’m not gonna see anybody really talking about on LinkedIn or Bluesky or you know, Twitter. And now with the advent of this this other AI tooling that has just recently come out, now we’re seeing hey, there’s this NPM attack and there’s this NPM attack and this happened the other day and this happened the other day, and there’s a new Shai-Hulud worm and

You’re just seeing it more and more and more. NPM was hacked or this package was installed and it was an extension and that happened at Vueconf. I try I I found out about that NPM VS Code attack and I couldn’t open VS Code. Like I closed it just to like do something and I couldn’t open it again. I’m like, shit, this isn’t good. Then eventually a couple hours later, I fired up VS Code and it it started, it upped

It upgraded itself, it did a bunch of other weird shit and I like, maybe this was unintentional and they just did like a full reset, you know, at GitHub, so or or Microsoft, yeah.

Kate Holterhoff (17:48)
man. Yeah. it it does make you wonder. because I’ll tell you what, I’ve been doing a lot of updates on everything. we’re recording this on Riverside, and before we recorded this, I had to update my my little app here, my desktop app. So, you know, you do wonder if it what the the rationale is going on behind the scenes with a lot of these companies, all your SaaS products. Okay. Well that’s yeah, that’s good. okay. So the CVE’s thing is interesting. because it’s it’s kind of a

Chris DeMars (18:05)
Right. Yep.

Kate Holterhoff (18:15)
A specific way of framing this. And CVEs, y so some of the research I’ve done recently has been on how CVEs are a lagging indicator. so yes, so we’ve got a lot of pro like products that are trying to find bugs. Bugs are one thing, but CVEs are are a whole other thing. And of course, you know, the the support from NIST and and the government and all these things are part of that. so yeah, just in terms of like CVEs specifically,

Are you seeing that as being fundamentally changing or or is that something that like you deal with a lot at TuxCare or is that just kinda wrapped into like the the updates and the patches and all

Chris DeMars (18:52)
I think there’s more and more CVEs that are coming out now. Cause every time I turn around I’m seeing something with like C VE twenty twenty six, right? right. And then as far as like the patching goes, like the second we hear about it or find find out about it, like we have a an amazing security team full of researchers and and every and engineers like security like front end engineers that strictly work in security and back end engineers that strictly work in the back end, like in Java and Python or whatever.

Kate Holterhoff (18:56)
Okay.

Right. Gotta number

Okay.

Yeah.

Chris DeMars (19:22)
and it just it just comes along with with the whole patching scenario. The the the whole CVE thing, you know, it’s identified by these these bigger companies, these security companies, and then it kind of just trickles down and then when we find out about it or other companies find out about it, then they start doing the patching process. but I I just I keep seeing it more and more, and that just comes with the NPM attacks because a lot of that is stemming from that that area of the front end.

Kate Holterhoff (19:26)
Mm-hmm.

Okay.

Yeah.

All right. So and I and I get that you’re not necessarily on the research team there, but I but your sense is that the CVE’s are still a worthwhile thing to follow, that trying to get ahead of them is or treat them like a lagging indicator is not your approach at the moment. That you’re like, okay, we’re gonna follow the CVE’s, we’re not gonna be doing like extra searching the way that, some of these other tools have done.

Chris DeMars (20:07)
Yeah, I think I think just finding out the CVE and I I can’t speak for the security team, but finding out what the CVE’s are and then patching those, is is like the main priority for the end of life stuff, right? Or even if stuff like nowadays, even if it’s not end of life for a product that we just launched, Secure Chain, even before it’s end of life will still keep you patched until it’s end of life. So any new CVE’s that are identified for newer versions of anything that might not go end of life for two years.

Kate Holterhoff (20:11)
Yeah. That’s fine.

Okay.

Chris DeMars (20:37)
We will still patch that and keep you patched until it goes end of life and then continue to help you still.

Kate Holterhoff (20:43)
Okay. Okay. Yeah. When I’ve talked to other security experts, like I had Tanya Janca on the podcast. she talked about there there’s a sort of disconnect between finding them and then like fixing them. And so you’re on the fixing side, which seems absolutely necessary that finding them is becoming easier, maybe, or there there’s more of them being found. I don’t know. So finding them is one part of it, but fixing is another. And maybe we can even add to that, like.

Chris DeMars (20:56)
Mm-hmm.

Yeah. Yep.

Kate Holterhoff (21:11)
Distributing the fix, right? Okay, fine, it’s fixed over here, but then if all these are, running outdated versions, then it doesn’t matter if you fixed it intellectually. It it needs to be fixed in production.

Chris DeMars (21:12)
Yeah.

Right.

Yeah, when when they’re identified, there’s a process behind it and it’s gotta go through a handful of different steps until it’s kind of like certified, right? With the C VE, like the the the CVE identifier and all that. Then it’s there’s a bunch of metadata that goes along with it. And then you have your companies that are taking those CVEs, doing the identification, the research work, fixing them, and then distributing them out to customers or or open source or however

However, it is. The way we do it is we have a private registry through sonar. And then when you you get a token from us after you talk to sales or whatever, you get a token, you set up an NPMRC file. This is at least how it works for the front end. You set up an NPMRC file with your token that points to our registry. And then you delete your package, your package lock and your node modules. Once you copy and paste our dependencies into your package JSON to replace your dependencies that it came along with.

You run NPM install and then it brings all those patch dependencies over and then we keep you patched as the source of the company that you’re using until you figure out your kind of own migration plan. Because that could take a very, very long time. We don’t want you to stick with us forever, right? We’re essentially a patch while you figure out your own patching strategy and and upgrade plan moving forward.

Kate Holterhoff (22:41)
Okay. How much of the way that you talk about it brings in the idea of modernization? Like is that is that part of how you’re formulating it or is it purely in terms of like patches and I don’t know what?

Chris DeMars (22:54)
It’s most it’s mostly patching unless you’re using modern applications. But I mean everything unless the unless the vendor or the community is fully supporting the product, chances are there’s gonna be a CVE out there. Like the the latest version of Angular, I got chances are if you were to install the latest version of Angular and all the tooling that comes along with it, it will show that there’s zero vulnerabilities in it. Because that is still it’s not end of life yet. It’s community supported and it’s also supported by Angular.

Kate Holterhoff (23:03)
Mm-hmm.

Okay.

Okay.

Chris DeMars (23:23)
the vendor supported with their own team of security people that are that are fixing these things. So modernization is great if you kinda want to stay ahead of, you know mod like attacks, but usually it’s stuff that’s older that people have installed that they can’t, you know, fix right away. That’s where then the rest of that comes into play.

Kate Holterhoff (23:24)
Mm.

Yeah.

Okay. Yeah. And of course there’s good reasons why folks stay on older versions of Java and all these, you know, whatever sort of legacy technologies. and so yeah, I’m I’m sympathetic to the folks who are like, we just cannot do it.

Chris DeMars (23:56)
Right.

Kate Holterhoff (24:01)
Well, another part of this that we haven’t talked about is like open source. So what is it that I don’t know, how are you addressing the open source maintainers who are feeling a lot of burnout right now? They’re dealing with AI slop. I does does TuxCare like contribute upstream? Like what’s what’s their sort of position with the open source community? talk to me about that.

Chris DeMars (24:20)
So most of it is just working with like we just got into the open source, what is it? Open source software federate open source software federation? I forgot what it’s called. The let’s see, not now. the open source security foundation. So we just became a part of that. We’re gold members of that, which is super huge. So I mean we are we’re giving back as much as we can with open source software. Like we c we won’t be able to take all of our products and put them out there for everybody.

Kate Holterhoff (24:39)

Chris DeMars (24:50)
Right, ’cause there’s no way you can make money that way. and some of the stuff it could be

proprietary, but chances are it’s not, but it all depends on languages and frameworks. But we’re we’re giving back to the community. We’re we’re supporting these open source pieces. So that’s pretty much kind of the the the brunt of what we’re doing in the open source community.

Kate Holterhoff (25:12)
Okay. All right. Fair enough. so, with your chops in Linux, I can imagine that that would be a particular open source project that you would spend a lot of time on. yeah, open source is in such an interesting place right now. I’ve had folks like Daniel Stenberg, the creator of curl on the podcast talk about AI slop. bug bounty programs are all sort of f falling apart right now. I mean they’re the perverse incentives for

inundating these projects with with these potential CVE’s that just are nothing burgers. So I don’t know. I my heart goes out to the open source maintainers that are trying to figure this out right now.

Chris DeMars (25:49)
Yeah, well I mean open source maintainers, they’ve always they’ve always had that kind of weight on their shoulders. And I I you know, it just comes along with open source. Like if you’re going to build a product and you’re gonna put it out there for everybody to use, there’s gonna be shit that you’re gonna have to carry on your back. Right. People bitching about things and the issues and pull you know, people bitching and pull requests and this sucks and that sucks, and you need to fix this, and we need this feature, we need this feature.

Kate Holterhoff (26:16)
Right.

Chris DeMars (26:17)
Like there’s

always going to be that overhead. And you take you just you just take that on. And then with the advent of all this AI stuff, now all these security vulnerabilities coming out, like it’s just it’s ramped up that burden, I guess you could say, tenfold. Because now you have to worry about all this other shit that, yeah, we had to worry about it, you know, in the past, but with the advent of AI and AI slop and vibe coding and the security stuff coming out.

Kate Holterhoff (26:34)
Yeah.

Chris DeMars (26:45)
There’s like ten times more things you have to be worried about and maintain as a maintainer if you want your product to thrive.

Kate Holterhoff (26:53)
Okay. Are is Tuxcare like how many customers do you talk to who are in like enterprise and how many are more just like, I don’t know, like digital marketing agencies or, you know, smaller SaaS companies?

Chris DeMars (27:07)
I would say a majority of them are all enterprise. Like they’re big companies. So Dell, Menards, like if you were to look at like our l the logos of the companies that we work with, they’re all big, big, big, big companies. I think Cisco, we do a lot with military and academia. so that side, maybe not so much enterprise, but the big, big companies, like I said, like Dell Menards, they’re the ones that are really in the because a lot of these companies are using enterprise.

Kate Holterhoff (27:09)
Wow. Okay.

Yeah.

Chris DeMars (27:36)
legacy, you know, software and they can’t upgrade right away or they’re they they don’t have the the means to do it right away. You see that a lot. So usually the grassroots startups and you know they’re trying to use the best of the best of the best of the best. And nothing that’s end of life, right? Because they have to do that to get ahead of the game. These big companies, these enterprises, they’re enterprises for a reason. They’re already established. So they don’t have to use the the like the newest, shiniest thing and flavor of the week.

Kate Holterhoff (27:44)
Mm-hmm.

Right.

Yeah, yeah. But the security needs have have shot way up, so then suddenly okay. Yeah. I wonder if AI and the fact that like migrating legacy code bases is becoming easier? I mean, you know, code is cheap now, right? is that something that you’re seeing where we talk about how th there’s just gonna be so much code written. Like the the amount is is huge. are you seeing that like

Chris DeMars (28:10)
Right. Exactly.

Yeah. Right.

Kate Holterhoff (28:33)
there is a move to, not not not use TuxCare, but like my you know, get past this to actually use the the non legacy

Chris DeMars (28:37)
Yeah.

I haven’t heard much even from people at conferences about like, yeah, hey, we’re using AI to upgrade our systems. Like I think people take a step back and just see what has happened with AI and the issues that do come along with it. And you’d have to jump through so many hoops in an enterprise company. You know, I’ve worked for plenty of enterprises in the years past, and hell, even to to get Slack on your computer, you had to go through a whole meeting.

Kate Holterhoff (28:46)
Вау.

Okay.

Chris DeMars (29:10)
And put a slide deck together and get a committee, like talk to the committee, right? And the committee had to vote yes or no. And they’re everybody like different security people are within the committee and architects and I it’s I think it’s it would be more headache to try like, hey, listen, we’re gonna Claude and upgrade our old Java systems to the newest one. Like how many other systems does that system touch, right, in effect? Like it’s a bigger, bigger conversation than having just a website with a product.

Kate Holterhoff (29:28)
Okay.

Chris DeMars (29:39)
that a startup would use and like, hey, we’re using this version of Angular, even though it’s not end of life, but the newest version just drops. So we’re just gonna vibe code and have Claude or Codex change everything from Angular version twenty two to Angular version twenty three. Right? It’s less overhead, especially if you only have five people at your company.

Kate Holterhoff (29:49)
Yeah. huh.

Ugh yeah, no this is good. I know. When I hear that things are drop in replacements, I just go I don’t think so.

Chris DeMars (30:05)
Right. No, no. And

there I mean, even even back back in the day though, before AI was a huge thing when Bootstrap was really big and everybody was using Bootstrap, Bootstrap version two to version three, the all everything changed. The whole syntax changed. So people tried to just drag and drop bootstrap three in, and it broke everything. And that wasn’t even AI. That was manually going through that process of, you know, it pulling in, like if you were to use the CDN, the bootstrap three CDN.

Kate Holterhoff (30:19)
Okay.

Yeah.

Wow.

Chris DeMars (30:35)
Replace the bootstrap to CDN. Well, nothing’s gonna work because the syntax totally changed 180. So you had to go back through and then change everything up manually, change all your code, change all the syntax to match that new version. There are such things as breaking changes. There’s a reason we have that term. Breaking changes is just that. So if you were to take a version of Angular and you say, hey, Claude, upgrade this to the newest version of Angular from this version.

You’re gonna you might have syntax changes, you might have new features, you might have things that worked one way in in this version that aren’t gonna work the same way in the next version. So there’s still a lot of work that needs to be done when it comes to situations like that.

Kate Holterhoff (31:17)
Yeah. It makes me think of there were some high profile reports around Cloudflare building new versions of software. So they made a version of WordPress and Next. and some of the podcasts that I listened to, folks had actually tried to to use it. And there’s of course no documentation and it didn’t work. but it does make you wonder about like, are folks doing it?

you know, to replace some of these libraries that maybe they’ve been pulling in, just to to make sure that their dependencies are less heavy. I don’t know. make the surface area a little bit less broad. but in r yeah, in these enterprise settings, like the majority of developers, it sounds like what you’re seeing is hey they’re not doing this, we’re not shifting everything left, we’re not modernizing at the speed that, maybe we’ve been promised.

Chris DeMars (31:53)
Yeah. Mm-hmm.

Yeah.

And then that’s what that’s what we’ve seen, at least personally that I’ve seen at conferences, that you know, we’re on these older versions of this. We need to get off that and or we’re we’re running end of life software and we’re going through the whole upgrade process, but it’s taking longer. Like if you’re gonna use AI to do that, it’s not gonna take I mean, I would think it wouldn’t take that long to do it. You just let it run or you run the agents or how many credits you have for your AI.

And then it’ll just do it, right? You’re not manually doing that. But I I think there’s a lot of companies that are still out there in the enterprise that are doing that manual shift over to try to get out of legacy systems or even upgrade to a newer version of a legacy system because they just can’t go, you know, full board.

Kate Holterhoff (32:51)
Yeah. Okay. in terms of some of the regulation around this too, because we’ve got some big legislation coming through, especially in the EU, there’s the CRA. is that something that you’re seeing folks be concerned about? Like so the EU Cyber Resilience Act, is that come up at all in the conversations you’re having?

Chris DeMars (33:08)
I have when the the times I’ve been in the EU, I don’t think I’ve talked to anybody personally that has and that’s just another thing. Like developers aren’t thinking about compliance either, right? Like we have a lot of compliance here in the US. You know, we have HIPAA and SOC 2, but it’s compliance changes all over the world, right? And if you’re just not staying ahead of it and you’re just really just not concerned, or you’re just not aware and you’re just you don’t you’re not educated on it.

Kate Holterhoff (33:18)
That’s another thing. Yeah.

Right.

Chris DeMars (33:37)
You just don’t know. You don’t know what you don’t know. And I don’t think I ever heard anybody, you know, whether I’ve been in London or or Germany or anywhere that has talked specifically to me about compliance. Maybe a couple other people that I work with at the booth have talked about compliance with them, but nothing specifically around EU compliance. But maybe that’s because they know I’m from the states and I might not know. Right? The people that I’m working with at the booth, you know, when I’m overseas, they’re from the area. So they would be more, you know.

Kate Holterhoff (33:40)
Right.

Chris DeMars (34:05)
Tuned into EIE or EU compliance and EU governance, stuff like that.

Kate Holterhoff (34:10)
Yeah. No, I I think anyone listening to this podcast recognizes that this is a qualitative conversation. You haven’t seen it all. You’re not bringing your you know, you don’t got your numbers in front of you, that’s fine. I think it’s just important to have these kind of conversations to try to get, you know, get a sense of what folks are seeing because I I think what the vendors are telling the analysts and, potential consumers is often very different than what developers who are actually in the trenches are are doing and seeing.

Chris DeMars (34:23)
Definitely.

Kate Holterhoff (34:38)
I mean, another good example of that I think has to do with so of course there have been a number of layoffs and folks are trying to blame AI on this. I don’t tend to see that as the the cause for kind of the reasons that you’ve articulated here that AI isn’t i it might be an accelerator, but there’s still a number of bottlenecks, folks aren’t modernizing everything, et cetera. but y you know, it it does one of the promises that I heard.

Is hilarious to me as a former QA engineer, which is that everybody’s gonna spend all this time working on their backlogs and that they’re going to improve the code smell of this these legacy systems. And they’re gonna spend all this time on things that don’t make money, basically. And I just find that to be extremely idealistic and potentially just untrue. I you know, i w as a QA engineer, we were always the last people to touch the code and in

Chris DeMars (35:20)
Mm-hmm.

Kate Holterhoff (35:33)
You know, th many of the things that would have been good to have fixed didn’t get fixed. I mean you’ve you’ve mentioned accessibility. Accessibility’s a huge one, right? I mean to talk about things like fall through the cracks. so have you seen more at like have you seen f companies addressing these backlogs the way that they’re I’ve been sold, you know?

Chris DeMars (35:40)
Mm. Yeah, yeah. Yeah.

Not really. I mean, from my experience, just being like I’ll put my front end hat on, like addressing the back someone’s gotta give a shit. You know what I mean? Or the debt the the the technical debt is going to stack and it’s gonna stack and it’s gonna stack and the backlog is gonna get bigger and bigger and bigger. And if nobody cares, nobody’s gonna bring it up. Nobody’s gonna say, Hey, we have this issue with performance, we have this security issue, or this feature is broken.

Kate Holterhoff (35:56)
Yeah.

I do.

Chris DeMars (36:23)
Or the accessibility of this is not what it needs to be. You know, those issues live in a backlog because if nothing, you know, if everything is a priority, right? Every if everything’s a priority, then nothing is a priority. You know, and backlogs need to be focused on more. And I’ve worked heavily, very, very close with QA people at other companies. Like I love working with QA. I hated QA and I loved QA because QA just had to try to break my shit.

And if I had it my way, I would just ship it. But I was also under the you know the mind frame that if there is not a QA team, and I was big on, you know, making sure that companies that I work for had QA teams. And when they said, we’re not hiring QA, it’s like, so you’re gonna trust your developer to say it works? They’re not gonna break their own shit. They wanna continue moving on with the next feature or pull the next thing off the the board or something out of the backlog. They’re not gonna fully test.

the the the product or their their feature, they’re just not gonna do it because they’re gonna wanna keep moving and moving and moving and moving and moving to get a product out the door. That’s why QA is so important. And I think it’s extremely important, not just for QA, but for developers to say, Hey, we need to start pulling this stuff off the backlog. You know, there there could be all these various issues, accessibility, performance, security, features, anything that needs to get fixed. And I that’s that’s a huge that’s a huge important piece.

Kate Holterhoff (37:50)
Mm-hmm. so we’ve talked about the fact that the just upgrade advice isn’t any more realistic than it ever has been. and yeah, and that the companies that you’re talking to at least are are not all moving to take care of this, yeah, the these legacy systems that there still is this need to to patch and to to maintain.

Interested, w one of the key words that you mentioned earlier was about live patching. For my small front end brain, could you explain what that means? Why does it have to be live?

Chris DeMars (38:24)
Yeah, so one of the reasons it has to be live is there’s a lot of big systems out there that are running Linux distributions, right? Like big, big systems. If you take that down, now the system is down entirely. Right. So with with live reboot list and I mean you could think of it as in medical scenarios and anything. Like you you can’t take these systems down, like or

Kate Holterhoff (38:39)
Okay. Yeah, that sounds bad.

Yeah.

Chris DeMars (38:50)
supply chain and like a warehouse in manufacturing like or like ups or dhl or FedEx like these systems can’t be taken down because the second they’re taken down now you you have all these other issues that are gonna arise or just anything can happen. So the way rebootless patching and it’s and it’s for the Linux distributions that we we do. You your organization connects to our servers and then it pings the server

I think it’s every four hours and checking for kernel patches. And it will it’ll check that every so often. And then if it finds one, it pulls it in and patches the kernel within nanoseconds of your system. So there’s really there is really no downtime at all. And it happens live without having to do up you know a a reboot.

Kate Holterhoff (39:40)
Wow. Interesting.

Chris DeMars (39:42)
Yeah. I

mean it’s it’s extremely high level stuff that happens. From what I know, we patch it within nanoseconds, your servers talk to ours and then like I think it’s every four hours and then yeah, see if there’s a patch for a cur the kernel and pulls it in and boom.

Kate Holterhoff (39:45)
Yeah, that’s fine.

Yeah.

Man. Well to go to your movie references, I’m imagining like in Jurassic Park how they have to set the you know, turn the whole system off and then go across the park and the dude gets eaten by the velociraptors, but they have like turn it off and the arms hanging from it and the whole thing. So this is to avoid having anyone die by dinosaurs by having to turn it off and back on.

Chris DeMars (40:18)
Yeah. I’m actually thinking about writing

a blog post on patching and Jurassic Park. And the reason ’cause I love Jurassic Park, I love the whole Jurassic Park franchise, the Jurassic World franchise, except for Dominion and Rebirth. Jurassic Park three sucked. But I worked at a company a few years ago and our director of Demand Gen, she wrote a blog post on how Jurassic like all the issues that Jurassic Park had could have been avoided.

Kate Holterhoff (40:25)
Ha ha ha.

Yeah.

Chris DeMars (40:47)
If they would have just used feature flags, because the company I worked for was a feature flag company. So it was an awesome article. I loved it. And I was just, I was talking to my girlfriend the other night. I’m like, you know, I could probably write a blog post on Jurassic Park and security patching and end of life or end of endless life cycle support. Because a lot of the issues they had, like, would Nedry put in that virus? you know, maybe there’s something there that I could bring in the Jurassic Park world with patching. So I’m I’m thinking about it.

Kate Holterhoff (40:53)
Yeah.

yeah.

You absolutely should. That’s my go to reference for any y Unix whatever clicking the the things. But also, I mean, is there a more iconic turning a system on and off? I you know.

Chris DeMars (41:27)
It’s the best on off like did you turn an on and off reference in the world, right?

Kate Holterhoff (41:32)
It absolutely is. I know I love this so much. Okay. Well, I think on that note, let’s go ahead and wrap up this conversation here. so how can folks keep up with you, Chris? Like, you know, what’s what’s your social stuff?

Chris DeMars (41:44)
Ooh.

let’s see. So you can catch me on Twitter, @saltnburnem S-A-L-T-N-B-U-R-N-E-M, if you’re a supernatural fan, you’ll understand the reference. you can catch me on bluesky, Chrisdemars.net, LinkedIn, Chris DeMars, if we’re in a Slack channel. Twitter is probably the best way to get a hold of me though, because I’m always on Twitter. I always say, you know, we’re we’re always working so many hours just keeping up with what’s going on on Twitter and the security world and the front end world. So Twitter’s probably the best bet, but you can get at me anywhere else.

Kate Holterhoff (42:15)
Fair enough. Okay. Sounds good. All right. Well, I’ve really enjoyed speaking with you today. Again, my guest has been Chris DeMars. again, my name is Kate Holterhoff. I’m an analyst at RedMonk. If you enjoyed this conversation, please like, and review the MonkCast on your podcast platform of choice. If you’re watching us on RedMonk’s YouTube channel, please like, subscribe, and engage with us in the

Chris DeMars (42:20)
Thank you.