---
title: Anne Bertucio de Hollembeak on Why Open Source Needs Trust & Safety
date: 2026-10-15T08:00:30Z
modified: 2026-10-14T19:27:01Z
permalink: "https://redmonk.com/videos/anne-bertucio-de-hollembeak/"
type: video
status: publish
excerpt: ""
wpid: 5105
categories:
  - Podcast
series:
  - Conversations
channel:
  - RedMonk Video
featured_image: "https://redmonk.com/wp-content/uploads/2026/09/Anne-Bertucio-de-Hollembeak.jpg"
timestamp: 2026-10-14T19:27:01Z
tags:
  - Podcast
  - Conversations
  - RedMonk Video
---

In this RedMonk Conversation, Anne Bertucio de Hollembeak, COO of ROOST (Robust Open Online Safety Tools) , chats with Kate Holterhoff about the importance of open source tools in trust and safety, especially in the age of AI. Anne shares insights on building safety tools, the role of open source, and how these efforts can create a safer online environment for everyone.

## Links

- [LinkedIn: Anne Bertucio de Hollembeak](https://www.linkedin.com/in/annebertucio/)
- [Bluesky: @annebdh.bsky.social](https://bsky.app/profile/annebdh.bsky.social)

# Transcript

**Kate Holterhoff (00:04)**
Hello there, my name is Kate Holterhoff. I’m a senior analyst at RedMonk, and my guest today is Anne Bertucio de Hollembeak she is the COO of ROOST, which is the acronym for the Robust Open Online Safety Tools. Anne, great to have you here.

**Anne Bertucio de Hollembeak (00:21)**
Thank you so much for having me. I am super excited.

**Kate Holterhoff (00:23)**
Okay, so before we dig into some of the cool stuff that you’re up to at ROOST, I’d love for you to lay out a little bit of a background here. I’m sure folks might not be as familiar with some of the other cool stuff that you’ve done before arriving at ROOST, which you’ve been at now since what, for a year? Is that right? Little over a year.

**Anne Bertucio de Hollembeak (00:40)**
It’s been a little over a year. Yeah, just about a year.

**Kate Holterhoff (00:43)**
Okay. All right. Well, before we dig into all that cool stuff, tell us some of that background.

**Anne Bertucio de Hollembeak (00:48)**
gosh, it’s been a very winding journey in the best way possible. I feel like, you know, it’s all roads have led to something interesting in the next one. so yeah, ROOST is only about itself 18 months old, and I joined shortly after launch. Before that, I was actually running Google’s open source office, which was really great. I had come there from the OpenStack Foundation, which is now known as the Open Infrastructure Foundation. I was working on a project very much in the container security space. Google was using that project at the time and says, gosh, you wanna you wanna.

come over here and keep doing that work and you know eight years one thing led to another towards the end was kind of having the honor and privilege of running this team of one of the companies that probably produces and uses some of the largest quantities of open source in the world. So it was really neat and there I’d be remiss to say if there wasn’t there was a big detour in there. May maybe not there’s no detours, Kate. All roads leave the next there there was a big journey in there around open source security which

**Kate Holterhoff (01:47)**
Okay.

**Anne Bertucio de Hollembeak (01:48)**
was very exciting. I thought we were all gonna have our moment with the Log4Shell and SolarWinds and then AI came and took our press moment away. But that’s okay. It’ll it’ll come back. It’ll come back.

**Kate Holterhoff (02:00)**
Yeah, I

believe it. yeah, no AI does have a tendency to suck all the air out of the room, I’m afraid. So yeah, good good good point.

**Anne Bertucio de Hollembeak (02:06)**
It’s okay, they gave us a few problems to talk about.

**Kate Holterhoff (02:08)**
okay, amazing. All right, so you joined ROOST about eighteen months ago. I wanna hear about the mission of ROOST. What is it that you are trying to achieve?

**Anne Bertucio de Hollembeak (02:19)**
Yes, I mean we believe that online safety is for everyone. Everyone deserves it. But from a technical perspective, the only way we’re going to get there is if everybody has the tools to do that.

I think right now, if you are not familiar with the state of trust and safety tooling, a lot of it is very much inexpensive proprietary ecosystems. it’s kind of unusual to me coming into it from the security and privacy side, being newer to safety, looking at this and going, they don’t do things over here the way the rest of computing does. And I think there’s an interesting history and a good reason for that.

But you know, we look across networking, security, databases, you have these open source primitives that kind of hold up how we build things, how we this how we design our systems. They didn’t have that in trust and safety. And so this group of folks said, we really need that. We desperately need it to be public a public good. We desperately need it to be public technology. So let’s go build it. And that’s what we’re up to.

**Kate Holterhoff (03:17)**
Okay. So much to dig into there. I think I first want to talk about how you’re defining trust and safety because those ideas are a little squishy in my head. Like I can I trust it. I you know, I feel safe. I don’t know. I I just I I w I wanna understand what that looks like. Is this like compliancey type of trust and safety? Or like how how vague or specific is it?

**Anne Bertucio de Hollembeak (03:40)**
It is very amorphous in the funnest way possible.

**Kate Holterhoff (03:44)**
That’s

**Anne Bertucio de Hollembeak (03:45)**
Right. So if you’re a liberal arts student, come on over. it’s it’s it’s there’s something really enjoyable and intellectually challenging about the fact that it is it is vague. And it overlaps, I think, from a technical perspective too, with privacy and security. there’s a great organization, the Trust and Safety Professionals Association, TSPA, and they’ve tried to build up this curriculum around it, but again, the defining the actual what is safety, it’s squishy, but I would say at the end of the day, it’s very human-centered.

And I think the people who work in the field are trying to think about what are the out the best outcomes for my users? What are my legal obligations by the jurisdictions?

I’m in that sort of fall under this user outcomes umbrella. And then on top of that, what kind of community do I want to have on my platform or you know, in my application and whatever I’m building? So it really runs this huge gamut of harms from, you know, your most egregious of Kate, let’s just start diving right into it. things like child sexual abuse material, terrorism, violent extremism content, exploitation, sexploitation, all the way up to things about what kind of speech might I allow on my platform, including

things that are innocuous like what topics. This is a knitting forum. I need a way to kick off all the car mechanics. You know, it it it is very broad, an incredibly broad field.

**Kate Holterhoff (05:01)**
So in terms of personas who would be interested in this, if I’m hearing that you’re going to be kicking people off, I’m thinking of like developers and folks who are actually building. Is that right? Or is that are there like legal teams involved? Like who who’s leveraging ROOST?

**Anne Bertucio de Hollembeak (05:18)**
Yes. So s the safety workflow is really interesting. I would say you think about it, there’s two parts. There’s the rules, which are what is allowed on my platform. And again, that is everything from what are my legal obligations to what kind of company are we, what do we want to be, to what is appropriate by user design. So, you know, if this is a platform for small children, looks very different from this is a platform for adults. You know, there’s design considerations.

I think there’s also then your things about editorial and topical and how are you gonna control that? So we have all these things around policies, but then we need the tools to implement them. And that’s maybe more where the engineering comes in. policies tend to be for the most part, sometimes they’re written in legal, sometimes they’re written by teams that are specialists in the in the field, you know, trust and safety specialists, sometimes it’s a policy team. but sometimes I think particularly now that we’re in this sort of new era of who’s a developer.

Talking about who’s the builder type thing, you might find yourself being, hey, I vibe coded this chat app, now I suddenly have to be the safety policy specialist also writing these things. So ROOST is really for that whole gamut, which from a design perspective is very fun because our users are everybody from I am the one-person shop who’s, you know, I have a real amount of users on the thing I’ve designed and I’m trying to keep them safe, to I’m

I’m Discord, I’m Bluesky, I have robust policy teams, I have robust safety engineering teams, and these are the tools I’m running.

**Kate Holterhoff (06:51)**
Okay. Okay. so you’ve alluded to some of the use cases around this, very dark and very necessary. but I I wanna finish the arc of like, you know, the who would be implementing this? Because I I feel like the the why and the who, we’ll we’ll we’ll get to it So just to to round out here, open source is a big part of your mission as well. So in terms of

the relationship between safety and open source, you know, these developer concerns. how does that relate to this this sort of disc disconnect of for understanding open source and like, where where does that come in?

**Anne Bertucio de Hollembeak (07:31)**
Yeah, so open source as a concept is really new to this field, to the field of trust and safety.

**Kate Holterhoff (07:35)**
Okay.

**Anne Bertucio de Hollembeak (07:37)**
you know, I think because historically a lot of it came out of legal and policy teams and sort of folks focused on the human aspects of safety rather than folks approaching it from the engineering side. The technologies that are possible and the things that we have in other parts of computing, the norms around things like open source.

they’re they’re not there yet and these concepts are brand new. So I think as ROOST it’s also been fun that we’re doing a lot of education as well of what does it mean to have shared software? What does it mean to have software that is doing defense on a very sensitive area out in the open.

What does it mean to have a community where people actually need to be quite private or prefer to be quite private and offline? That’s been very challenging from a like contributor community building. That’s been an interesting one to navigate. So it’s it’s been fun. I think even just in the last year though, we’ve seen a big shift of how much people are embracing it. Where, you know, we’re finding other people working in this space are putting their tools free online. They’re, you know, embracing open source licenses for them. So I think it’s caught on because when people hear, no, this is not.

product differentiator, this is something everybody should have, they go, yeah, I I agree and I felt that for a long time. We’re like, great, get in. It’s like it’s this is open source car.

**Kate Holterhoff (08:53)**
Yeah. I mean it almost seems to intersect with jokes that I’ve heard that like developers just don’t care about security. that, we’re in this brave new world where suddenly NPM is getting hacked all the time. We got all these mini Shai-Huluds supply chain security is newly problematic in ways that, we haven’t seen historically, because of AI. And so it sounds like the AI is also part of this move that you

are seeing that’s saying that, suddenly we need to make teams aware that trust and safety is a concern across the board but particularly in the open source community.

**Anne Bertucio de Hollembeak (09:31)**
Yeah, I mean across the board, it it I think because of that human element, you know, when you explain the use cases or what might happen on a platform you’ve suddenly built, people go, yeah, I don’t want that or I don’t you know, I don’t want to see that. And so from that sense, you know, once people understand and they have it, they’re go, Yeah, what can I do about that? But I think that’s still really new, you know.

I am grateful for all the people who work in Trust and Safety to date because a lot of the reasons people don’t think about this stuff is because they haven’t seen it, they haven’t experienced it, because somebody in safety was doing their job.

**Kate Holterhoff (10:05)**
Right. So now that we’ve laid that groundwork, I’d like to hear some of these specific uses that that you are working on right now. I know Mila is among them, Coop 1.0 So, take your pick here. Tell us what you specifically are doing around trust and safety in in this, age of AI where we’re increasingly having to deal with these problems in ways that maybe are accelerated and and just,

More critical than ever.

**Anne Bertucio de Hollembeak (10:29)**
Yes. So I think the biggest one for us is child safety. When we think about the the tools we’re working on, that’s one of the use use cases that we kind of have at the core because we feel like it’s the most complicated and it’s the most complex. And so if we can get it right from a workflow perspective in our tooling, that’ll let us get a lot of the other harms right. So things we are building there. We have a tool called Coop, and it really is your foundational content review console where you want to be able to detect your content coming into your platform.

label it and if it’s child safety related if it’s child sexual abuse material you actually have some reporting obligations again varies by your jurisdiction but the biggest one is an organization called NCMEC National Center for Missing and Exploited Children there we go NCMEC and so we have built-in NCMEC reporting in Coop because actually it’s a fascinating challenging problem of what makes a very actionable and helpful report to NCMEC who then passes it on to law enforcement but you’re

you know your average developer is not going to be able to have time to dive into all that research all that you know years of discussion between industry and NickMick and law enforcement to conclude like here are the best practices. So Coop is just like here are the default fields. Here’s what you should put in there. Go okay. Can do. So it really is from an infrastructure side you can think these are things that look a lot like I’m just shuffling data around. But then we put the safety layer on top where there are things features in Coop like

Moderator wellness is really important. So how do I say, you know, this image is coming in, I’m putting it in front of a person, I’m going to put it into grayscale by default because I think that’s going to have less harm on the person who has to look at this image. Or I’m going to blur it to a certain degree by default. And then I’m going to maybe turn off audio by default. These are just some of the features that our team is and the community members too have built into Coop, coming from their experiences working in safety and thinking about how do we want to pass these on.

to all the people who might be entering into this field.

So one of the exciting things I think we found that really like validated our mission and why we exist was a small messaging app called Kyoto. There are about two developers who made an app that’s got hundreds of thousands of users. And in the first prior to coming across our tools, they’d never been able to do child safety reporting. And then they were able in the first week, I believe, they caught nine potential instances of child predation on their platform and reported to NickMick. And by the second week, it was up to 19.

So it’s that for us was exciting. Like that’s the way that child sexual abuse material should be handled in these platforms. And we are very grateful to be able to give them this free solution.

**Kate Holterhoff (13:13)**
That’s huge. and how are you getting the word out? My goodness. I you I I could see that this should be used by way more platforms. and nineteen as horrible as a number as that sounds, I I feel like we we need to, ensure that all of these are getting caught, right?

**Anne Bertucio de Hollembeak (13:27)**
Yeah.

I mean, honestly, we’re this is the fun part of the open source. We’re an open source community. We’re just doing the doing the circuits, doing the rounds.

**Kate Holterhoff (13:34)**
Yeah.

**Anne Bertucio de Hollembeak (13:35)**
We have an amazing community manager who’s, you know, knocking on every door. and these doors kind of vary depending on is it somebody who’s maybe familiar with trust and safety, they just need a better tool, or is it somebody’s like, hey, let me walk you through the absolute basics of what might happen here. I think sometimes we also are in the unfun position of being the bearers of bad news, where a lot of people think that’ll never happen on my platform.

I don’t need that. And then it does. And then they call us back.

**Kate Holterhoff (14:02)**
All right. So you’re getting the word out, you’re at that phase. in terms of education, you alluded to that. so going to conferences, attend it you know, trying to be in places where developers are. are you creating like a certification program? Like I don’t know, YouTube videos? Like what what does this look like?

**Anne Bertucio de Hollembeak (14:21)**
Yeah, no, we’re not I think there’s some really great partners in the safety field that are probably

**Kate Holterhoff (14:25)**
Yeah.

**Anne Bertucio de Hollembeak (14:26)**
better placed to take on things like certification, like the TSPA. they do curriculum and sort of training like that. We’re we’re the tool nerds. We’re just here to give you free tools, you know, we’re just here to ship code. That’s really our our specialty. yeah, we’re we’re the tool nerds.

**Kate Holterhoff (14:45)**
Okay.

And in terms of like community feedback, are there folks that are, in your GitHub repo, creating issues, like talking back or like how are you

**Anne Bertucio de Hollembeak (14:55)**
Yes.

**Kate Holterhoff (14:55)**
knowing what the best strategy is moving forward?

**Anne Bertucio de Hollembeak (14:59)**
Yes, that has been really fun because like I said, the open source is new to safety. And so I think we’ve had to do a lot of education and sort of coaching folks along when they, you know, email us with a s private suggestion of, hey, the tool should do this, like mm, you should do a GitHub issue. And

**Kate Holterhoff (15:15)**
Yeah.

**Anne Bertucio de Hollembeak (15:15)**
i I love the open source 101 stuff. So that’s been really fun of encouraging people to do that. if you go to our repos, you’ll see there’s a good variety. I think one of the things that ROOST has decided to do, which will be a little bit unusual.

That we’re actually going to figure out how to put some what are essentially core contributors on staff. So, you know, you’ll see we’ve been hiring for engineering positions. Our thinking being if we’re trying to help the people who don’t have the resources and the financial means to have these expensive tools, then they probably don’t have all the time to be the super active contributor and keep up with that core. So we’re gonna we’re gonna take that on as best we can.

**Kate Holterhoff (15:58)**
Right. Okay. and I I’d like to hear more about the Mila suicide prevention guardrail because that seems like I don’t know, it it it intersects in significant ways with with the coupe one point zero

**Anne Bertucio de Hollembeak (16:13)**
Yes.

**Kate Holterhoff (16:13)**
that you’ve described.

**Anne Bertucio de Hollembeak (16:15)**
Yes. So Mila is a Quebec AI lab. They’re just very thoughtful, incredible people. I was gonna say up there ’cause I’m north of or south of Quebec, but yeah, incredible, thoughtful

**Kate Holterhoff (16:25)**
Okay.

**Anne Bertucio de Hollembeak (16:26)**
people. you know, really specialists in AI. And so there’s something interesting here. We should actually maybe back up a couple steps and talk about safety’s relationship with language models and AI and all this. So there’s

a framework called Dire that my colleagues worked on and they will tell you because the situation is dire, which is true but it’s a horrible pun. But I it is a good memory it’s a good memory mechanism, dire. recognizing a safety workflow as detection, investigation, review, and enforcement.

So when I was talking about Coop, that addresses review and enforcement. We can talk about our other tool, Osprey, hits investigation. But D detection is your first step. Like you have to be able to figure out what is this, what’s coming in, what are the patterns, what’s the content. And for forever, the state of the art and safety has been to use large language models to do that. Like you’re looking for pattern matching.

So as AI has advanced, there has been a lot of interesting defenses and increases and advances and all the all the good stuff has trickled down to some of those detection things. And that’s where I think we’re starting to get some really sophisticated, interesting things, like the work that came out of Mila. So what we did with them, and this is incredibly specific, but I’m I think it’s a good

It’s good to start small, validate what you’re doing, and then expand. So what’s important is how are we thinking about youth mental health with AI chatbots? How are we thinking about, you know, how we protect these outputs, how we

really prevent some of the egregious things people have seen, like youth ending their lives through suicide because of directions that they received from a chatbot. And so

Mila created this safeguard that is very focused on how do you stop a chatbot from giving suicide directions. It applies to anybody, but in particular their training was looking at youth in particular. I think one of the really interesting things they found was that in training, to train on a general corpus of human-generated directions wasn’t giving them the results they were looking at. They really actually had to train on chatbot language. And I think there’s something very interesting that, you know, they’re

I’m sure their researchers are continuing to unpack in why was that so such a significant change in the output? But that’s what they ended up training on, was more the chatbot language. And now we have this openly licensed safeguard that if you’re saying I want to integrate in open or actually any chatbot into my application, this is really important to me. I’m gonna run this alongside, I’m gonna run this on top to make sure that my youth users don’t receive these directions if they’re asking for them.

**Kate Holterhoff (19:09)**
That’s huge. Okay. so to be clear, this a dire project, so that was something that ROOST that was an initiative that ROOST was involved in, or is it just a project that some of the folks at ROOST kind of knew about and and had some partnerships with?

**Anne Bertucio de Hollembeak (19:23)**
So this was some of the pre-work that inspired the idea for ROOST. Yeah. So like Camille François who’s our president, Juliet Shen who’s our director of product, were part of the group that built out this dire framework. And as part of that, they were doing research, looking, talking to people about their tools and saying, you know, here’s this framework, what do you have? How do you work through these different quadrants? And I think they found everybody’s tools were very similar.

People felt that their tools might be different because their policies are different. The rules they have on their platforms and how they think about what’s allowed on their platforms are different. But the actual tools and the infrastructure that does label action, move to investigator, you know, take action, follow enforced rule, it it was all very similar. And that’s what kind of sparked this conversation about, okay, not only do we need to solve access in trust and safety, but open source is going to be the way to do it.

**Kate Holterhoff (20:14)**
I like this idea of it being a guardrail too, because that has a very DevOpsy feel. so yeah, I d I don’t know, was it what was the thought behind calling it a guardrail?

**Anne Bertucio de Hollembeak (20:26)**
I think that’s kind of become the the nomenclature of of this you know, we th people call things safeguards, people call things guardrails, and I think everybody’s

**Kate Holterhoff (20:36)**
Right.

**Anne Bertucio de Hollembeak (20:36)**
always saying something slightly different. But kind of this idea of I wanna stop a thing. I I push my bowling ball down the lane and I want a bumper to bounce it back in. but I don’t think we wanna call them bumpers, Kate. That’s not exciting. That’s not exciting.

**Kate Holterhoff (20:51)**
No. Okay. Bumpers

are right out. Guardrails are in. Okay. All right.

**Anne Bertucio de Hollembeak (20:55)**
Guardians are in, bumpers are out.

**Kate Holterhoff (20:57)**
Fair enough. I like it. So you’re also thinking about maintainer burnout as it relates to AI slop. Talk to me about how ROOST is also helping with that. it’s something I’ve written about a fair bit, so I’m I’m

Always got my ear to the ground with how we’re trying to help maintainers at this time.

**Anne Bertucio de Hollembeak (21:12)**
Yes, yes. This is something as we were, you know, putting our community together. We were trying to figure out like what is our policy about AI contributions going to be.

One of the things too that was important to us was we’re Apache 2 licensed. That was a very intentional choice because of the commercialization opportunities it provides. We think there’s a route in which ROOST Tools being out there kind of becoming hopefully this de facto standard that there’s a huge commercial ecosystem around things. All that to say we need to have the right to distribute what what we’re building.

And so from a bunch of different angles and all the stories of people being like, I can’t take 20,000 line code changes. That’s not reviewable. We’re like, we have to do something here. I think our approach was to really try and keep things very simple until we have the next problem. I there’s a pal in open source, Sarah Novotny, who has this phrase minimum viable governance, and I absolutely love it for you know any problem in building an open source community thinking about what is the minimum thing I need, and let’s not put the cart way before the horse.

and build something really complicated before I get to the get to the basics.

So for us, we were like, okay, we are seeing some of these challenges. We are seeing things where people drop in massive code changes that you can’t really figure out. You drop in a thing that’s not related to an issue. But we tried to kind of pull it back and say, well, to other developers, these are actually very helpful tools. At the end of the day, AI is just another developer tool in this context. And so our engineering, you know, expectations of what makes a good commit message, what makes a reviewable code review, that hasn’t changed at all. And so our

policy as a community kind of became, you know what, use the tools you want to use. And that includes AI. Use what you want to use. But at the end of the day, it just has to be work that you have to be able to explain, that you’re ready to stand behind, and that you can say, yes, I wrote this and I’m submitting it to this project.

**Kate Holterhoff (23:09)**
That’s amazing. And how did you codify that? Did you create like what, like a policy?

**Anne Bertucio de Hollembeak (23:14)**
Yes, yes. We’ve got we’ve got a policy in our repos and says this is, you know, part of contributing. Read our AI contribution policy. Here we are. Yeah. Yeah. And I mean

You know, it’s not a hundred percent foolproof. You know, we still we still get the occasional. I think we had one the other day, you know, seven thousand lines, here it comes in, and you’re like this, I I can’t read this, I can’t digest this. but I think when we’ve talked to people, y you know, we’ve gone back and said, Hey, I understand what you’re trying to do here. Here’s why we can’t take this as is. Here’s our recommendation of how you break it out. Please follow these policies. I think people have been receptive. I think to date we haven’t seen anybody say

No, you must take all seven thousand of my lines at once because Claude told me to. We we haven’t seen that.

**Kate Holterhoff (23:59)**
small mercies. I I I appreciate that. Okay. So you’ve got your AI policy in place. you are permissive, it sounds like, in terms of allowing it. and have you been inundated with more issues?

than you anticipated. because of course you were birthed in the time of AI, so you were within the slot the slop was already happening. but did did you prepare for that like preemptively? Is this like how new open source projects are going to have to enter the fray and say like, well, get ready for it, here they come.

**Anne Bertucio de Hollembeak (24:33)**
Yeah.

You know, I mean I think it was on our mind of that this is gonna be something and that’s where we tried to see what would happen and get ahead of it with our thoughts about what do we wanna do here

**Kate Holterhoff (24:43)**
Yeah.

**Anne Bertucio de Hollembeak (24:44)**
when this becomes a problem. I I think great thankfully we’re niche enough that we’re not inundated with a bunch of weird issues that are irrelevant and, you know, gamified. I don’t think people really find us unless something else relevant has led us there. Although I have been told on multiple occasions we have good GEO.

Is is that generative

**Kate Holterhoff (25:07)**
I don’t know that one.

**Anne Bertucio de Hollembeak (25:08)**
instead of SEO, this is like generative AI engine optim I don’t know. This is a thing.

**Kate Holterhoff (25:14)**
I’ve heard it called

AX if it’s what I’m thinking that you’re referring to, like agent experience, where the agents are able to use your code. And that is it’s findable. Mm-hmm.

**Anne Bertucio de Hollembeak (25:21)**
that sounds much better. Ooh, that sounds much better

than GEO, because I’m from the Pacific Northwest and so that sounds like geoduck which is a slippery, slimy

**Kate Holterhoff (25:31)**
Yes.

**Anne Bertucio de Hollembeak (25:31)**
delic delicacy. yeah,

**Kate Holterhoff (25:33)**
huh.

**Anne Bertucio de Hollembeak (25:34)**
AX sound we have good AX, all right. So I’ve been told. So

**Kate Holterhoff (25:36)**
Okay. Yeah.

**Anne Bertucio de Hollembeak (25:38)**
people are only finding us to date, you know, when they’re it’s relevant and they were already looking for that. Yeah.

**Kate Holterhoff (25:45)**
Okay. Okay. Yeah, I mean it comes down to incentivization in a lot of ways in in terms of like being inundated with slop. So like the curl project, you know, they had a very famous bug bounty historically. And so I don’t know if if at ROOST you you are paying people to find these bugs, but it sounds like yeah, okay. So that probably saved you a little bit at least.

**Anne Bertucio de Hollembeak (26:04)**
No. Not yet. Yes.

There’s no incentive here. I don’t have t-shirts. I mean I d I have good stickers actually. But

aside from that, I

**Kate Holterhoff (26:13)**
Yeah. Okay.

**Anne Bertucio de Hollembeak (26:16)**
no, actually I do have t-shirts. We just made t-shirts. Okay. But there’s not gonna be any t-shirts. But no t-shirts, no stickers. yeah, And we’re not

**Kate Holterhoff (26:22)**
No t shirts. All right. This is Okay. So if you’re finding

bugs, it’s through the goodness of your heart, not

**Anne Bertucio de Hollembeak (26:29)**
Yes.

**Kate Holterhoff (26:29)**
because you’re being incentivized. Okay. Fair enough.

**Anne Bertucio de Hollembeak (26:32)**
Yes, and today

we are not you know, we’re we’re only 18 months old and part of my work in open source has been in incident response and vulnerability management. And you have to have a really good, robust response process before you’re ready for the bug bounty where you’re dangling money carrots at people. We’re not

**Kate Holterhoff (26:49)**
Mm-hmm.

**Anne Bertucio de Hollembeak (26:50)**
we’re not there yet. We have a good process. We’ve had our first security issues and all that, but we’re not ready to dangle money carrots.

**Kate Holterhoff (26:57)**
Understood.

Okay. All right. So to continue on with these bird metaphors, we’re roosting. I want to talk about Osprey now. talk to what is Osprey? and and what what does it do? How are you leveraging it, etcetera?

**Anne Bertucio de Hollembeak (27:12)**
Yes. So if you remember that dire framework, the I, investigation. So you can also think about Coop being, if you were to say, I’m brand new to this, where do I start? You would start with Coop. That’s the thing that goes, content goes in front of a person, human moderator, you know, actioning, making decisions. Osprey is a little bit more mature. You’re thinking about I am seeing some sort of pattern, or there’s something I am hunting for on my platform.

And I’m trying to figure it out and I’m trying to automate what is automate away what is obvious. Things like spam, fraud, just say I am very high confidence that I don’t want this here, here’s a rule to just kick it.

Or you’re like, what is what is going on? Why are all these accounts suddenly being created that share these keywords or they share these set of IP addresses or these timestamps or something funky is here? Osprey is the tool that lets you query, query, query in these user-defined ways to figure that out and then decide what kind of action you want to take. Osprey actually has a very interesting backstory and it kind of speaks to why ROOST exists. yeah, are we here for stories? We’re here for stories.

**Kate Holterhoff (28:15)**
Let’s hear it. Yes, I’m ready for a story.

**Anne Bertucio de Hollembeak (28:18)**
So there was a safety tool called Smyte this is probably ooh, I have to get my timestamps right here. but maybe the 2016 time or so. 2016 time. and it had a bunch of different users. People di platforms really liked it. Safety analysts really liked this tool for doing their investigations. a company with

Birds bought it because they also really liked it. But they didn’t have any interest in running the business. They bought it because for them it was penciled out that we want to have this as a tool internally. And so they shut it off for everyone else.

And I think it just goes to show how you know overnight people were missing this critical investigation tool because one market just said, nope, we’re we’re done with this tool, and it hurt everyone online. So there was a group of folks who were at Discord and said this cannot happen again to anyone, to us, to anyone else. And so they rewrote their own version. and they ended up donating that source code as the core to ROOST that became

the core of Osprey. So when we encounter people who have been in safety for a long time, say do you remember Smyte? They yeah, I’m like we’ve we’ve we’ve got Smyte back. Smyte’s over here now. so it’s it’s kind of a fun story but I think it does speak to how we just we can’t have such critical technology tied up in proprietary ecosystems.

**Kate Holterhoff (29:46)**
Very good story. You’re right. Was Smyte like did it go under the like business source license? Like what how did how were they able to do that? Was it MIT at any point? Like yeah, how’d that work?

**Anne Bertucio de Hollembeak (29:57)**
It so

the original one it was never open source. It was just a vendor tool. That a lot of people had. And then they

**Kate Holterhoff (30:01)**
Ooh I see.

**Anne Bertucio de Hollembeak (30:03)**
acquired it. And then they’re like, we don’t want to run this for other people. We just wanted to run it for ourselves. And people got the API shut off. Yeah. Yes. Yes.

**Kate Holterhoff (30:11)**
I see, I see. So they were paying for it. Got it. Okay.

And so I mean that’s kind of like an expression of your philosophy a little bit, right? Like security, safety should be available to everyone, open sourced.

**Anne Bertucio de Hollembeak (30:22)**
This should not happen. you’re you know, if for some reason things change, you should still be in control of the technology you need to keep your user safe. This this shouldn’t happen.

**Kate Holterhoff (30:31)**
Yeah.

**Anne Bertucio de Hollembeak (30:32)**
Yeah.

**Kate Holterhoff (30:34)**
Now many folks in your position with this open source version of Smyte with ROOST, go with the foundation route. Why did you opt to not do that this time?

**Anne Bertucio de Hollembeak (30:45)**
To not put it in a different foundation.

**Kate Holterhoff (30:46)**
Yeah.

Like an well, is ROOST a foundation?

**Anne Bertucio de Hollembeak (30:50)**
I mean you could say we are. I I think we didn’t we didn’t use the

**Kate Holterhoff (30:52)**
I’m sorry, I missed that.

**Anne Bertucio de Hollembeak (30:54)**
I think we didn’t use the phrase foundation, but we’re a nonprofit and we’re we’re an open source

**Kate Holterhoff (30:57)**
Yeah. Okay.

**Anne Bertucio de Hollembeak (30:58)**
nonprofit. yeah, we just decided not to use the name ROOST Foundation or Safety Foundation.

**Kate Holterhoff (31:06)**
Yeah.

**Anne Bertucio de Hollembeak (31:06)**
but yeah, no, I’m I’m I’m in the the the what do I say the table where the open source foundation people come together and go, what are your problems? That’s where we’re that’s where we’re we’re at

**Kate Holterhoff (31:16)**
Yeah. You’re at that table.

**Anne Bertucio de Hollembeak (31:18)**
that table.

**Kate Holterhoff (31:19)**
is there any backstories in not calling it a foundation? ‘Cause I feel like that is what many people call it, when they’re like we run an open source project, we are a foundation.

**Anne Bertucio de Hollembeak (31:29)**
That’s a great question. you know I don’t think it was. I think maybe the bird puns were strong and

**Kate Holterhoff (31:37)**
Like that.

**Anne Bertucio de Hollembeak (31:38)**
roost foundation is a mouthful. yeah, yeah. Doesn’t happen. We’re focused on the tools.

**Kate Holterhoff (31:41)**
Yeah, sure. Just didn’t happen that way. All right. Fair enough. Okay.

That’s great. I mean, you you’re doing good work here. Non profit, that’s great. okay. And talk to me about some of these partners because you have an amazing logo page. how are you working with some of the the big ones here? And I won I won’t reveal any I’ll let I’ll let you do that.

**Anne Bertucio de Hollembeak (32:04)**
Yeah.

Yes, I can reveal them. We have both philanthropic and corporate partners and you know other ones who are fellow travelers in the nonprofit open source space like Mozilla. It’s it’s a wonderful group of people. so the more kind of direct connections on the corporate side, these are people who understand and realize that if the bar for safety isn’t lifted for the bar, the boat, the ride, the tide, if safety is not elevated

**Kate Holterhoff (32:30)**
Yes.

**Anne Bertucio de Hollembeak (32:30)**
for everybody, it’s bad business for them, it’s not right.

for humanity.

I think they see that. So Google, for example, is one of our corporate partners, and that’s because they have something called the Google Content Safety Ape API, which is a free tool for CSAM scanning. That’s the child sexual abuse material we were talking about earlier. However,

**Kate Holterhoff (32:50)**
Yeah.

**Anne Bertucio de Hollembeak (32:51)**
you can only get access to this tool if you are if you have something like a content review console to properly do NCMEC reporting. And so, pun intended, there’s a chicken in the egg problem there in which you know Google wants these small

Smaller companies to become bigger companies and these startups to exist, and CSAM scanning is part of that, but if they don’t have the content review tool, they can’t give them access to it. So here comes ROOST, we’re here’s the open source free tool. Now this amazing thing that Google offers with the free CSAM scanning is available to way more people. So that’s kind of the flywheel that happens there with our corporate partners.

**Kate Holterhoff (33:29)**
I also saw Bluesky on there and you’ve got a Bluesky story, right? I I I’d love to hear more about that. I I actually was hanging out with some some cool Bluesky celebrities at at Render ATL, which was just in Atlanta so I definitely have Bluesky I’m Bluesky pilled.

**Anne Bertucio de Hollembeak (33:46)**
Yes, yes. they’ve been so great, such a great partner. so they’re an Osprey user. And they came in pretty early on it as kind of that first pilot user of like, it’s leaving Discord. we’re trying to build this more generic project together that’s you know, works great for everyone outside I was gonna say outside Discord, Discord included, they’re also an Osprey user, but y you know, it’s generalized, it’s good for everyone. And they were just so generous and willing to be that kind of first trial.

of saying here’s what we would need, here’s what’s working well, here’s what’s not. So they’ve been a fantastic partner in that way. And obviously running at huge scale, which is really helpful for us to be able to figure out, you know, what’s sort of the edge of this, what are improvements we need to make, they’ve been wonderful.

**Kate Holterhoff (34:33)**
That’s fantastic. Okay. Well, shout out to Paul Frazee, CTO over there. I’ve mostly heard him talk about like atproto but I love that, he’s also talking about Osprey. Then this is, you know, how else are we thinking about, scaling some of the problems that are unique to these social media sites?

**Anne Bertucio de Hollembeak (34:50)**
Right.

And I think

the the gener in general, like I guess we could have a whole other hour of decent the decentralized and roost story. Because I think too for these organizations, this is something they desperately need. The Mastodons, the folks building on AT proto. when we talk about who might not have access to expensive proprietary safety tools, those are great examples. So

**Kate Holterhoff (35:13)**
Mm-hmm.

**Anne Bertucio de Hollembeak (35:14)**
it’s been it’s been really neat to have conversations with those different communities. interesting challenges about how do you scale.

safety in decentralized communications, but everybody’s just been so wonderful and it’s we have fun challenges to tackle ahead of us, I’ll put I’ll put it that way, but with great people.

**Kate Holterhoff (35:33)**
I mean I know that they’re thinking through a lot of challenges that maybe had been left in black boxes historically with some of the older, more legacy social media sites. So it’s interesting the the idea of trying to distribute them and make them owned by the user, that’s that open source mentality, right?

I I I want this to be mine so that I I have the ability to leave afterwards if I decide to go on a different Mastodon server or whatever. can you give us a teaser of like what would you say those challenges are that are so unique?

**Anne Bertucio de Hollembeak (36:00)**
Yeah, no, no, no. So I think one of the most challenging ones when you think about whether it’s Activity Pub or AT proto, you want to enable people to build platforms on top of that protocol. And that does put them in the driver’s seat in many instances of moderation. However, if you think about if I think of a world where there’s, you know, eleven thousand instances of chat things and chat communities outdoor, do I really want eleven thousand groups of people to have to moderate

Terrorism and extremism and violent content and child sexual abuse content. Like from a burnout and a mental safety of the people doing this work, I don’t think we want that. So there’s some something to be figured out and to be sorted out that respects the decentralized philosophy without saying we decentralized the worst part of running social platforms. You know, I think there’s there’s something to be found there, and I think these are the conversations that are being had and people trying to figure out because they do want to.

to have that decentralization at different levels of safety, but maybe not some of our most egregious harms. And so that’s kind of the the challenge of the decentralized movement and the scaling when it comes to safety.

**Kate Holterhoff (37:14)**
Yeah.

so Discord and Robolox are also partners of yours. Are they also users of Osprey or are they thinking about it in a different way?

**Anne Bertucio de Hollembeak (37:24)**
So Discord is. Robolox actually just the other morning announced that they joined something we have called our ROOST model community with some interesting open safety models that they’ve released. what’s interesting about when we think about how do we make generalized safety tech is that in some of these verticals, sometimes the content is so specific that the platform, you know, maybe the the tools look very similar, but those policies start being

Like a policy written for gaming is probably not applicable at all to LinkedIn. you just think about gamer language and other things like that. It’s it’s not transferable. And then you add on top of it things that like this is a platform built for youth, their language, their customs, not transferable to platforms built for adults. Roblox has been kind enough to open source some very interesting tech though that they have built around things, detections in particular for grooming, for child grooming, because

What’s challenging from a technical perspective about that is that grooming is a thing that happens over a long period of time. So if I’m thinking about how do I look at data and detect this is it, there will never be one instance. And in fact, if I look at that instance in isolation, it probably looks innocuous. Like the exchange doesn’t clue me into anything. It’s that it’s happening over a long period of time, and within the context of that window, then I say, wait a second, something dangerous is happening here. So they have a library called Sentinel.

That does look over that long window, but then actually flags things at the user level. Because again, if I just pull out any one message and say, this is dangerous, that’s not super helpful, I want to flag the user to an investigator and say something is going on here that somebody needs to take a look at. So that’s kind of the Roblox story. They’ve also been great partners in thinking about like how do we change this industry overall and are bringing, you know, some of the insights of what’s happening on their platform to everyone else.

**Kate Holterhoff (39:23)**
Hm. Well now I’m depressed again, so thank you for that. God.

**Anne Bertucio de Hollembeak (39:28)**
Like, do I have

any hopeful stories? I mean I I the people of this are the hopeful story. The

**Kate Holterhoff (39:33)**
Yeah.

**Anne Bertucio de Hollembeak (39:34)**
you know. Should we talk about other awful things? You wanna I mean

**Kate Holterhoff (39:36)**
I I don’t know. no, I No, we I don’t

know. We got it’s important work. I know. It’s just I thinking about grooming. I don’t know. It’s really it’s it’s bad stuff.

**Anne Bertucio de Hollembeak (39:48)**
It’s

it’s challenging. It’s challenging. But these kind of multi turn conversations like this and things that happen where people in particular youth are bouncing between platforms from a technology perspective are some of the hardest problems. so I’m you know the hope is that everyone is coming together and maybe we can solve these together a little bit better.

**Kate Holterhoff (40:12)**
I I love that and I think that’s great. So I that is you’re right, that is the silver lining and me being a mother of small children with, her head in the sand is not a good look. So I should look up and realize that we need to confront these problems. So I’m I’m doing that with you here and this is this is all for the good. but I yeah, but I you know, I I I enjoy the story about Roblox, you know, I I think of like Minecraft and a lot of these

Places where young people are chatting with strangers on the internet.

**Anne Bertucio de Hollembeak (40:45)**
Yeah. Well n you know, if I’m give you some optimism here, like the reason we’re doing all this is that the internet is fun. Like, you know, i it is valuable, it is fun, it’s where you go to learn, it’s where you go to find new interests, find new people, reach corners and ideas that you couldn’t reach sans internet. it

**Kate Holterhoff (41:02)**
Mm-hmm.

**Anne Bertucio de Hollembeak (41:03)**
digital spaces are great and they have huge power and potential. So, you know, even though some of the things happening on them right now are horrific, I think we can get o we

We can’t get over that. We can win. We can we can win. We can we can

**Kate Holterhoff (41:17)**
We can win. All right.

**Anne Bertucio de Hollembeak (41:20)**
do the work to make sure that these s spaces are safe. You know, it’s it’s work that’s

**Kate Holterhoff (41:24)**
Yeah.

**Anne Bertucio de Hollembeak (41:25)**
never gonna be done. but I think we can do it smarter, we can get it out to everyone, and we can have the fun internet again.

**Kate Holterhoff (41:33)**
All right. I’m all for fun internet.

**Anne Bertucio de Hollembeak (41:35)**
Fine.

**Kate Holterhoff (41:37)**
I do want to talk, so I’m gonna go dark here. I want to talk about the dead internet. How are you dealing with the bot issue? I know Cloudflare’s been thinking about that a lot. a lot of folks have been doing thought leadership about, you know, X percentage of the internet is whatever. I know I lost many of my my my Twitter slash X followers are gone now because they were my very loyal bots.

And now they’re gone. And so you know, it’s very depressing. I’m under ten thousand followers. I feel my self-worth has just plummeted. anyway, so the internet’s full of bots. i i is there a way that you’re you’re trying to grapple with the the what the the automization of of these you know fake people that we’re interacting with.

**Anne Bertucio de Hollembeak (42:15)**
Yeah. I mean this is the the challenges AI has has brought this field is that things are at massive, massive scale. And the consequences from a bad actor’s perspective of having what I essentially would call like a low effort campaign just went to zero. And what I mean by that is that, you know, before if you were like, hey, I want to scam this person,

you maybe would put some effort into a couple different targets. And now it’s like I’ve got nothing to lose. I can just roll this all automated out a hundred times, a thousand times. If one bites, great. And it the before you could never operate that way. Now you can. So I think when it comes to things about how do we think about what tooling we need, it we need tools that are gonna act at that scale and find, you know, be able to handle that volume, be able to find those patterns at a world in which a bad actor

and just roll out hundreds of of fraudulent things if that’s what they want to do.

**Kate Holterhoff (43:13)**
Okay. I wanna ask one more dark question. So a thing that occurs to me is if you’re open sourcing some of these policies, then bad actors could ostensibly point their LLM at it to bypass them. Is that a concern or like how are you remaining nimble enough? Like I can see openness being kind of a counter to to these abil like the fact that we’ve all got a hundred X developers in our back pockets with Claude.

So, you know, how how is that a concern? Like how what do you how does that work?

**Anne Bertucio de Hollembeak (43:47)**
This is something we talk about a lot at our organization and with, you know, people in this space. I think it’s part of the history of why trust and safety has not really had a robust open source movement until now.

Because I think obscurity has been a main defense and a strong defense around a lot of these practices. I think for us though,

**Kate Holterhoff (44:06)**
Mm.

**Anne Bertucio de Hollembeak (44:07)**
it’s sort of like anything else in open source software where we’re thinking about what is the net benefit that is gained by people having much more access to this tooling, by people being able to customize it, by people being able to redistribute it, compared to could some bad actor abuse it? Yes. will it happen?

Can’t say, don’t know, but we have we can’t pretend that that’s not a consideration. But I think when we look at the net benefit that could happen, we go, that’s bigger, that’s better, that’s the thing that we need to do. I think from what ROOST is building too, there’s something to keep in mind that without all the other things that make up a platform, like the user data, you know, whatever those special fields are and the content.

The tool that just says label route, you know, Q, it’s not much. but I think just the idea that the critical tooling for trust and safety is open source, it it does make people a little twitchy. They’re a little uncomfortable. try and go in and say, hold on a second, let’s really think through, you know, do our adversarial testing, what really could happen by these tools, source code being available.

probably not the the dire outcome that you you’re have in mind, you know.

**Kate Holterhoff (45:26)**
Okay, okay. All right. Well, I feel like that is a good place for us to wrap up thinking about this. So before we go, Anne, what links would you direct folks to if they’re interested in learning more? and even your personal musings? Like do you have a particular blog or social media account that you prefer folks to to look for you at?

**Anne Bertucio de Hollembeak (45:49)**
sure. so we are roost.tools is our website. Our GitHub is @roostorg and they’ll you’ll see Coop, you’ll see Osprey, and you can also find the links to our community channels like on Discord where people hang out. So we definitely encourage people to check out the tools. even if you’re not in safety, if you’re just a developer who’s really passionate about it and you’re familiar with open source, we’ll take your contributions.

Yeah, I think that can be a n nice way for some folks to feel like they give back in this space. and then me personally, I’m really enjoying Bluesky.

**Kate Holterhoff (46:22)**
Nice.

**Anne Bertucio de Hollembeak (46:23)**
Yeah, I find it fun. I find it fun. I’ve see I’m like, dear, what’s my handle? I think it’s @annebdh.bsky.social I think that’s right. But yeah. There we go. There we go.

**Kate Holterhoff (46:31)**
All right. Well we’ll I’ll confirm and put it in the show notes. How ’bout that? That way we’ll be we’ll be sure.

**Anne Bertucio de Hollembeak (46:36)**
Yeah. But no, I I think it’s a fun platform. you know, I I again I remember the early days of social media stuff and I s I see that spark back and so I’m having a good time there.

**Kate Holterhoff (46:49)**
That’s fantastic. Yes, that is it is my preferred place to go. I, you know, shout out to Jeremy Meiss, who has recently created a pop punk emo 2000s bot to share, yes, to share

**Anne Bertucio de Hollembeak (47:00)**
No

**Kate Holterhoff (47:03)**
lyrics with me whenever I’m feeling like I need that in my life, which is, you know, most days. So yeah, so anyways, good good fun stuff going on at Bluesky. and yeah, so so that we we’ll we’ll keep that energy going.

**Anne Bertucio de Hollembeak (47:16)**
So next time we talk,

do we have to have swoopy bangs?

**Kate Holterhoff (47:21)**
I I am not a bangs person. My hair does not do that. That is not a thing for me. but I I love I love the commitment for people who do. And I I applaud them and I all I want to be like Zooey Deschanel as well. but I it just I know

**Anne Bertucio de Hollembeak (47:34)**
Good things.

**Kate Holterhoff (47:35)**
know thyself and it was it is not a thing. But you know, and I I you know y i if this is the the journey you’re on, I I support you entirely. I know that you are right now. Not happening.

**Anne Bertucio de Hollembeak (47:44)**
No, no, no. No thy hair. It’s it’s not happening for us.

**Kate Holterhoff (47:49)**
Okay, beautiful. all right. Well, I have really enjoyed speaking with you today, and this has been a lot of fun. again, my name is Kate Holterhoff. I’m a senior analyst at RedMonk. If you’ve enjoyed this conversation, please like, subscribe, and review the Monkcast on your podcast platform of choice. If you’re watching us on RedMonk’s YouTube channel, please like, subscribe, and engage with us in the comments.