What happens when a psychologist for software teams is diagnosed with a rare disease? In this Monktoberfest talk, Cat applies everything she knew about research, measurement and community to a new, highly personal challenge. From this experience, she had vital lessons for the Monktoberfest audience about how to navigate uncertainty, build psychological safety, and design evidence that truly serves people.
Transcript
Thank you, folks, I’m Cat. I’m a psychologist for software teams. I like to call myself a research architect. I’ve led a lot of open science for developer teams and I’m interested in a lot of different things, like including agency, which I’ve measured in my studies the importance of it, actually, so when Julia was talking about that, it filled my heart. And I’m going to talk about a moment of losing agency that some of you might feel like you recognize.
All right, so we’ve all been there. I was handed a nightmare of a project, and I didn’t want it.
I didn’t even know anything about it. And I tried to refuse it. I thought somehow my skills, all of the things that I had spent all of these years working on, my, you know, my abilities, my experiences that I had filled my brain with, all of the things I thought of as myself, you know, this was surely going to matter.
It didn’t.
It was not an important part of the decision to hand me this project. And I’m sure you’ve been here, too, right? There’s like some, hard to even describe how this happens. Some web of interactions and signals and messages and decisions, there’s like a chain of command, maybe, that nobody really understands, but nobody’s allowed to question, and at the end of that chain of command you find yourself, like you’re the project manager of the nightmare project, even when project manager is not your job title.
So, I began to suspect that this job was going to demand a lot of glue work, like it had that shape immediately, that smell, that glue smell, right? Immediately, I was going to have to convince a lot of decisionmakers that I possessed very important information while never making them feel like they did not possess important information. I had to sit in meetings. There were very brief, like the length of time in which you could try to convince someone that psychology is a real field that people can work in, and if I did it right, probably no one would know, and if I did it wrong, it was really a disaster, maybe even with irreversible consequences, so it was a high-stakes nightmare project. So I did the first thing that a smart person does when they’re overwhelmed: I looked for the fucking manual to read!
And that’s when I discovered that there was no goddamned documentation. Why do we do this? So I have an observability problem. The nature of my nightmare project is that it was very dynamic, so you could have a meeting happen on a certain day and it would look kind of like it wasn’t that big of a deal and then you would go home and it would immediately be a very big deal, right? So it was an observability issue and just for fun, it was also very poorly instrumented. I had to roll my own metrics, right, to measure my progress. I started to feel like I’m the only person who has all the pieces here, and also I need to learn like four or five different languages to placate everyone around me. And then there was a light in the darkness, that thing that you can turn to when all the hope is lost, darkness covers the land, the internet! So I got online and I discovered — I had a very specific problem to describe and I knew it was the kind of thing people would want to complain about, right?
That means you can get online and find some people, some collection of like overlooked very cranky kind of complicated old-timers. That’s me, also, in psychology, so it’s not like an age thing, right?
But I knew, I’m like other people have been given their own nightmare projects, and maybe some of them have the same shape as mine, right? And just like that, a group chat materialized. Ohhhh, the group chat. In this group chat we started to put things together and do translation in the back part. We shared horror stories, but there was wisdom in the horror stories. I started to realize that the best guidance I was going to find was listening, listening to the people who had fought many battles, some of which had been won and some of which had been lost and that was a really good thing, because it made me feel like I was truly alone in this experience. Now, at this point you’re thinking, what is it, Cat? Was it a research project? Was it one of your papers that I read? Because, like, give us the tea. Did you get involved in open source? Well, what I’m describing is when I was diagnosed with a rare disease. Now, before you worry, this is going to turn into a nightmare talk, I am OK, all right? I’m OK, we’re all OK.
[applause]
Thank you. I’m pretty psyched about it, too. This is not a life-threatening condition, all right, but just like being alive and becoming a psychologist or attending Monktoberfest, it is a lifelong condition, so it’s treatable, but it now shapes my reality. Now, for transparency sake, I do have to tell you I got to this place after like a solid week of listening to music on the ground in our side yard crying like the whole time. I made a playlist actually that I named the sick sad girl playlist. It’s full of bangers, and I’m looking for recommendations all the time. I love new music. I truly was diagnosed with a rare disease, which is just bizarre, right? And it was two days after I had to close the research lab that I had been building for three years.
So if you’re looking for a really big distraction in that situation, it worked.
But this also wasn’t my first rodeo. Alongside any of the research that you’ve ever seen for me, I have been managing complicated health stuff the whole time. And I started to think, I want to talk about this, because there’s something that’s been very profound here for me besides all the really good music, which honestly, I revolve my life around it. The thing that I really didn’t expect in this experience is that going through all this hard, weird stuff has made me a much better scientist.
And I think this is kind of counter to what we expect, right? We kind of are like, God, let me get through life, right, and hopefully nothing really strange will happen to me. Maybe, you know, nothing that’s demonstrably in the statistics very strange, but it might, and I wanted to share, if it does, you might be okay, and you might actually feel a little bit better on the other side of it than ever. Because the way that my rare disease taught me to think and problem solve and connect, my science has never been sharper. I want to propose something really radical. I think that we could all make this a reality especially in our industry. When things are always changing, we could look at something that actually makes us better at what we want to do in the world.
I want to tell you a little bit about that, because my experience is strange, but I think there’s something very universal in this. All around us in this room, we have aging parents, we have something happening with somebody’s kids, we have our pets, OK? You know? We have, like, real biological life around us all the time and that life is always going to throw curve balls at us. So maybe this talk is something like a few tips for doing science at the end of the world, your particular world, which was where I found myself.
The first thing I want to start with is like the entanglement of access and evidence.
I love measurement, and measurement is a very contentious topic in software spaces, so some of us probably have very strong feelings about being measured as human beings. When I first started working as a quantitative scientist in this industry, I was really gung-ho on putting out content that says, we can bring rigorous methods of science in, we can use it to take a structural approach to how we study developer experience. We can look at things like interesting questions that we’ve had, you know, how could our developers achieve learning outcomes. And this was not exactly welcomed with open arms at the time when I started doing this. I’ll never forget this day I logged into my LinkedIn and I found a quite well known software influencer had decided to leave comments after comments on my little baby posts, which I don’t know, like 20 views each at that point and the comments were telling me, a psychologist like you does not belong here in tech, is not welcome in tech. Any attempt to measure what happens to humans is unethical.
But when I was diagnosed with this rare disease, I had a million measurement questions, my world had changed from where I could get what I needed from like the splice particular averages and the generalized assumptions that we can make about millions of people. My world went like whew, you know, there’s me and there’s the people like me and we need to know what happens to us. The math of my life is different forever
And in order to know how to take care of myself, I needed that information. The first people I needed are the people who put together those measurements, and the thing you learn when you have a rare disease is, we don’t know a goddamned thing about rare diseases, OK? This just is so annoying to me. There is one conference in the world or at least one English the speaking world, US/Canada kind of conference and it happened the very week I got diagnosed which was like so amazing, so I want you to picture like, close my lab, I get diagnosed, by the weekend I’m in a new conference. So you can take the girl out of the lab, you cannot take the lab out of the girl, right? at the same time I was scared, embarrassed. I changed my Zoom name to like a made-up name. I took my picture down. We’re going hacker self. I was in my sick girl playlist area. But I was very excited because there was a session at this conference that brought one of the few specialists in the world, somebody who actually authored the clinical care guide for this condition — there was only one of those, as well, and brought this person in front of anyone who came to this conference, right, we’ll get 30 minutes of this person’s time. And one of the sessions — I love that she did this, like, so big of her. And one of the sessions was about the frontier of research. Now, this disease is really complicated. A lot of conditions are complicated. This one is just kind of bananas, so it’s full of cell interactions and signaling pathways and systems that are hard to measure and all kinds of things going on. But unfortunately that complexity makes it very, very difficult for people who are struggling to understand what she should do in their lives, so it’s kind of a target for snake oil salesmen.
I was sitting there in the Zoom and I’m looking at the Zoom chat, and I’m looking at the science. Two parts of me following the science and the Zoom chat is full of people who are lost, confused, same questions over and over again. Researcher going on with cell names. I married a biologist, so I understand trying to keep up with that. So in that moment I dropped my mask and I came on the video. It was like the scientist in me could not let this stand. I translated what I was seeing in the chat into questions about outcome measures and I really asked some hard questions, actually and it was a very proud moment for me that this scientist asked me to common to camera. We had a back and forth about like the state of the research, the strength of evidence about emerging treatments and I was asked to write a little bit of that up, like in the Zoom chat and in our conference stuff afterwards.
The best part about this was the messages that I got from people saying thank you. One woman said I’m so sorry this happened to you, too, but we’re so lucky to have you.
Because I speak the language of evidence and measurement, I realized I have a superpower and after several years of feeling emotions that ranged from like mildly embarrassed to downright defensive about trying to defend measurement of humans in front of developer teams, those feelings are gone. Like we’re in a new Cat era with that. Thank you.
[applause]
Evidence is a light in the goddamned darkness and when you have the scariest decisions in your life, you will hope someone tried to measure the things you’re going through.
The next tip for the science at the end of the world, right, that I learned is about efficacy versus effectiveness. Efficacy for me is the stuff that starts you on the path of surviving, but effectiveness is the stuff that keeps you on it. So what are these concepts any need to use a somewhat dangerous medication and it has to be used like the exact right way to halt the progression of this disease and it’s not that hard once you wrap your head around it, but you know, it was a lot to take in. So we looked to the science and the science tells us about something that we call the mechanism of action.
There’s a reason that people say, X changes Y, so it’s worth doing, right? So you can trust this drug. Kind of like a fundamental contribution of science to our world. Should I take this scary drug, right?
That is efficacy, and we go through huge lengths to prove efficacy, but my rare disease has also taught me to think very deeply about effectiveness, you see, there’s some qualitative research about what happens with people who live with this condition and we find that a lot of people stop using this medication. One of the reasons is it’s really hard to learn to apply it correctly. That is effectiveness, something in the real world, so you roll something out, how are people actually using it. Now, as long as we have information flow that’s really working, communal kind of flow, I think we can solve this effectiveness issue. How I saw this happen in my community is, in this one patient community that I became connected to, they started putting up YouTube videos with these few experts, and they recognized that people — not everybody was me who was going to read through this whole clinical book and they made — I’m not kidding — like one YouTube video that is passed around like a secret talisman, you get inducted. This is like the one guy who gets on YouTube and says this is how you fix your plumbing for everyone who didn’t have a dad, which is also something I go to YouTube for. This single video has become this gold standard for learning and I am certain it has helped thousands of people stick with this and do it right. Peer to peer learning in my opinion is perhaps our mechanism of action to reach effectiveness.
All right, I think a lot, too, about how our evidence needs to be both deep and wide, like, cover a wide distance. Kind of an example to think about this. At that conference there was a doctor who came to speak with us and she told us a story about how, just like everyone who has this rare disease, as someone who studied the rare disease, she felt super alone. She had always been a solo practitioners — they do this kind of rite of having research on the side and also maybe you’re the only person who can treat this so she’s built up this deep knowledge, this amazing information across caseloads. So we all know if you can get into one of these people, but she had this frustration that she couldn’t go broad. She couldn’t talk to other populations across the world, share notes, right, like the power of data science has not come to rare disease necessarily and she ripped out a page in her notebook and she wrote on it, do you do this specific kind of thing? Do you treat this, which is really specialized and would you join a data-sharing project for it? Would you be interested in that? And just passed it around the rows. This happened in 2023 and from that, she created the very first data-sharing collective for this disease — it makes me cry because I love data so much, you guys.
So the first data-sharing collective, 2023, I was diagnosed this year, 2025. This summer we heard some of the very first insights from this network of practitioners and researchers, which was amazing news, because it told us that many of the worst-case outcomes of this condition are actually far less likely than we thought. The textbooks are super outdated and the only way that people knew was to get talking with each other and sharing. So do not make a mistake of thinking that every manual has been written. Do not underestimate the power of being that one person who passes the notes around in the back of the room. This condition has also give us — kind of like a coherent framework to help us get out of chaos and start to build systems of prediction and falsifiability. Now, when we have a theory, we can start to move from the land of just like — I saw this, I felt this, I felt this, and like our intuitive explanations and develop like a shared collective understanding.
I threw this up here because I just love this paper I wrote with my collaborator on this. We put together a theory of developer problem solving and what we wanted to do in this work was pull from social psychology, pull from the theoretical frameworks that psychologists now about how human beings problem solve and propose that our communal networks in technology are drivers of information much more than lone geniuses. With this theory you can falsify prediction. That’s something I love to do, create those falsifiable, testable theories.
Before I was diagnosed, I had no unifying framework and you have to get in there with doctors, which can be awful, the glue work, right? But also you have to start testing hypotheses for what’s happening to you, once you have a diagnosis you have that theory and you can keep testing it over time and it becomes this amazing pathway to bring your subjective experience together with like the structured information that you are receiving. So I really believe everyone needs to bring theoretical thinking to their life. Another tip for your science at the end of the world is this made me really value descriptive research. Often you’re asked to predict things. It’s kind of like you feel like a vending machine as a scientist. But we really need more describing in the world.
So for instance, when I name like important psychological experiences that software teams have, we sit down and observe them together, often like it just become very, very obvious what is happening. We can make things legible to each other if we’re allowing ourselves to have the time and space to like really map all these chaotic different signals, put them in a picture together.
All right. Bit of a challenging tip, but one I really believe in. Having this bad rare crazy thing happen to me taught me a lot about what we owe to each other.
I am in a long-running group chat with an anti-vaxxer right now and she’s about my age. We have a lot to connect on. She’s had a hard time finding a doctor. She’s kind of a techy, we talk about that, actually and she’s an anti-vaxxer. Sometimes she sends me things that are like I truly do not believe in and that I think are really quite damaging if I did believe them.
There is a thing about a thing happening to you like a rare disease, it is a leveler, it happens across money and belief systems and jobs and race and gender and all of these things. You are sharing something with people that you never imagined you would have shared things with, so we have this little group chat and we all decides, we have to keep her in the group chat. We cannot exclude this person. There are so few of us.
So we work on it together. Causality is really, really hard, a lot of people struggle with it. I see these kinds of biases show up in the way that my friend thinks. I see her think about, you know, — make extrapolations from very small groups of people, like her friends, you know, and weight those. I cheerfully text her back my in examples and my stories and just like with lots of warm, friendly emojis, try to weight the selection that she’s biased on here. And I think I’ve made an in-road on this. There’s also availability bias. Every single thing that is salient in her life she thinks is the cause of her disease, this is frustrating, but I also find it hilarious. Today it’s the muffin, tomorrow it’s the coffee, but I’m there for her, and say listen, life is hard to interpret.
The information that feels really cognitively accessible is the information that our minds want to grab. We also of course have confirmation bias. This is when you interpret all the information you receive as only fitting into your preexisting narrative. I’ve been thinking about reading about this for a long time because I really care about my friend. She’s a real sweetie and I care about her a lot. I’ve found if you work laterally, kind of build out new theories for this person, much of the medical language that she hears from her doctors is scary for her. I use — I also shamelessly tell her, I have a friend that works at Pfizer — I do, this is true — who cares a lot about you and that shifted her belief that companies like that could perhaps create medicine that could be trusted. So there’s a path forward. I think the path forward is psychological safety. Maybe not all of us can extend psychological safety to like every person out there, but I had the capacity to do it for this person.
And I know in tech we love to talk about psych safety, but it’s often frustrating for me as a psychologist, because we sort of describe it as being like warm and fuzzy. But I think of psych safety as an information problem. When people in our experience an environment where there are fewer interpersonal risks for volunteering difficult information, that is when you will start to see the collaboration and the information sharing that characterizes psych safety. So you know, I feel like our cognitive biases are the strongest when we are trying to design and interpret evidence alone, and the more that we can pull together, create these little pockets of psych safety, the more progress we might make and my friend who is afraid of vaccines does text me before and after each doctor’s appointment now, to ask me what I think. So I think that’s important.
I’ve always been a little bit of a methods nerd in psychology, but this rare disease experience has made me think really carefully about this. One of the most frustrating things to me as I read like the few papers — I read all of them because there aren’t that many that come out, as I dig deep into their supplemental methods, I realize these surveys suck. I’m a survey expert and this would not stand for me. They’re running incomplete symptom inventories, they’re using jargon that I already know the patients won’t understand, so I started working with a pro bono survey project to share with the communities that I’m a part of. It’s been a really hilarious and chaotic project. It’s really fun to dump messages into forums of frustrated people and say, what would you want to be measured. And it’s deeply insightful. I’ve known that patients have a lot of words to describe symptoms that doctors would not know how to disaggregate. I’ve learned that we really need science translation.
I have seen that it’s really empowering to just look the problem in the face and say to a group of people: Do you hate the way we’re being measured? Let’s do something about it. Let’s, you know distribute the means of knowledge production, like, to the people, right?
So I’m collaborating with someone now who’s on the other side of the planet who’s been as frustrated as I am who also has this condition, we’re pretty sure that we’re currently designing one of the best and only comprehensive inventory surveys that’s been created for this condition so look out for that, I guess. Thank you.
[applause]
Yeah, symptoms inventories, either way. All right. I want to name some of the values I’ve been describing. As a social scientist I have long been really inspired by like this collection of research philosophies that come from different traditions, but they’re called participatory action research. In action research you try to affirm the value of people’s lived experiences. It’s trying to fix some of the big failures of science to not reach people and answer people’s needs, so what we do is listen to and start our design from the lived experience of people. We think about research projects as being things that need to have immediate benefit. Maybe even right now for the people who are in the project themselves.
I have always tried to reach this with my developer work. Now that I am also a participant and a researcher at every other thing I can get my hands into, I feel the value of this even more deeply. This is the vision that I believe in, bringing rigorous empirical studies, theory, evidence development, the thing that we experts are good at, but bringing it always steered by the lived experience and needs of the real people in the real world. Together, I think that we can get to some really interesting answers to our hardest problems. All right, I’m going to skip to my last slide. Oh, I have to show you this, though: In my research studies, we find that so many technologists care about these same values, of community, of connection. What I’m showing you here is 92% of developers, that’s across a couple of studies where we ask this question, do you think that an important part of being a developer is helping other people? This is how many of you say yes.
I love this statistic.
All right. This is where you can find me. I have a book coming out next year called The Psychology of Software teams and if Monktoberfest alum people don’t buy Monktoberfest alum books, what are we even here for, right? Thank you.
[applause]
I write a research letter drive us towards a vision of rehumanizing tech. I also consult right now on evidence science strategy for software teams. Find me online. Thank you so much.
[applause]
