What happens when you grow up without school? How do you navigate the world without any concept of the structure, organization, metrics and testing involved? This talk takes the lessons learned from that experience and provides guidelines to build more productive resilient teams that don’t merely survive, but thrive – and explores the statistically correlated relationship between thriving and productivity. It’s based on objective and analytical studies of the importance of factors like agency, culture, motivation and support.  

Transcript

All right, hi, folks, I’m Cat — and I work with Carol and Morgan, so I’m so lucky here. And yesterday as I was listening to all of the talks, I decided I was going to take a moment to do one of the riskier things I’ve ever tried to do in a talk and I’m going to give you a vote about which version of this talk you’re gonna hear, all right? Yes, yes, social scientists love to ask people questions. So please raise your hands if you’d like to hear where we’re going wrong with developer productivity. OK, all right, and then the talk that I had a very scary like moment of awakening yesterday, realizing, I think this is a crowd that might like this talk: What I learned about productivity from growing up without school. I did it to myself. All right.

     All right, I put these slides first, so I think you know I knew this was coming. Thank you for this.  — I’m Cat Hicks, the very first test that I can remember taking was an original assessment designed by my dad, and it was called a test of courage.

     Now, the test of courage looked something like this. He took my three little brothers and me out to the very boundary of our property in northeastern Pennsylvania, so we had these deep fields that led into these very, very large acres of woods.

     And we did at sunset and we lit a beautiful fire and he took one kid at a time in the woods and he had you close your eyes and then he left you there. And your job was to make your way back to the fire in the dark, alone.

     Which was really delightfully terrifying, but I never refused to do it. And sometimes I wonder why I never refused to do it, and I think it had to do with this: It is really powerful to be asked to be brave, right?

     So fast forward ten years from that test of courage, and I faced a very different problem in the world. But it turned out this problem was also going to involve a lot of tests. I decided to go to college.

     Now, I wasn’t super, super clear on what college was, but I had the picture in my head something like this, green fields, big school building, you know, probably made of brick. And a typical college application, as you probably know, consists mostly of measurements of you and your experience, what you did, right? And we do this because, as Carol just gave a very great clear talk on, we need to define success and try to capture it somehow in the world, right? And so we do this in college applications with things like grades and transcripts and letters from guidance counselors or teachers, lists of extra curriculars, boiled into and we want to standardize these. And I did note have any one of those things. I was nothing like a standardized applicant, right? So I had no grades, I had no transcript, I had no guidance counselor.

     I grew up in the woods without ever going to school. And I mostly taught myself through high school. But I really, really wanted to get to whatever this thing was. I wasn’t super-clear on the details of college.

     But I decided if I could — I knew I had done a lot. If I could just have a conversation with college, we could work this out. And then I realized I didn’t know how to talk to college.

     But college was foolish enough to put its phone number online. And I had access to the internet in about 20-minute increments at the Public Library.

     So I logged in, and I looked for the names of college. So like Harvard and Yale and Princeton were the only names that I knew, right?

     And I found their phone numbers, and I wrote them down on a grocery receipt in my pocket.

     And I called college. And a bunch of confused 20-something, probably student workers in admissions offices answered the phone because they still did that back then, and we proceeded to have a conversation where I got very strong impression that college wanted nothing to do with me, OK?

     We have absolutely no idea what you should do to apply to our college, one person said.

     Another person said, if you don’t have any data, I don’t know that you’re real.

     And finally, kind of far down this list of very prestigious schools, one compassionate kid, you know, just a few years older than me, actually working in this admissions office, said I don’t know what you should do, but you could take a lot of tests, a lot of standardized tests. OK, awesome, I was elated, actually, because I was really good at solving quests and doing missions. Maybe this was like homework, which I’d heard about. So I was ready, I was going to achieve, we were going to rock it. As soon as I figured out what a standardized test was. So I wrote that down on the grocery receipt, too.

     I don’t know if you know about this ecosystem, but at least a few years ago, there were about 20SAT2 tests administered in the US and maybe something like 38 AP subject tests. Now, I made a little bit of bank, because I worked all day while everyone else my age, like a loser, was in school. But I still only made about $7.25 an hour, so I knew I could not afford 58 tests. So what could I do? I went again to the library, because probably like most of you, I figured out that every problem should start going to something like a library.

     And they had a shelf of standardized test prep books, about a decade old, and I can still remember all of their, like, spines, you know, SAT-2, math 1, AP world history, AP psychology, AP art history for some reason. These — this shelf of books became the tests that I decided I was going to take.

     All right. To get to college plan was in motion. I was going to make it. I was really elated, I was really psyched, I was really proud of myself. And then a wrinkle in the plan emerged.

     I am a pretty good researcher. I can go to the library. I can find out answers to things I didn’t know.

     But I am a really bad test taker.

     Yeah, I see the looks on your faces. That was like me. When I took the PSAT the first time, I threw up in the bathroom. When I took the SAT, I threw up in my mouth, twice, and when I took the national math exam, I threw up in the trash can in the corner.

     I grew up on a sheep farm. I grew up with three brothers, I was regularly splattered in unpleasant substances, I could handle a little bit of puking, right? And so I kept going.

     But then I realized the real wrinkle in this plan. When I sat down to take the PSAT, my very first standardized test, I realized that the scariest test question on the test was the worst one. It was the box where they asked you to write down your name. Because I had been homeschooled my whole life, because my parents were deeply afraid of the government. Now, I knew actually about as much about government as I knew about college, so the details on this were very unclear to me. But I knew that we only left our property to go to church most of the time and I knew that I grew up hearing that what we were doing was very special and secret and that you don’t tell people what we’re doing, because the government might come and take you away. And I was sitting in the most governmental room I had ever been in, a public school classroom, to take the most governmental form I’d never seen something like this, and I looked at the box where you’re supposed to write down your name and I could not do it. I failed my first test.

     So hundreds of college brochures came to my home, I gave them my address, but I gave them a fake name. These came to my home addressed to Kartriona, Paltriona, some fake name I made up as a little kid. But they still doom came, hundreds of glossy brochures, and I realized maybe college is real, maybe they don’t know I’m real yet, but I saw the pictures, there were pictures of big green fields and red brick buildings and students who kinda looked like me on there.

     So I decided I was going to try again. And I sat down in another classroom, and I thought to myself, maybe the government will come to get me, I don’t know.

     But I have to try. Because I wanted to learn. I wanted to learn so badly, it felt like breathing. It felt like oxygen, it didn’t even really feel like a choice at that point. And I wrote down my real name.

     And the government came to get me. They came to give me a national merit scholarship and AP commendation, and some sort of AP board for for taking tests that anybody shouldn’t really take.

     Now, I know that for a lot of us, so many of the systems that we interact with in the world can feel like a rigged game, a very unfair system and the more you understand these systems, the more difficult it becomes, sometimes to stay in them or no know what you’re going to do with them. I went to college and I liked school so much that I went through a Ph.D. and I ended up doing a dissertation on classrooms, and I have worked on topics around how we measure people for the rest of my career so far, and I think that it’s really important to understand, you know, that all of us have some version of this struggle that we go through. We think about, is someone ever going to hear me, right? And we’ve heard so many important, beautiful talks around this topic. There’s a concept I really love on this that I’m going to share with you if you haven’t heard it, from the Kapoor center and it’s called the idea of distance traveled so this means in our definitions of success and our measurements, we can think about the distance someone had to travel to get there, to sign their name on the test and to do any of these things that we consider part of being our professional world, right? So in a school district about 45 minutes from where I lived, while I was tutoring a bunch of kids in SATs, I like to do that now, the school system there, cut the bus routes, because they couldn’t afford to run them and so a lot of the students that I worked with walked two hours to school and two hours back home.

     That is a lot of distance traveled.

     And I knew every one of those kids is real.

     And our developers, I think, are real, too. You have some of these kids on your teams, and in your organizations. So with that, I want to share a few stories that have come out of our lab about studying developer productivity.

     I’m going to take a moat of the time, because this is not how I meant to use this talk time and I’m going to try to go through a shortened version of how we come to bring this way of thinking to software teams in the hopes that you can use it and benefit it and maybe you can even be seen by it a little bit.

     So we try to approach everything with this mindset, that we’re looking for better data about software teams, right? Not just bigger data or data that kind of fits with a certain cycle, but that we’re going to bring best practices in social science to how we measure really important things that we know are happening for folks. So we do this at the dev success lab.com and these are tools that you I hope you can use * I’d like to try to open up access to this way of thinking. And here’s an example I’d like to share with you. This is managers who came into our studies and we asked them this question, do you think that technical effort, really seen invisible inside of your engineering workplace is a critical part of your job? So no surprise, almost all of the managers agree, about 88% across a large sample agree, yes, technical effort important to be seen. So what do you think happened when I asked the developers whether they felt this was happening. I see some smiles back there. All right, yeah, 24% of the developers in a large sample of about 1282 said no  — only 24% agreed it’s happening. Most of them said my manager does have the right level of visibility into my work. So what does this mean? Again, as my beautiful coworkers shared in the last talk, we see misalignment all over the place, in how we are looking the way people are doing the generic work.

     When we misalignment, we love to lean in. Leaning into this, when I think about how we think about productivity, as someone who’s grown up, becoming an achievement scientist, becoming a learning scientist and working in all these environments, I see these three big heuristics that people in software especially seem to be holding onto and carrying around with us. So the first heuristic is this belief that productivity is the stuff that is really hard. So when I ask developers in our research, can you describe a time you think you were really productive, they say, well, I stayed up all night on this one project, I was burning the midnight oil, so I was work working really hard. It’s true that you were working hard, right, but is that productive? We learn very fast to associate brutal, uncomfortable things with productivity when we place those two things together. Now, the second belief is by productivity by definition means never stopping. I see looks on your faces, you’re all people who probably already wrestled and gotten beyond these beliefs a little bit, but take seriously the way that this still creeps in. Yes, I can take a break, because then I will be able to max out later, and I’m an incredible high achiever right? So I can do double the work later — this is still productivity only means never stopping. Feel.

     Belief No. 3 that we see pervasively is that developer activity is very lonely. This is a developer in the dark, by themselves, fingers on the keyboard, and I think this one is tricky. I actually feel sometimes a little fear saying this to an audience like you, because I think there is a feeling, I want to protect our knowledge work, I want to have focus time, I want to care about that focus. That’s not what I’m saying here, right? I’m saying that you shouldn’t depend on isolating people as your only mechanism for getting work done, and in fact, that has a huge toll, right?

     All right, so what’s a different approach?

     Well, first let me game — sorry, I mixed my slides up last night. This I think these three beliefs give us a particular way of thinking about the world and this way of thinking about the world, I call brittle productivity and here’s the thing I want to say about it. In the world of brittle productivity, it’s really dangerous, because your org, your team, your people, you know, your kid struggling with taking a test, whatever, survive through school, it will look very productive if you’re choosing these strategies of never stopping or grinding. It will look very productivity, as long as nothing goes wrong, and then the thing that is wild about this is that as soon as something goes wrong and breaks, you think about topics like resilience and engineering, it breaks catastrophically, right? This happens for human beings. So how many of us have seen maybe there’s an engineering team that is so wonderful and they’re so reliable and they’re diligent and they have good collaboration, they have that alignment Morgan was talking about with product teams and so we’re going to hand them project after project after project and they’re not going to say no, and then one day, half of them quit and where’s your company now? Painful laughter, I heard at that one, right? Or maybe there’s a team that got laid off, because I don’t know, maybe they were just hanging out and they weren’t producing in the way that we would measure, right? And then I would give it maybe 6 weeks, not very long at all, before something enormous breaks. Maybe for your customers, maybe for your users, maybe for a hospital that your team supported or something like that, because it turns out that person was ensuring other people’s work, right? So what we want to move towards is and the modeled that my lab tries to put out is the idea of resilient productivity. And when something goes wrong, resilient productivity cycles deliver. This is kind of the key indicator for it, right? I see some nods because I think we’ve all kind of lived there, right? I don’t think I’m giving this talk to people who haven’t felt this.

     What I would love to say to you is that this is really important.

     And if you are letting other people shove you into the brittle productivity box even when you know it’s not gonna work, and you see a thing breaking. A lot of stuff in this world is going to break. My team is trying to put out a model around what we call developer thriving and I’m going to go through this quickly and give you an example of it and an example of the evidence of it. But it’s important for you to know that thriving is a concept that’s out there. It’s something that exists in the social sciences, in the clinical sciences, and we think about thriving as being better than OK. You’re flourishing, actually. I think that developers should demand thriving in their environments.

     And so what does it look like when we take thriving seriously? We put there, it is free for you to download, the. These are the four levers that we propose that you pay some serious attention to. Developer agency, which is when developers feel like they have ownership, they are creators of technical decisions. This can be boosted by supporting developer-driven initiatives. By supporting their bottoms-up sources of information about what is actually happening to change leadership decisions.

     Learning culture, this is not just individual learning, all right? A million developers come into my research and say, I absolutely love learning, life-long learner, of course we need to learn, new technology every day and then you start to ask them, do your team have a learning practice? Do they set goals together? If you say I need two more weeks and I need to upskill, will that space be made for you by your manager. That’s learning culture stuff, right?

     Learning self-efficacy. It has to do with this important mechanism we use to tell ourselves, I don’t know what I need to do, I don’t know what it is, but I’m going to figure it out, I can figure it out, right? And this is a place where engineering managers can play an incredibly important role.

     And support and belonging. We just had a wonderful talk walking you through belonging. There’s research on this, as well.

     And when look at this, this is data aggregated across those 1282 developers is, so many different points stacked on top of each other here, this is self-reported by them, which is actually the way software research likes to do this, because people can reflect back on something like the last work of month I had, on average how productive was I, do that at scale and no productivity. It’s complicated another talk that would I love to give about that. But when we look at those team scores on thriving and their productivity, we see a couple of different things, so again, social science way of thinking about this. Of course, productivity is complex. Many, many things drive it down like you woke up and you slept well or you didn’t sleep well, but we see a critical significant relationship between just their scores on thriving. Think about how easy, scalable, and cheap it is, to ask somebody if they feel like their team has a learning culture, or they have agency, we love to find predictive relationships at this level for something so complex.

     So another way you can think about this, too, is paying attention to the quadrants that drive these kinds of relationships around. So something that I noticed that some of people that study interventions, the small levers that we can use to make a classroom better or to make a team better, you start to pay attention, not just to like, is there a relationship over the whole population, right, but things like, do we see very many teams that are high in thriving but low in productivity. Of course all things exist in the world, you’ll get that answer from every scientist. So we do see some, but this is the kind of thing we look for our statistical models to say, there is evidence that stuff is a unique, important lever, and it’s one that we think you can affect. So we’ve replicated this across our research. We’re starting to build what I would call an empirical model for developer productivity. We see this across industries, across ages, if you start to measure what Carol introduced as a sociocognitive mechanisms for these things, to does help drive developer productivity.

     So there are more examples of our research and more to download, but I actually want to flip to the end of my slides here, and end this talk with something kind of personal, again.

     And that’s this. I saw this quote that I really loved, which was, you know, writers don’t get to exist unless you share our work, and I think that really applies to scientists, completely. Our research matters if it helps the people that we did research with. That’s the end of the day for us. And I founded the *developer success lab about a year ago, so it’s really beautiful to come to a space like this and get to talk about it. But when I told one of my closest friends that I was founding a research lab to study software teams, she said, Cat, I thought you studied people!

    [laughter]

     I love it, but it hurts.

     You are people!  To me, you are very real people. We are all very real people in this industry, and we deserve a social science for these amazing people that we’re working with, for the people that we are. We’re trying to build this social science. So please share our work, use it, engage with it. That is how we keep going. Thank you so much.

    [applause]