In this The Docs Are In conversation, Kate Holterhoff, industry analyst at RedMonk, and Sam Kleinman, staff engineer working in distributed systems, discuss documentation: the good, the bad, and how to improve the documentation ecosystem. Sam talks about his experience as a technical communication specialist at MongoDB, as well as his “spicy takes” on what makes the documentation strategies of some companies more successful than others.
Resources
Rather listen to this conversation as a podcast?
Transcript
Kate Holterhoff: I’m here with Sam Kleinman, staff engineer, working in distributed systems as part of RedMonk’s Doc’s Are In series, where we talk about all things docs from academic doctorates to documentation and technical communication more broadly. My name is Kate Holterhoff, doctor Kate Holterhoff, analyst at RedMonk and documentation enthusiast. I’ve brought Sam here today because he has a background as a technical writer and technical communication specialist, and so I want to just begin this conversation by having Sam tell us a little bit about his background, so tell us a little bit about how documentation factored into your life.
Sam Kleinman: Well, so I am I kind of like the like the punchline of the story is that I spent six years like being a technical writer, which I never intended to do. I went to a liberal arts college. I was always a nerdy person, but I sort of believed that you should, like, use your (this was wrong), but that you should use your college to do something kind of impractical. And I was like very committed to this, and so I majored in psychology and women’s studies. I was 21 or 22 and I was like, I need to have a job. What kind of computer thing can you do if you studied Judith Butler and not Guy Steele. You know, and like, how do you like how do you like move into that? And so I wrote a blog, which is kind of an anachronism, but I still do that some, and I accidentally sent out, I applied for a customer support job, and like signature signature files (there’s another anachronism) that I link to. My blog was in my signature file and someone looked at it and was like, “Oh, he can write. We should hire him to be our technical writer.” And then it all started, right, like that’s the beginning.
And I moved across the country. I sort of just like typed, I just like started writing like, like there is like a bunch of systems administration things. I knew how Linux worked a little, or I thought I did in the way that you can think you know how something works when you’re 22, and then I did that for a while. I ended up moving to New York City and got a job at a company that is now known as MongoDB as their first technical writer.
Because I had been writing systems administration guides like “Here’s how to install Apache on your Suse, or whatever. And then I. And I like. And so I was like the I started writing a bunch of a bunch of documentation for them because they needed, you know, one person to tell the story. I mean, documentation isn’t a story, but like you need to like have a bunch of different kinds of docs and sort of see the whole thing from a bay and from a big picture. And there’s a lot of this like systems administration kind of work in systems like MongoDB and so I could write that, and I could figure things out because, like, there’s a lot of being autodidactic that helps me learn how to write the documentation. Like I certainly talked to engineers, but mostly like I like, it was open source, I just like looked at the code. One of the interviews, I’ve given this interview a bunch of times where like MongoDB has this feature where it like it’s it’s like a very it kind of like simple index that deletes documents once they get older than a certain point. It’s like for like ETL kinds of like workloads where you want to expire data over time. And the way we documented that was like it was, it’s, it’s the original implementation is less than 200 lines of C++, the co-founder and CTO co-founder, like, wrote it clearly over a week and we were like sitting on a couch in the back of an office in Soho. And he was like, “I’m going to write some emails, read this code, write the documentation, and I’ll like look at it and tell you if it’s wrong.” Which is nerve wracking as hell. I did it, apparently, and it was actually like I then sort of for the next basically ten years, like had like tech writers, engineers just like read this, read this piece of code as a, as a like exercise of like: here, can you take something that you have never seen before in a language that you probably don’t know and like figure out what it’s doing because, like, you can’t see, you can use interpretive like skills to like kind of do that. And then MongoDB became MongoDB in the way that it is today, and the documentation. Like I, I left in like after like fourish years to, to go to, to sort of. Do do other kinds of engineering work and sort of become an engineer, which is like been my career since then. But I the thing like the way the MongoDB does documentation I think is like pretty cool as a as a. Not just that the docs are decent, which I imagine that they are, but they sort of like continue to sort of like it’s a very lean team. They’re very technical. They’re not just like translating comments from code. They’re like they’re they’re it’s a very it continues to be a very technical team and they, like, think a lot about sort of documentation is this like pedagogical instrument for like teaching people about the about the system. And so it fulfills both the like API reference for for developers but also like, like what does it mean to have a, have a distributed system as a, as a database that’s like capable of quasi autonomous operation? And, what are the things going on inside of the database? And that and the documentation sort of tells you how to use it, what’s possible and like how to how like ,what the procedures that you might if you are like not just like what’s put sorry, not just what’s possible but (which is important), but also how to use it. And then like what the nouns are in that or the verbs are in the system that you can use to sort of do all of those things.
Kate Holterhoff:I think what’s interesting about documentation and how it’s evolved over the past decade is maybe we’re moving towards a less reactive model. I mean, it’s so interesting, your story about, you know, sitting on the couch in Soho and, you know, looking at code and then creating the documentation. You know, I wonder with tools like copilot, if they’re going to allow us to write documentation, that then becomes the code, you know, as a form of like pseudocode, right? You know what you want to happen and then the code comes from there.
Sam Kleinman: I have always believed that. I feel like the history of documentation is like made up of (it sounds like it’s going to be really poetic and it isn’t), but like made up of a lot of like we’re going to build a tool that’s going to be that’s going to make it so that you don’t have to write documentation is based is that is the premise of like a lot like you know like and this what this gives you is things like Javadoc, which are not terrible, but, they’re terrible. Let’s just say it, like they’re not good, and like and and they certainly do present why you have a website that uses that like is produced as a result of your code comments, or your special code, like worse, your special code comments. I’ve spent a number of years writing rather a lot of Go code, and Go has a much more sensible approach to the Javadoc problem, which is that it has basically none of the features and is on the whole just is useful. And so that’s that’s nice. The thing that as an engineer once. So there’s two things. One, I must confess that as someone who has written a lot of documentation in the past and therefore knows kind of how the sausage is made, I basically never read documentation, which like causes, like, among other things, some amount of domestic distress, “why didn’t you read the instruction book for this?”
Kate Holterhoff: I mean, I love these spicy takes, so I’m curious.
Sam Kleinman: I didn’t think they were that spicy.
Kate Holterhoff: Well, I think what’s interesting about it is there’s been this real push to increase the amount of documentation and at least put more of a focus on it. And so I think of the DORA metrics. But what I have noticed is that documentation is treated as a sort of monolith, and what I’m getting from you a little bit is that documentation is a little more nuanced, and that there’s good and bad documentation—better and worse documentation—and there are ways to improve that.
Sam Kleinman: The wrong kind.
Kate Holterhoff: Yeah. So I guess, do you have any insight into like what sort of documentation specifically we should be focusing on, like what is useful? And maybe it has to do with the audience like, yeah, a staff engineer’s been doing this for over a decade. Clearly you don’t need it in the same way that like Stack Overflow is typically the domain of folks who are beginning, that really have these these sort of questions, you know, is that the issue? The documentation just isn’t for the type of engineer that you are?
Sam Kleinman: Yeah, real personal, right. Like for me. Right. Like your staff engineer who happened to spend six years writing documentation, and has a bunch of analytical textual skills in the art of close reading source code.
Kate Holterhoff: Right.
Sam Kleinman: Which maybe is all staff engineers, maybe isn’t. But so like I’m, you know, like I’m completely willing to believe that there’s like a lot of documentation that’s, that’s actually sort of useful. I think, like, there are a lot of different kinds of APIs which require which require better documentation. And you can imagine, you know, there are ways in which like programming languages which separate the headers from implementation details are sort of in their own way a form of documentation, right? Like C and C++ code like kind of has this like natural separation which lets you sort of think about about documentation. So to answer your actual question. I think it matters. Audience matters so much, and user story matters so much. A thing that we don’t see enough of in companies, in the corporate production of software, is that we don’t often have product managers. This is an actual “spicy take,” of like infrastructure teams need product managers because like, otherwise they will build completely random things that like no one wants, but that everyone then gets stuck using. And so, like maybe having a product manager to say “the business goals are X, and can we solve this problem not the, It would be really great if we had metrics of this time or or whatever, whatever the case are.” But I think that’s like very true of documentation and like these, like basic engineering questions of like 1) who are our users, 2) what are the problems that they’re trying to solve? 3) And how can we iterate towards that? I think the thing that I like, I told us this long story about MongoDB and my experience there, and I focus as I am wont to do on things like building the tools and the organization and the process, but it’s also probably the case. And so we can add this additional layer here. But there’s this like maybe in another story where I at, like people were like, we need to tell people how to change the size of their like replication operation log or off log. And, and this was like, you know, like, here’s an engineer who wrote. Who wrote a blog post that everyone used and I just like lifted it in because it was correct, and she was good and, you know, and, and, and so I like, I like and it’s like a kind of crazy procedure. And it’s still like I found it a couple of days ago because there were a few weeks ago because it’s kind of actually, you know, like stop your stop your your node and it started on this new node, and then run this very specific query, and and then started up again and things will be okay. Sounds kind of nuts. And, and I and I think that like you that like one of the things that we did is we like we sort of saw the kinds of questions that people needed. We saw like support teams were like, we need a procedure to tell people how to help to to solve this problem. And so we were able to kind of like address those needs. And and and so we I kind of. The thing that I like definitely that was absolutely happening is that those like user stories and, and, and business cases for like how do we want people to use the documentation, or things that were, were going on around me and I was sort of like focused on typing really fast, or focused on building the tools. And this is a thing that I think happens to all sorts of engineers, including technical writers, is that you you get focused on that and it’s really hard to sort of say like, okay, what are, you know, like take a step back and say, “what are that? What are we trying to accomplish? What are the business goals for the users? What’s the thing that’s most painful for them?” And, and it’s it’s so hard to do that because because that, like, it requires a completely different perspective and, and like way of thinking about, about your work. You can’t really sort of be like, okay, what’s the like, what’s the procedure that everyone needs to know about? I’m also going to go write this completely different tool to like do recursive, you know, composition of documentation and things like there’s just like very different things in that switch is like going from one to the other is like very hard. And so I think that people, I think that like having. Teams would be well served by having someone to sort of focus them from a product perspective. This is the “spicy take” is that is that we there is this now 30 year old sort of idea that you that things like wikis will democratize the documentation process. And what they actually do, I think, is diffuse responsibility for documentation. And so in some ways, like, while we need better tooling, we need tooling that supports ownership, not tooling that diffuses ownership. And so I think I actually think that that. In some that that that it’s kind of good that Google Docs killed the Wiki or the enterprise wiki and like who knows what notion he’s going to do and how how they’re going to come out of this. But the the enterprise wiki really sort of like. To my mind was a failure because you need people to manage to take on that sort of thing and no one ever did in any way or ever. And I think that like. And and I think that and I think that that’s that was like a definite dip in in the like documentation like in life world was that there is this like there’s this period of time when like everyone was like, “oh, we’re not going to hire technical writers, we’re going to just have a wiki and everyone’s going to edit it, and it’s going to be great.” And you’re like. Like, “No.” And If we can figure out a way of like building tools that support people and owners and, and, and owners of documentation, then like there is and, and without sort of like these like magic snake oil type like. “And then you won’t have to worry about documentation.” Like you will still need someone to care about it. Like you need someone to care about it or else it’s not going to be good.
Kate Holterhoff: Right. I think I mean, the way that I’m hearing what you’re saying is that documentation is relationships, that it becomes. Yeah, that it’s really it’s about people. It’s about the structure of the organization. You know, that there’s a leadership element and that there’s a way that they can’t just be these static sort of things living in their wikis. When I was doing frontend we would have “Documentapaloozas” and order pizza and everyone in the engineering team would get in a room together and update our wiki and, my God, yeah, we couldn’t get it done in a day. when we got a few done. But it was yeah, it was kind of gestures toward the need to have everybody in the room looking at this and that. Yeah. Trying to, to jam it into a quarterly event is just not going to work, especially because trying to get an entire engineering team together is like it was just impossible.
Sam Kleinman: I mean, I think there’s a lot of ways in which docket like that kind of engineer, like that’s just like not I’m sorry to interrupt for you, but like but like we I’ve also watched teams have similar sorts of approaches to like let’s all fix flaky tests on Friday or yeah. Sothere are a number of things that I’ve watched engineering organizations do 1) the rotation, the like somewhere it will be someone’s job to be miserable once a month, or 2) the party approach of like, let’s all do it. Like we’ll spend a week all together and so we’ll all be miserable. These are all sort of symptoms of we didn’t hire a dedicated someone. We did not hire the right person to do the job. We think is valuable but not so valuable that we’re willing to pay for it. And the truth in that that like taking your entire engineering team for a week for like, like I don’t know how big it was, but like, I, but like that’s, that’s enough to hire two technical writers.
Kate Holterhoff: That’s a good point.
And I and engineers do a lot of like, ah, like ironically, like really bad at is like cost benefit analyses about like hiring more people. Just hire to like hire two technical writers, and like it’ll like you’ll save money, like it’s worth it. So I think there are ways to like, like intelligent, like and so in some ways, like documentation is, is one of the things that you use to solve this like intelligibility of code problem. And I think that like. But it’s a solution and not the light. And we should keep asking ourselves that question, like, how do we make this? Like, how do we solve this problem of like, “I am, I am an intern and I want to be able to make a meaningful contribution to this code base inside of ten weeks or 12 weeks or however long the program is.” And that’s a problem Which flag is is a little bit on, like how do we scope in term projects and some of it’s on like how do we like keep our code base well-organized and, you know, like not with crazy legacy names.
Kate Holterhoff: Well, it’s interesting because I do think of documentation as being this what like the, the, the thing that comes out of brownfield projects that we have to write all of these exceptions and these reasons that things don’t work right or, you know, whatever plugin is breaking. And that’s why we need to like make a note that, you know, if you’re doing this in order to get it up locally, you have to turn this out, whatever, you know, whatever it is.
Sam Kleinman: Absolutely.
Kate Holterhoff: Greenfield is it seems like that’s what you’re you’re aspiring to a little bit, you know, or at least like polished.
Sam Kleinman
Yeah. I mean, but I think, like, you can add polish to systems and and. I don’t mean to sort of say that you never you never write documentation but that. And because obviously you need to and like comments are definitely like you definitely do want to add comments of the like there’s dragons here know that that know that there are dragons and yes but but and but like aspiring to. Like sometimes you fix problems by fixing problems, but sometimes you fix them by by making them not be problems. And, and, and there’s like and depending on which layer you’re working at or what the like constraints are, those are like options. So, like. Certainly. Certainly interesting.
Kate Holterhoff: All right. Well, I think that’s a good time for us to wrap up. So I want to thank Sam for coming on The Docs are In today. I’ve learned a lot. Okay. I like that. It went from, like, “tell me why docs are great” to “Maybe, do we need docs at all?Maybe there’s a better way.” Not what I anticipated. So, Sam, would you like to. To tell us where folks can find more information? You know, hear more of you spicy takes?
Sam Kleinman: Yeah, I don’t know, I’m on Twitter and tycoish.com are both there.
Kate Holterhoff: All right, I’ll put that in the notes. Well, thank you so much for coming on. And with that, the Docs are out.
Disclosure: While MongoDB is a RedMonk client this video is an independent piece of work and was not commissioned by any entity.
