In this RedMonk conversation, Cassidy Williams, Senior Director of Developer Advocacy at GitHub, discusses GitHub Copilot’s role as a coding companion with Kate Holterhoff, senior analyst at RedMonk. Cassidy addresses Copilot’s evolution and newest features, its business value, adoption trends, and the importance of context. In addition to tackling skepticism about AI coding assistants among developers and engineering leadership, Cassidy emphasizes GitHub’s commitment to the open source community and the accessibility of Copilot for all developers.
This RedMonk conversation is sponsored by GitHub.
Links
Transcript
Kate Holterhoff (00:12)
Hello and welcome to this RedMonk conversation. My name is Kate Holterhoff, Senior Analyst at RedMonk, and with me today is Cassidy Williams, Senior Director of Developer Advocacy at GitHub. And you might also know her as Cassidoo on the internet. And Cassidy, I’m just so excited to have you join me on the MonkCast.
Cassidy Williams (00:28)
Thank you so much for having me. I’m excited to be here.
Kate Holterhoff (00:31)
Yes, the excitement is mutual. Cassidy, when I attended GitHub Universe in the fall, I loved your Copilot demo during the keynote and I have been dying to have you on to chat about all the cool new things that are happening in GitHub Copilot land. I know they’re numerous. So for the two or three folks out there who may not follow the AI code assistant world with the equal enthusiasm of you and I, could you just give us a quick intro to GitHub Copilot?
Cassidy Williams (01:00)
Yeah, so GitHub Copilot is a coding companion. And I think that’s really the shortest way that I could phrase it, but it’s a tool that helps you code with AI. And what I like about it in particular is that I think it’s very aptly named Copilot in that it doesn’t just write all your code for you, but it works alongside you and understands the intent that you have when you’re… doing a certain smart auto completion or when you’re asking for certain things in the chat and you’re very much in control and can choose how you want to use it, which I like a lot.
Kate Holterhoff (01:33)
Yeah, companion. Yeah, that does seem to be part of the parlance of all of these AI tools that are cropping up every other day, I do tend to think of Copilot as one of the first AI code assistants on the market. So I’m interested to hear how it’s evolved recently. Of course, there were a lot of announcements at Universe, but what would you say are some of the new things that we should be aware of?
Cassidy Williams (01:38)
Yeah, and it was, I think there were a few others, but this was definitely like the first big monster that came into the scene and everyone was like, wait, that’s kind of cool. So it was really fun to play with in those early days. I remember I was a GitHub star before joining GitHub and playing with it for the first time in like that early batch. It felt so magical. I remember I had it like write some regex for me and
I was like, this is the future, this is it. It was thrilling. And yeah, now there are other ones that have come on the scene, but I think because Copilot has definitely been around, there’s a blessing and a curse of that, of constantly innovating, but also being that like stable presence in the industry for sure. But yeah, in terms of things that you should know, oh, there’s so many things. Just in the…
demo itself, I had to like cram as many things as I could into that demo and it’d be just like, okay, don’t forget this feature and rehearsals were chaotic. But anyway, so when GitHub Copilot first came out, it was basically just really, really, really smart autocomplete. But since then, a bunch more things have been added and it’s been interesting to see how different developers use the different features because…
There’s also Copilot Chat where you can just ask questions. You could say, hey, can you explain this code or can you fix just this part or tell me how I should fix this part, that sort of thing. Something that came out during GitHub Universe was GitHub Copilot Edits, which allows you to do multi-file editing, which is pretty cool. So you could say, hey, I know that I want a component that’s based on this component. It’s going to have these features. Could you just write it for me? And, and.
It will add tests, it’ll add a note, it’ll add your imports, it’ll actually create the file and also show you a diff. And so you don’t actually have to use the code that it writes for you until you accept it, which is kind of nice to, you know, again, stay in control there. There’s also different agents that you can work with. And so for example, there’s the workspace agent. So you could say, hey workspace, are there any files in my code base that generally do this? Because as you have a much more massive code base, you might…
lose track of certain things. And it’ll actually do repo indexing and be able to navigate your code base for you and say, you want this particular graph component or whatever. And it’ll pull up. There’s also a GitHub agent. So you could say, hey, what pull requests do I have open right now? Or what do I need to do to fix issue number eight? And it’ll just work, which is really awesome. My
favorite feature that has come out recently that I’ve been using in so many repos is custom instructions. And that’s where you can say, hey, this is a silly example, but you could say, hey, Copilot, I want all of your responses to be in the voice of Blackbeard the pirate. all of your responses will be pirate themed, which is not practical, but it’s fun. And you can also do more serious things. For example, you could say, I want all of the react components I’ve
generate to have a use client at the top, or I want to always include a feature flag in a component or something like that. And it will just follow the instructions that you have, whether it’s like, I want you to follow certain linting standards, or I want you to make sure these tests are established or this code style is followed, which is pretty awesome. And I think those are the only ones I can think of off the top of my head. I know there’s more, but there’s so many, and it’s really exciting.
Kate Holterhoff (05:13)
Yes, well, I’m sure there’s a exacting list with all the different features if folks really want to have it all laid out for them. But I like that you focused on the fun ones. mean, this is pirate themed. I’m here for the fun, which obviously makes me want to flip this on its head and talk about the grumpiest developers I know. what I speak with, yes, all the sysadmins out there. Yes.
Cassidy Williams (05:17)
Sure, they’re…
there are documentation things. Yeah.
Mm-hmm. You ready?
Hello. We respect you all.
Kate Holterhoff (05:40)
Yes, exactly. know. they’re all on Hacker News and they have lots of opinions. So when I talk to this sort of grumpy subset of developers, I think AI code assistants can be a really great thing to bring up because they have such knee-jerk reactions to them.
it has historically been the folks who have been doing this for a decade plus that have been resistant to this. So when I think about this particular persona of developer here, they’ve been known to grouse about these tools. And I’m interested in hearing how you would respond to someone who’s maybe a little bit skeptical when they say, isn’t, you know, isn’t GitHub Copilot just chat GPT in an IDE.
Cassidy Williams (06:24)
I have so many opinions here and I need to keep an eye on the time. So first of all, I understand. I am someone who has been that grumpy person. I remember very, very early in my career where I was learning Java in Notepad and I was hand compiling it and I was like, yes, this is how code should be written. And then when I got access to the Eclipse IDE, so you know how long ago that was.
I was so mad that autocomplete existed. I was just like, how will people ever learn if they don’t have to write their print statements or whatever? I have, I have evolved from that because it’s nice to have help and it’s, it’s useful to have these tools that get you from A to B a little bit faster. And it’s not that you’re not using your brain. It’s not that you’re losing any control. It’s just, you’re getting assistance to get what’s already in your brain.
out a little bit faster. So that’s, I fully understand the sentiment because I have been there. It’s really nice for things to be a little bit easier and things for me to be a little bit faster. And I like to also think I talked about how Copilot’s like a companion. It is kind of like pair programming. And that’s something that I have really, really grown to appreciate over time in my career. Whenever I pair program with a human being, being able to kind of bounce ideas.
off of each other and say like, okay, I’m gonna build this. And then they say, well, are you sure? And they suggest a different way. And even if I disagree, it kind of forces me to think a little bit harder about how I write something. And I’ve found that as I develop software with Copilot, it feels very similar where sometimes it’ll suggest something and it’ll be spot on and I’ll be like, that was nice and convenient. That’s pretty great. And I could just auto complete move on with my life.
But then there’s times where it does suggest something wrong because that is just the nature of non-deterministic systems. And it does force me to think like, now, why would it do that? What about the rest of my code isn’t clear or what about my intentions aren’t clear that would make it suggest something like that? And there have been times where a wrong suggestion actually makes me do something better elsewhere in the code base because clearly I’ve made something that isn’t fully understandable. And so anyway,
I think it is definitely a to each their own thing. I would never force something down someone’s throat, but at the same time, it is truly a tool that can only help you go a little bit faster or at least think a little bit better as you build with it. And something that I like is that it’s not always just in your face, like AI has to be here right now in your face all the time. Always. It’s nice to sometimes like quietly turn it off, do some code, turn it back on and be like, okay, what do you think of this? And then just
see what it says. There have been times where I’ve done that and I’ve had it like review my code and I look and I’m just like, okay, I wrote this terribly. And you know what, that’s okay. I’ve got my little AI code review and when we can clean it up and I don’t have to be embarrassed by a human roasting me. Instead, it’s a computer roasting me, which I don’t know if it’s worse or not. But anyway, all that being said, I get the sentiment for sure. And I think it’s one of those things where
You use it, you try it, and if it’s not for you, it’s not for you. But if it is, which honestly it has been for most people that I’ve worked with, it can be a real game changer to unlock you to that next step of not only speed, but also just kind of enjoyment in how you think about coding.
Kate Holterhoff (09:51)
All right, so say we’ve converted the grumpiest developers in the room. I’d like to change our perspective a little bit by thinking through some other folks who maybe have some hesitations about adopting these kind of tools. And that would be engineering leadership. They’re the ones usually in charge of making dev tool purchasing decisions. And at the end of the day, they’re responsible for the privacy implications that this adoption might come with it. I’ve heard some pretty specious numbers relating to productivity gains. so between you and me and everyone listening to this, what would you say it is? Let’s whisper it. God.
Cassidy Williams (10:25)
It’s all a secret.
Okay. It’ll be an ASMR podcast from now on talking about usage numbers
and productivity.
Kate Holterhoff (10:37)
Yes, So that said, what would you say when folks do come at you about the business value of Copilot?
Cassidy Williams (10:44)
Sure.
Yeah. And I think that’s where, first of all, the numbers do speak for themselves and how fast someone can work and how it can unlock someone. But there’s also a lot more that Copilot does that a typical AI tool or LLM tool might not necessarily do. And this is something that I probably should have mentioned in my previous response as well, in that there’s, for example, things like toxicity filters.
And so it’ll make sure you’re, you know, it’ll detect and discard bad stuff, both in how you prompt and what is actually produced. In terms of like the security side of things, we don’t retain your prompting in any way, or form. And so developers can ask whatever questions they want. And it doesn’t have to be anything that is stored or trained upon or anything.
We make sure that there’s identifier filters that make sure you’re not putting in anything that might be legally ambiguous or anything, no gray areas. There’s code quality filters that once again can only help your engineers move that much faster and reduces any security vulnerabilities that might happen. There’s also duplication checks to make sure that there’s
you’re not just straight up copying another repo on the internet or code that might actually exist. And once your users, your developers on your team get those suggestions, we at GitHub just discard it and it’s gone. And you get your suggestions, you choose what you want to use with it. And so we’re not just trying to take over your development team, we’re working with your development team to give you best practices in terms of security, in terms of efficiency.
terms of all sorts of things, this is such a businessy answer. But anyway, all of these things to empower them to build better and not just take over their work.
Kate Holterhoff (12:37)
And since we are sharing deep business secrets on this podcast,
Cassidy Williams (12:40)
Yes, business.
Kate Holterhoff (12:41)
I suspect you won’t be sharing any numbers with us today but I want to hear a little bit about how adoption has shifted from like classic dev tool rollouts. I assume you know there there’s been some some shifts that you’re seeing either anecdotally or you know just in terms of the numbers that you’re privy to.
Cassidy Williams (12:50)
Mm. Yeah, and I don’t know all the numbers and so it’s a good thing. I probably am not allowed to say the numbers. What’s been particularly cool is this past December, we added a free tier for Co-Pilot. And so anybody can use it for free. No strengths attached, no trial time or no questions asked. And what’s been particularly cool for me to see is all of the people
who couldn’t afford AI tools now using it and being empowered by it and being able to build with it. And I think that’s really, really exciting to see because making things more accessible to the masses is the most cool thing. And that’s something that I particularly am really, really excited to see. And similar, not just geographically, but also groups like students and people who are just kind of learning to code and aren’t
entirely sure how to get from A to B, like they aren’t entirely sure how to Google the problem that they’re trying to solve. And a tool like this can get them there. Where I was actually talking to a medical student this past week where he’s just like, I don’t know anything about coding, but I used Copilot and I was able to make this thing that helps me filter a spreadsheet. That was amazing. And it was such a small thing, but it’s really powerful that he was able to kind of build something on his own.
with this tool and not have to kind of feel silly and stumble through the process nearly as much. And so I think just providing access has been the most exciting thing to see. And we see various expertise ranges and various levels of seniority across the spectrum using Copilot. And something that’s been interesting for me to see in particular too is
More senior engineers tend to just use the autocomplete. Like they lean on the autocomplete just because that’s their coding, they get their autocompletes and they continue. But people who are earlier in their careers tend to use chat a little bit more, which makes sense because chat GPT and other tools that might’ve introduced them to the concept of AI tooling are very chat-based. But it is interesting to see that.
perspective and I think it’s a little bit too early to say like will they transition into the non-chat experience or will chat be just the future of it but it’s been cool to see those perspectives.
Kate Holterhoff (15:25)
Yeah, I agree. And I think that that really resonates with what we’re seeing at Redmonk, which has to do with the fact that we’ve moved beyond just early adopters, the hobbyists, the folks who like to tinker with these exciting new toys. And we’ve settled into more of like an enterprise mainstream groove. Like everybody’s using it. it’s become part of the workflow for work-a-day developers.
Cassidy Williams (15:43)
Yeah. Yeah, it’s something that I think surprises me but shouldn’t, just how mainstream it’s become. Because it is just a very natural thing that once again, it just kind of can exist wherever you code and whatever IDE you use. it’s something where it was so magical to me whenever it came out all those years ago that I was just like, wow.
someday people will really be able to access this magic and it’ll be cool. And now I’m like, now it’s not, it’s like, it’s still magical, but people are just using it. And it just is part of the tool chain now that people use. it’s, that’s, that’s kind of wild, but this is the future, I guess.
Kate Holterhoff (16:25)
Yeah, it absolutely is. And I like that you mentioned that you can use multiple IDEs for Copilot because it’s true that the AI code assistant space is becoming a little bit crowded at this point. yes, yes. developers have very strong preferences in terms of the IDEs that they want to use, the workflows that they prefer. So I’m interested to hear how Copilot is sort of meeting those challenges and making sure that it’s got a feature set that’s going to be desirable for so many developers to use.
Cassidy Williams (16:29)
Yeah, and I’m going to sound so biased, but I promised I thought this way before I joined the company. But I like that Copilot gives me those choices where if I do want to use something that’s not VS Code, like IntelliJ or something, I can because there’s a Copilot extension. It’s very IDE agnostic and at GitHub University released Xcode support. And so now it’s available in there. Now, because we are a company owned by Microsoft,
VS Code will be the first class experience for sure and the thing that works the best, but I like having the option. And I am someone who is very opinionated about which plugins I use, which plugins I toggle for certain things, which features I use and disable in my editor. And I like that it just works with that. And so I can be my nitpicky grumpy developer self.
And it’ll just work with me on that. And that’s something that I appreciate that the team has really tried to emphasize over time is enable developer choice as much as possible.
Kate Holterhoff (17:57)
Yeah, I think that’s a real gold standard. And it actually really pairs well with some of the research that I’ve done into this space, which has had to do with trying to track what it is that developers are saying that they actually want out of these code assistants. so I’m interested in digging into one of those in particular. And that’s the idea of context. Maybe I should put quote marks around that. Because I think that the definition sort of changes depending on who you talk to. And so it’s super interesting for that reason. But what is absolutely true is that if you go to places where developers are congregating to talk about these tools, they all talk about context and how important it is. And so I’m interested to hear a little bit about this idea of context, because I will die on the hill that context is what is going to define the killer app. It comes up so often. It’s so important. It really seems to be at the cusp of how these tools are evolving and what’s going to make them.
Cassidy Williams (18:42)
Yeah.
Kate Holterhoff (18:50)
be the most beneficial to developers at the end of the day. So what is your sense around context? mean, does it strike you as being as deeply, even like existentially important to the success of these tools as sort of I’m articulating it?
Cassidy Williams (19:04)
I love how you phrased that and I agree with you. I hadn’t thought of it that way, but I like that angle of it. Yeah, because it’s true, context is everything. If I just say like, hey, write a function that does this, it might not do it in the language that I expect. It might not actually follow my instructions absolutely perfectly. It might not know who I am or what I actually wanted to do. I could be saying, I want you to do…
a function that adds two numbers together in JavaScript, but it’ll give me an Excel formula or something. So I do think like that kind of context matters both at the small level, but also at a much larger level when you’re looking at the context of a code base and a whole system and the tools that it’s using and everything. I do think that context will ultimately be a huge aspect of that. And one feature that…
Again, I sound so shilly. But once again, one feature that I think helps that a lot and kind of speaks to what I’ve been talking about and what we’re talking about now is Copilot Extensions. And it’s a feature where you can build your own extensions to GitHub Copilot. So you could say, for example, there’s a LaunchDarkly extension where you can create feature flags that are automatically enabled or disabled just in your IDE right there by Copilot.
And there’s a bunch of other really cool ones. There’s a Stack Overflow one, there’s a Sentry one. I made a brainstorming one. You can build your own extensions. And I think something that will be very cool, and not a lot of it has happened yet, but something I think that will be very cool is people building extensions to provide more context for what they want Copilot to do. Whether they build an internal one where it’s just providing internal documentation for this is how our system.
it should work in an ideal world. Make sure all of your suggestions and everything are based on that and that sort of thing. But also, I think having the context of how code is written in a broader sense outside of your just given workspace and what is indexed in your repo is really useful again for security vulnerabilities and then also just for moving forward. But right now, I think a lot of AI models
particularly in the coding space, are limited by when they were trained, where they’re really, really good at stuff trained before 2021, some a little bit before 2023, you know, those sorts of really specific things. So how do you give the models the context, whether it’s documentation, whether it’s just a massive prompt and like a context window or the given code base itself, to continue to provide you with
good suggestions and best practices without needing to know what the latest and greatest technologies are using at the given moment. It’s kind of turned into a ramble, but I like the idea of that, how context is king and probably will be the differentiator moving forward.
Kate Holterhoff (22:00)
I love where this conversation has gone. I have one last major question for you and that has to do with sort of, I know, the legacy of GitHub, which has been so tied historically to the open source community. So can you talk to me about how GitHub’s positioning in terms of the Copilot tool is continuing their commitment to the open source community?
Cassidy Williams (22:11)
Yeah.
I love this because this is basically what my job is right now is continuing to work with the open source community. So you don’t feel, I don’t want to say left behind, but we want to continue to work with the open source community because that is what made GitHub GitHub. Everything that we do is because the open source community has worked with us, has worked amongst us. And the tools that we build are ultimately for the open source community because
If software is eating the world, think open source is going to eat software and open source is the core of everything, I think, and everything that we build. I very, very strongly think this way. so what is great is everybody internally is pretty aligned on that and stuff. And we want to advance and show all of these awesome tools. We want to be
first and foremost on people’s minds when people think of some of these tools. But we also know that open source maintainers have very specific concerns about certain things. For example, fighting spam, or for example, figuring out how do we bring in new contributors that aren’t just saying, hey, fix this bug, and then throwing a PR out there, and then they have to figure out how to code review it and work with the human who might not necessarily be the human that wrote the code.
That sort of thing. And it’s really interesting problems and there’s been some cool tools that we’ve been building internally and cool discussions we’ve been having on making sure that maintainers are well taken care of so that all of these tools that we do build, whether it’s Copilot or non-copilot GitHub things like actions and updates to issues and updates to pull requests and that sort of thing, ultimately make
it easier to maintain an open source community and a contributor community. Some of this sounds kind of hand wavy, but we care about it deeply. something that I will bring up is in a few months is maintainer month. And we’re going to be having a bunch of events and talks and all kinds of things to talk to open source maintainers and folks out there who want to ultimately be an open source maintainer or contribute to open source. And we want to figure out how we can enable developers out there to write more for open source and build the future of software in the open.
Kate Holterhoff (24:44)
On that inspiring note, think that we could wrap up this conversation. I love talking AI code assistants. I love talking Copilot. This has been great. So before we go, how can folks hear more from you, Cassidy? What are your preferred social channels? And beyond this event coming up next month, do you have any other speaking engagements lined up?
Cassidy Williams (25:06)
gee, so you can find me at Cassidoo, C-A-S-S-I-D-O-O on most things on the internet. Lately I’ve been spending most time on Blue Sky and just my own blog, so casadoo.co. And then I also have a weekly newsletter. So if you wanna practice coding questions or hear some tech news, that’s where I do things. But besides that,
Check out the GitHub blog for a bunch of events and things that are coming up for the GitHub team. Maintainer month is coming up in May and that’ll be really awesome to see all of the new things coming out for maintainers. And then also keep an eye out on the change log at GitHub because there’s some really cool features that developers might be interested in.
Kate Holterhoff (25:44)
I’ve really enjoyed speaking with you, Cassidy. Again, my name is Kate Holterhoff, senior analyst at RedMonk. If you enjoyed this conversation, please like, subscribe, and review the MonkCast on your podcast platform of choice. If you are watching us on RedMonk’s YouTube channel, please like, subscribe, and engage with us in the comments.
No Comments