Stephen O’Grady: Good morning, good afternoon, good evening, depending on where you’re listening to this from. We are here live at IBM Think, and I am with Matt and Ruchir. Matt and Ruchir, would you like to introduce yourselves?

Matt Hicks: Sure thing. I’m Matt Hicks, the CEO of Red Hat.

Ruchir Puri: Hey, thank you, Steve. I’m Ruchir Puri. I’m Chief Scientist for IBM Research.

Steve: Excellent. We’re here to talk about the intersection of open source and AI. IBM and Red Hat obviously collectively have a lot of experience in both of those things. So the obvious question, I think, for a lot of people is, AI and open source appear to be on a collision course. They’re certainly coming together. So the question for me for the two of you is, is this important? If so, how does open source impact artificial intelligence?

Matt: I can start and then we’ll see if Ruchir adds to it. But the first thing I would say is, I do think they’ll combine. I would say less of a collision course and more like an amplifier to it. The why is it important, I’m in the camp of — I think just every aspect of smaller models will be the most effective thing for enterprises. They’re easier to train, they’re cheaper to run, but to be able to make that work, you have to have a collaborative system where there can be an open innovation model driven to it. And so in terms of the, why does it matter? Our goal was put this in reach of developers on laptops, make this scalable to early bets and investment, make it an incremental process where you can see value versus just like a big black box that you have to depend on.

Steve: Right, sure.

Ruchir: I’ll just start with never bet against hungry graduate students who are for open innovation. They want to graduate. That’s a more techie answer. Having said that, I would really say we are getting to a point where models are becoming amazingly powerful, these small to medium sized models. And given what is happening in the community, the real value lies in the data of the enterprise. And what Matt just talked about, incrementally building upon these models and having them be in the open and having the technical community innovate on top of it, making them stronger and stronger, and the real value being unlocked when the enterprises add their data on top of it, which becomes obviously their private models. But it’s really the base, which is the foundation, which is open, benefits from the community. It’s exactly like the software ecosystem, actually, except now it’s for AI.

Steve: Yeah. So on that note, so both here at Think and certainly at Red Hat Summit, a couple of weeks ago, IBM and Red Hat have announced, well, two big things, right? So InstructLab and Granite. So why don’t we start, Ruchir, with Granite? Can you tell us about just what that is in brief?

Ruchir: So we are incredibly excited about what has been happening in the open community. And I’ll start with the following, actually, I think there is lot of versions of “open models” out there. If anybody has a boutique license and they say it’s open, it’s not really open.

Steve: No argument here.

Ruchir: I think we will all agree to that. Now, having said that, there weren’t that many alternatives before, and whether it is Llama or Gemini or Gemma or others, actually, we’re getting to a point where truly open models — which I’ll say assigned with a well known license not owned by the entity themselves, actually, like Apache 2 — those models are becoming very, very good. And we’ve been working on the Granite models for many, many years. We have built products on them already as well. And we really thought the juncture was right to bring these kind of models, as Matt said earlier on, small to medium size, but also some heading towards larger sizes as well. So it’s kind of a range from 3 to 34 billion parameters with transparency of the data as well, regarding how we build these models, what data actually went into building these models. And even more importantly, they are amazingly well performing. Because you can put something out there in open. If it doesn’t perform, I’m not very sure. The license may be great, but doesn’t work. Like, I don’t think you want to buy a car, which is a great flexible open car, but it really doesn’t work.

And I think we are really getting to that point. And since we’ve been using these models internally for building products and scaling out to enterprises, we are very, very confident in their performance. And for the technical report we put out, it’s one of the best performing models. And since several of us built it, I can say this honestly, it’s one of the best published reports I have seen to the point with truly honest comparisons across the board. We didn’t just check it on one benchmark, we did it on like 19 benchmarks across the board. So it’s really a milestone for us and for the community as well.

Steve: All right, so that’s the model. Right. And obviously there’s a question of contributing to models, which obviously brings us to InstructLab. So, Matt, you want to talk about that briefly?

Matt: Yeah. So I still remember the first demo we saw before it was called InstructLab, where the IBM research team overly simplified, were like, we’ve made training work like put some constraints on it. It’s within reach of mere mortals. What we saw on the Red Hat side was given a small package of data, we can get people working together on this. Like we can actually build a community. And we’ve kicked around a lot of the — I love the gravity that’s in Hugging Face, fine tunes just, it’s a hard thing to work together on. So you know, there’s desire, you know, there’s excitement, the ability to not just change a model, but then for me to work on your changes or you to work against Ruchir’s, that’s what makes this Linux on it. That’s been the exciting point of smaller models, as good as they are, they’re never going to come out of the box like a trillion parameter model. But if you can train them on your use case, it doesn’t matter. I run these on my laptop. It’s changed how I engineer and write code, but the ability to train it can make me make it my own.

But in parallel to that, the ability for me to work with your changes and your innovation around the world, that’s what taps into what we’re known for at Red Hat is that open source insight.

Steve: Yeah. So that begs, I guess, the next question, right, which is when we think about open source, there’s a certain process that comes with that, there’s a certain energy that comes with that, contributions and flow and all that. So clearly you have open model and you have a means of hopefully democratizing access to contributions. For your average user enterprise, pick your persona of choice, what do you hope this unlocks? And we’ll start with Matt.

Matt: Yeah, so two things. I think there are two elements in this, because AI is a bit different than software.

Steve: 100%.

Matt: It’s compute intensive, you need something. And so on one side, our worldwide goal is make it as easy as possible for you, me, anyone listening to contribute knowledge and skills. Like if I take my knowledge, my skills, and I’m willing to contribute them, how do I get them in a model? We’ve said like, we will take the compute intensive part. If you make that contribution, we’ll do this new thing of building it into a foundation model. So you can now get this working in a model. I think that’s really important for building that high quality taxonomy of alignment data. The second part to this, I think for a lot of companies, they will start off private. There’s an open source, taxonomy. I’m going to do my own thing. I don’t know, but I think over time, I’ve described this as like, if you think of this almost as the next SEO, let’s say you’re a startup, you’re building the next greatest thing in programming. You can keep it all private, or, if you contribute a bit, are the foundation models of the future your way to help create awareness and teach developers? And I think we’ll see that balancing. Those things are just sustainability, like when people contribute and when there’s equal value, you build really powerful communities out of it. So that’s our goal in the company and world side.

Steve: Sounds fair for sure.

Ruchir: So I think in addition to what Matt said, and he alluded to this as well, one of the best things I see about InstructLab is like when I see us humans, we learn very different ways. I’ll give you example. Let’s say somebody asked you Steve, Steve, go learn Python. That’s great. I’m going to give you 5 million files of Python on GitHub. Go learn it. You will really say, you are out of your mind. I don’t have time.

Steve: That’s right.

Ruchir: Nor the skills to go and crawl 5 million lines of Python. This is where state of the art today is. I call it the blender approach. Here is a blender. Put 5 million files in there, press the button, cross the fingers, something good happens. Versus how do we humans learn? We go to school, we take a course on algorithms and data structures. Nothing to do with Python just yet. We say how to think about solving a problem through a systematic way of approaching things. Then we take a course in python with its control flow, data flow, data types, and so on. InstructLab brings the process of teaching these large language models similar to the way we humans learn, through a curriculum of knowledge and skills. That to me is fundamental, actually, this is the first time we are bringing such a technology, working with Matt, and team to the community, broadly speaking. And the second one, I would say will be, and Matt referred to it in the very beginning, which is, when something doesn’t work, there will always be time. When something doesn’t work, this model doesn’t do x, you can throw in 100,000 more files in there, and again, cross your fingers, or you can say, I’m going to give you five examples.

That’s exactly how you teach your child. When something doesn’t work. You tell them, no, kiddo, that’s how you solve it. You give them five examples, let the back end process take care of everything else for you. And that’s exactly this point of incrementality. Something doesn’t work, fix it incrementally. And this point of teaching it with knowledge and curriculum are two of the most fundamental things we are bringing together from a technology perspective.

Steve: Okay, so last question for me. So we’ve talked about, all right, so open source and AI are impacting each other, and we now have open models and we have a open, I don’t know, call it a contribution mechanism in InstructLab. So when you put all that together, if I’m the average enterprise, why do I care? Like what’s in it for me? How does this sort of improve the way that I operate? How does it help my business move forward and just generally sort of change the way that we’re doing things today? So we’ll start with Ruchir.

Ruchir: So I would say from an enterprise point of view… so what are enterprises struggling with? AI is going to be everywhere. I think Arvind said it obviously in the keynote as well. And if you look at if it’s going to be everywhere, everybody will have to find in the current way, resources to get people. These people are pretty expensive right now, actually. Everybody doesn’t have a $3 trillion valuation. There are companies who are really contributing a lot to society, but bread and butter companies. And what InstructLab does for the enterprises is it brings the process of training these large language models with my enterprise knowledge to its subject matter expert. Every company has an SME. Every company has SME’s, but what they don’t have is AI skills. So it sort of refactors this process of training LLMs and brings it down to, you focus on your subject matter expertise, give us few examples on the skills you really want. Rest of it will be taken care for you and it will be, if you really want, it will be your private model at that point in time.

Matt: Yeah, I would say if you look at the grand challenge for it, I think there are a lot of people trying to figure out, am I in service to this tool or is this tool in service to me? When you can’t change it, when you can’t train it, it certainly feels like you’re in service to it. You’re going to ask it questions and it’s going to learn from them. And my subject matter expertise slowly evaporates. We’re trying to tackle the opposite side of that. If I can give you something that you can use as an amplifier for your work, your ability, this becomes a tool for you and you can do it on your own terms with your own data in a way that you control and then you have that option for contribution to like there are areas that now I know it and I can sort of make the world better. That’s the power of everyone knows they’ll have to use it to solve business problems. It’s the how we think this is a much better vein of how you both make associates feel good about it as well as solve like a core enterprise challenge.

Steve: Yeah. And I think it’ll be just on a closing note, I think it’ll be interesting to see obviously you two and sort of probably everybody here at this event knows that open source has been a dramatic accelerant to the process of software and with any luck, hopefully you’ll do some of the same for AI. So with that, Matt and Ruchir, thank you so much.

Matt: Thanks.

Ruchir: Thank you.