As the industry increasingly talks about AI “agents” and “agentic” apps, that leaves some important questions to answer: what are they? What do those terms mean? In this RedMonk conversation, analyst Steve O’Grady is joined by Couchbase’s Jeff Morris to discuss agentic apps, examples of them in practice, and the role of the database in building them.

This RedMonk video was sponsored by Couchbase.

Rather listen to this conversation as a podcast?

 

Transcript

Steve
Good morning, good afternoon, good evening. I am Steve O’Grady with RedMonk, and we are here to talk today about, well, a topic near and dear to everybody’s heart, it would seem at this point. Agentic applications. So I’m here with Jeff Morris from Couchbase. Jeff, would you care to introduce yourself?

Jeff
Thanks, Steve. I’m Jeff Morris. I run product marketing at Couchbase. I’ve been here at Couchbase a little over five years. I have a pretty long-standing background in databases and analytics. But yeah, the application world as we start thinking about Agentic apps is fun.

Steve
Indeed. And I think that gets us to the first and most important question, and the one that, frankly, if you ask 10 people, you might get 10 different answers for. But from your perspective, obviously, you’re having a lot of conversations with customers and so on. When you think about this, how do you define an Agentic app? How do you either help a customer narrow that down? How do you present that idea to them? How would you describe this to the folks listening?

Jeff
I think it’s… The way I look at it is, let’s start with the idea of a chatbot, but then taking everything that you might do with a Gen AI chatbot that you’re conversing with, let’s say for customer service or something, everything you do with that, and then contain it in a program and let it go ahead by itself. So instead of a human intervening with commentary or prompt adjustments or reactions to answers that a Gen AI model might give you is that’s all taken care of programmatically. Imagine writing these independent autonomous programs that go and talk to one or more language models and figure stuff out and then go do something about it. One of the most earliest examples we gave when the hot thing was vector search, which was only a couple a couple of months ago, but that seems to have gone by. When the hot thing was vector search, we had to come up with a way to teach people what vector search meant and how would you use it. So the example that we did was we trained an iPad on some visuals, some pictures, and then use the camera to identify and recognize the pictures.

And by doing that, it’s simply, here, feed it some information, have it recognize or find that information within itself, and then do something about it. So agents are the same thing, but they’re programmatic and run in the wild. As they go, you’re going to have a lot of data. People are still concerned about all the other issues around using Gen AI, and they could, if we get to the point where we’re really building them, they could be really clever.

Steve
So to that end, in terms of that example, if I’m an enterprise or anybody who wants to build an agentic pap, and that gives me a rough idea of what we’re talking about, what in your mind are the requisite basic building block technologies? What do people need to have?

Jeff
Well, clearly, you need a model that you want to talk to, and then you need your framework for programming, whether that’s Python or anything else to talk to it. And then you start into the exercise of perhaps just conversing with the model, but there’s probably data that you got to go gather that you want to put into the model and give it context in your prompt. So that means you’re going to need a data source that helps you with that information. And then you run into the other problem, which is the first problem is you don’t like the answer that the model is giving you. It’s too vague. And too vague in the best case and completely wrong and spurious and false in the hallucinatory situation. So you don’t like what it’s saying. So then you got to go and figure out, well, how do I make it more accurate? And that’s when you run into the problem of needing to learn a retrieval augmented generation. And doing that, you’re involving yourself in another model to take the data that you want to feed the original, but vectorize it so that you can point your ultimate conversation model to where to look within itself to figure out and maybe get more accurate information.

So as a developer, I just created, I just painted a whole brand new workflow for you to have to deal with. And you’re programming stuff. It’s no longer like, I got a database, I’m making application functionality, let it go. It’s not that anymore at all.

Steve
Yeah. Amen. And so we should ask, obviously, given that Couchbase is a database company, so how do you view the role of the database in all of this? So data clearly is the heart of what folks are trying to do with AI. But when you think about it from the perspective of the database itself and the people trying to build on top of it. What’s the role? How does it fit into an Agentic app?

Jeff
Well, this is a great one because there’s lots of roles. And depending on the database you’re talking to, they’re going to carve out their niche. If I went and talked to Snowflake or Databricks, they’d say, I’m the training data. I’m the company data that keeps the model informed. And then you build agents from that and do reg from that. I would tend to say we’re a little bit of that, but we’re also the operational data that you collect and need when the agent is running. And we’re also the store for all the vectors that you made when you’re trying to guide the model into looking correctly inside of itself in the first place. So there’s a bunch of different types of data that you end up wanting to store and persist and use when you’re actually building or running a real agent. So you got the originating company data, great. You need to vectorize that for privacy sake, among other things, and accuracy, as I mentioned. So you turn it into vectors. Vectors is a new weird data type that we’re all scrambling to be able to store and retrieve at super scale. And then if the agent runs, you want to remember what it said, and you want to remember what you told it.

So you need a whole transcript of all of the conversations that the agent has with the model. And the upside for me is that’s great JSON data. It’s great to put in a JSON database because it’s extremely flexible. You can modify it on the fly, you can version it, and you can ultimately store it in a catalog. So that you are logging what the agent is saying, and you can try and verify it because you can imagine as the LLMs get smarter, as your corporate data changes, as the external data that you’re also working in, I don’t know, weather, that those things are going to influence what the agent says and does. And we want to make sure that it doesn’t drift off of its focus.

Steve
Indeed.

Jeff
And so you got to keep the old data and probably run a side car model that is your validation model that says, yes, this is still accurate. And I like what it’s saying. It’s not falling apart and then runs in a while. Now, the other fun part is imagine agents talking to other agents, talking to other models, or an agent having a number of tools and talking to a number of different models all the same time.

There’s the degrees of complexity of this exercise of keeping track of the transcript, keeping track of the vectors, keeping track of the original data, updating all the original data for real-time stuff because company data changes every moment, and doing that at scale. There’s so many different data types I just called out, so many different data access patterns I just called out, so many different API interactions with either models or programs or whatnot. Right now, I don’t think of anyone but myself as being a vehicle that can store all of that at the same time. And then I left out all the analytic data that you might use for coming up with real-time calculations. Oh and I left out, maybe you want to run these things on a mobile device or put some portion of the agent on a mobile device. Well, I’m the only database, no SQL system that does vector search on mobile. So all of those things, all these weird data types and all these locations where you’re running the data, that’s what makes having a consolidator store that still operates in a distributed manner. So you have elements operating all over the planet is a pretty good plan for how to ultimately run these things. And then the challenge becomes like, what am I going to run?

Steve
Well, and that brings us to our last and final question. And to your point, I think it’s one of the biggest ones in terms of a lot of the conversations you have with enterprises today are, they’re excited about the capabilities, they’re excited about what they see as the promise from a technical perspective. And honestly, this goes back literally decades to Watson and earlier from an AI standpoint. And yet when we see this in practice today, you go in and talk to your average enterprise, the biggest question that they have, in many cases, isn’t the technology itself. That’s hard, and there’s a bunch of hoops they need to jump through. But the question is, what do they build? And from the conversations you and I have had, you’ve got some interesting thinking about this, but also some interesting examples. So how would you suggest enterprises think about it?

Jeff
So the suggestion really is, and our goal certainly, is to try and move beyond the chatbotty stuff that you’re already accustomed to. But I also suspect that the first sets of Agentic applications you’re going to make will look and feel like a chatbot because that’s the natural pace of the way we as human beings can absorb information and talk to robots and stuff. But think of it as something that is doing more work than just a serial conversation with a knowledge base, customer support knowledge base. That’s the common application that we see right now. The good ones that we see, we’ve got a customer building a device that is like a smart television for an operating room. Give soft acuity the plug. And what they do, they’re trying to figure out how to operate and guide a surgeon through their activities, tell them what instruments to be using, how to lay them out, work with the anesthesiologist at the same time, and if they need to phone a friend, allow them to do so. Now, the funny part of all of this is this is operating in a sterile environment, so you can’t touch anything to authenticate yourself.

It’s all got to be face, facial recognition and voice. And of course, the guy is wearing a mask, so it’s got to be pretty good at both of these things. And then they can call somebody at UCSF and figure out what they’re supposed to be doing and have a full conversation about the procedure. That’s a pretty cool use case, and that’s about as far as we’ve seen, but I think there’s some commonalities to what we are seeing is these are starting to help automate human workflows. And surgery is a complicated human workflow, but you could back this up and think of other workflows either in your everyday life or what regular people are doing. From the customer’s perspective, a lot of these things end up coming down to helping match an account profile with some catalog of goods or stuff that the vendor happens to be providing, like airline tickets or rental cars or movies that you’ve watched. Those are all examples of couch-based oriented applications that if you turn them into agentic things, your travel planner would do a heck of a lot more than what you’re able to do now.

Instead of first establishing your airline tickets, then moving on to your rental car, then moving to your lodging, you could get all of that done all at the same time through an Agentic application that knows where to look for all of that information. And then because it’s got the context of where you’re going, what the weather is going to be, it can help guide you as to what to pack. And then because it still knows where you are and what your interests are, I’m a concert fanatic. So I’ll go see live music anywhere where it’s playing, especially when I travel. So go and get tickets me for a local event that you already know I’m going to like and get a babysitter for us. So all of those activities are these regular sequential kinds of human workflows that an agent could compact and do into one thing. So take that back in other industries. Back in the medical profession, your doctor can feed their private model blood test results and tell you the narrative as to what they ought to be talking about after they’ve evaluated the results of that. There’s opportunities for these all over the place.

The problem that we see is still happening, and I think you guys do, too, is that what to build an idea, we still have to plant the seeds for people. Because there’s so much opportunity that the imagination runs wild with possibilities and doesn’t land on something concrete. That’s it. But we will end up on something concrete pretty soon. It could be days. And then everybody’s going to copy that for a minute. And then we’ll see the explosion of like, that’s a great idea. And agents will take off.

Steve
Yeah. I think it’s to me, it’s like if you go back and think about the first databases, it took a while for people to understand. Like, okay, I’m good. This is how I use it. This is how I think of it. This is how I store things. And so I think we’ll have to go in through a similar learning process here, but I’m sure we’ll get there.

Jeff
Oh yeah, we lived through this with analytics ages ago when flash was a thing and we were making dashboards that people didn’t know what a dashboard was. So we had to show them and show them that if you change the data over here, the gauge moves, goes red and green.

Steve
Flash, wow. Takes me back.

Jeff
So we’re in that same state right now. And gosh, I want to help people get out of that state.

Steve
That certainly is a grand ambition. And with that, we’ll wrap up. Jeff, thank you so much for your time today. I appreciate it.

Jeff
Steve, thanks.