Kate Holterhoff, analyst with Redmonk, and Carl Sverre, creator of SQLSync and founder of orbitinghail, discuss database management for frontend engineers: an issue that has never been more pressing. Carl expands on the subject of client versus server side data caching, explaining how SQLSync uniquely addresses the data management problem on the frontend. He explains the importance of “offline-first,” meaning that applications are intended to function offline, and lays out his optimism about the future of WebAssembly (Wasm).

This was a RedMonk video, not sponsored by any entity.

Resources

• Learn more about SQLSync on sqlsync.dev
• Follow Carl on carlsverre.com
• Kate’s research on the Server/Client two-step

Rather listen to this conversation as a podcast?

 

Transcript

Kate Holterhoff: Hello, and welcome to this RedMonk Conversation. My name is Kate Holterhoff, Analyst with RedMonk, and with me today is Carl Sverre, creator of SQLSync and founder of Orbitinghail. Carl, thanks so much for joining me today.

Carl Sverre: Thanks, Kate. Yeah, I’m super excited to be here.

Kate: All right. So I invited Carl here after reading a post he wrote titled Stop Building Databases, in which he lays out what he calls “a front-end optimized database stack.” So needless to say, I was very intrigued. But before we start digging into what he means by that, and it’s super interesting. Carl, can you share a little bit about your career and technical background, just so folks can get a sense of who you are in your realm of expertise?

Carl: Absolutely. Let’s see. So about 12-ish years ago, I was working on a company called Mixpanel, which is a user analytics company. And at Mixpanel, I started my entry into the world of data. I started to learn about… I was a front-end programmer, so I was building JavaScript applications. But I started to learn about how you could use data to build really amazing client experiences. We were taking data, which was user analytics data, and we were rendering into funnels and segmentation and basically helping people learn about how they’re users that they’re using their app. But in that process, I got to really love how there was this interesting relationship between the server side and the client side. When you’re moving very large volume of data, when you’re moving any data, it just made me start to think about that side of thing. That was where I started this like, noodling in my head about this relationship 12-ish years ago. Then I had the opportunity to join a company called SingleStore, which is a database company. It used to be called Memsql when I joined. And at SingleStore, we were building an analytical and transactional database.

So a database optimized for both kinds of workloads. Sometimes people call this type of database an HTAP database or a Hybrid Transactional Analytics Processing database. But the premise is that you can build really amazing applications when the same system which is holding your data, can do both sides of the story. And so I learned a lot about databases at SingleStore. I did everything from building our front-end stack, our dashboard system, helping people deploy the system. And then when I left, I was doing pretty low-level stuff in the database. The team I was part of, we added WebAssembly to the database. So we basically added the ability to program the database via compiling from any language that can target WebAssembly, which was an interesting paradigm. And we just tried a lot a lot of really cool things inside of SingleStore. That was what got me to where I left SingleStore, which was earlier this year in February 2023. I decided that it was 10 years exactly to the day that I worked at SingleStore. I had a decade of database experience, and I’ve seen a lot of different things in the data world, and I decided to go back to my roots.

I want to go back to front-end programming, and I want to approach it from a little bit of a new angle, which is with all this knowledge about how databases work at a on a low level, can we maybe rethink how we manage data in the front-end and that relationship between the front-end and the server? And so that’s my quick TLDR journey.

Kate: No, it’s awesome. Okay. And I got to say, I’m always interested in speaking with folks that love databases and data management. So I’ve met several self-proclaimed data nerds, and I’m just always curious what makes this issue so compelling for them, just intellectually. So, Carl, have you always been a data nerd, or what is it about this particular domain that interests you and excites your, I guess, willingness to spend all of your work capital on?

Carl: Yeah, that’s a really great question. I don’t think I’ve always been a data nerd. If I go back before at Mixpanel, I think I was definitely… I approached computer science less about data and more about visualization. I was really excited about teaching the computer how to make things visually. That was a really big draw for me. Where data came into the picture was really at Mixpanel, where I learned that when you have interesting data, you can use that data to build data bits. You could essentially combine data with that passion that I had, which is visualization. It turns out after a really long time, after an entire career of data, that visualization in almost any form is a data activity. It really is that if I go back to the data of this stuff I was doing in university, even though I didn’t think of it as data in the way that we think of data and databases, it is very much like this interesting relationship between data and visualization. So maybe I was a data nerd, but I don’t think I identify as such. Sort of a fun round trip that I got to.

Kate: I love that because I guess when I tend to think of front-end engineers, you tend to think of them as being in interactive teams with a lot of designers and UX folks. But yeah, there’s this whole other side to to UI, and that’s like data visualizations. So when you think about business intelligence, I mean, that’s really their wheelhouse. It’s creating these charts and trying to visualize some of these important concepts. So you’re right. You’re making me think about this in a different way than I have historically.

So one of the things that I thought was really interesting from your piece was this concept, or I guess it’s more of a philosophy of offline-first. Could you explain what that means? Because I’m not sure that that’s something that everybody’s familiar with.

Carl: Absolutely. So offline-first is the idea that the application is designed to work offline. Okay, so that’s the first part of the thing, offline-first. So the offline part is the application continues to provide some capability offline. Now, some people might be like, well, you can’t have all capabilities offline. What about email? I can’t send an email if I don’t have a network connection. And sure enough, that’s fine. Offline-first doesn’t mean that all capabilities are offline. It just means that the user can open the app, they can see the last state that they saw prior to closing the app, they can interact with at least enough of the app that it’s useful and it provides some value. That’s the offline side. Now, first side says essentially that the way in which the app thinks about data and the relationship between the app and the back-end is that it interacts with data offline-first, so it interacts with the data on the device first, and then the data is then synchronized asynchronously to the back-end. So it changes the relationship between the app and data from the classic relationship, which might be, first we’re going to send an API request to the server and wait for the response.

That’s the very classic strategy. The second strategy might be parallel. So we send an API request to the server, and in parallel, we do some local optimistic data change. That would be where we try to do both things at the same time. Then offline-first is we do the operation locally first. We show the result of that operation right away. And the user just gets instant feedback. And then asynchronously, basically, not in parallel, but asynchronously, we’re also telling the server about it. And so the server still gets updated, but we have a the app experience moves forward locally in a way that is most beneficial to users. So that’s the aspect of offline-ness and first-ness.

Kate: So my own research at RedMonk, it covers what I’ve referred to as the Server/Client two-step, by which I just mean the challenge that’s faced by all front-end and full-stack engineers of determining what should be done on the client’s browser and what needs to be sent to the server. But then at the same time, I always joke as someone who’s a front-of-the-front-end engineer. So I tend to write a lot of HTML, CSS, a little JavaScript. I didn’t really deal with databases when I was a practitioner. And so I’m super self-aware when I talk about data. And I actually took a Microsoft certification in data fundamentals as a part of my research into certifications because I was like, well, where is it that I need to learn the most? And it’s certainly about databases. When I was a practitioner, the back-end handled it just full stop. And yet we’re seeing that this is more and more of an important issue for front-end folks to take on. So let’s lay out this problem space. How are you thinking about data in terms of the front-end? And how are you trying to solve it with SQLSync?

Carl: That’s a really good question. So I’ve been thinking about stepping back to maybe the two-step. I really like that way of saying it. What did you call it? Did you say the Client/Server?

Kate: Client/Server two-step.

Carl: Yeah, I love that. That is a great phrasing. It really offers the spirit of a lot of complexity in front-end engineering. In front-end engineering, as front-end engineers, we love creating the visual aspect, that interaction layer between the application and the user. That’s really what I think a lot of front-end engineers would say, at least for myself. I guess I can’t speak for everyone, but maybe for you and maybe for me, that we really like that. We like that relationship to the user. And a lot of our engineers love the fast iteration cycle as well. That local experience where you change the code and you’re immediately seeing the result. That’s pretty tangible. It’s very cool. So the two-step is so interesting to me because it’s always been the biggest source of friction between me trying to build a really cool experience experience for a user and the experience actually existing. It’s always this… I spend probably more time thinking about the two-step, that data interaction piece, than maybe sometimes the application itself. Ultimately, this is always frustrating me. I don’t want us to have this. I want our front end engineers to basically be able to use data much more naturally, much more as if it was just another part of that local application model, and leave it up to a different system to figure out how to move data in between the front-end and the back end, rather than the front-end engineer having to manage that coordination themselves.

So that’s always been how I thought about it. When I was at SingleStore, I was doing this WebAssembly. So for anyone who’s watching, maybe they aren’t familiar with WebAssembly. WebAssembly is basically a… You could think of it as… There’s a lot of ways to think about it. Let’s say that it is a general purpose way of representing logic or computation. We have code, we all know code. We can sometimes compile code into other representations of that code, and then that compiled code can run on different machines. We can think about this relationship between code and computers. We can think of WebAssembly as another computer that we can compile to. We can run basically different kinds of code on this WebAssembly computer. The best thing about WebAssembly, and maybe the name hints at it, is the ability for WebAssembly to run inside the browser as well as on the server side. We have this flexible computing abstraction where if we compile code to this virtual computer, this WebAssembly computer, we can run the exact same code on multiple places, in the web browser, right beside JavaScript and also in the server. Now, why do I bring this up?

Because when I was at SingleStore, I got really into this concept of moving compute to data rather than moving data to compute. When we think about databases, generally in a back-end system, we’ll have some centralized database. We’ll have Postgres, we’ll have MySQL, something, a very standard database, and we’ll have our app. Usually, we think of them as two different entities. When the app wants to do some processing on data, it will send a query over to the database. The database will process the query and return the results to the app, so you’ll get back rows or query results to the application. And then the application will do additional computation. It might run multiple queries, might do different things. Now, this relationship is interesting because we could also consider a different option, which is instead of having this app over here in this database over here, what if we just take the app and we put it inside of the database? So suddenly we’ve eliminated this round trip between the application and the database to be able to process that data. We’ve essentially moved to data. So why am I bringing this up? Because this is a similar two-step that we do with front-end engineering.

With front-end engineering, we have our front-end on the left side or my left side, maybe your right side. I don’t know. Okay, so we have front-end on one side, we have the back-end on the other. And the two-step is basically you’re doing some compute over here, you’re doing some compute over here, you have some data over here, you have some data over here, and you’re trying to figure out how to coordinate those two sides of the same coin. This is a hard problem, a really hard problem. And at SingleStore, I started to experiment, as I said, with putting the app the inside of the database. What if we compile the app to WebAssembly? We take advantage of the fact that we now have this abstract version of our app that we can run anywhere, and we copy that application directly into the database. So now, instead of having this two-step, we just have a one step. We’re just in the database. We run the app with the data. We’re doing super high frequency, low latency operations back and forth. Maybe it’s a simpler paradigm. Maybe you no longer have to worry about moving stuff around.

So when I left SingleStore, what did I do? I said, can we do that? Can we do that interesting relationship? Can we change the relationship between our app and the database? Can we do that with the front-end? Can we change the relationship between the front-end and the back-end? And so in SQLSync, which is a database that I’ve created, under the hood, SQLSync uses SQLite, which is a very foundational database. It’s a super mature database being around for a really long time, very robust. I take SQLite and I wrap SQLite with, you can think of as essentially the API layer of your application. And that API layer is compiled to WebAssembly, which means I can run it both in the client, beside the JavaScript, and I can also run that same API layer on the server. And so now what I can do is if I figure out how to move data from the server, like the underlying database from the server to the client, I can now run that API locally in the client, and I can eliminate things like like network latency, complex caching, complex optimizations, complex query execution. Imagine you’re building a table view in your front-end application, and probably most front-end engineers have done something like this.

They have some table view. What do you call it? Like, program? No. Project manager? No, product manager! There’s too many P’s. Okay, so the product manager says, Hey, front-end engineer, can you sort this column, please? If the user clicks on the column, sort it. This sucks. Usually, it means you’re going to send a new API request to the server, you’re going to resort the data, it’s going to come back, you have to refresh, reload the data in your database. With SQLSync as an example, because we can move that API layer, no matter how complex the API layer, we can move it from the back end to the front end. When you want to resort the data, you just issue a local API call. You say, Give me the data in a different sort order. SQLSync manages understanding what data is local, and it can immediately resort that information without you having to do anything different. The same exact code that knows those custom sort systems. Because sorting is not always the simplest thing. It doesn’t necessarily mean you just alphabetically sort. You might have to sort using a custom comparison function.

You might have some complex code. Maybe you have some more interesting query that you’re running locally. So all of this is to say, why did I build SQLSync? Because I believe that this two-step is very annoying, and I’d like to be able to solve this by moving that back-end system into the front-end and then intelligently dealing with knowing when we’re going to run that front-end code, when we’re going to run the back-end code, and eliminating the need for the front-end engineer to think about it.

Kate: Okay. So I think I have a bit more of a handle on why we should, “stop building databases” and instead build a front-end optimized database stack here. And SQLSync is the way to do this, right? So I’m interested in your argument that there is a mismatch in the data model for the client and the data model for the server. So the implication is that there’s some transformation which creates that disconnect. Is there a precedent for offline-first in industry that has been able to successfully bypass the issues that come from being, I guess the opposite of offline-first would be online first. So is there a way that this is already happening, that we’ve seen this work in the past?

Carl: I think that’s a very interesting question. So maybe there’s two questions there. So one question is, of course, who’s in the industry is maybe enabling some of these things. And then the other part of the question is offline-first as perhaps a forcing function, offline-first as perhaps a different way of representing a data model and what that means. So we’ll start with the first one, which is industry that have adopted different ways of building apps that that maybe have benefits over the classic application architecture. Some of these are really old-school apps that everyone will know. So I’ll say two of them, which is email and calendar clients. Basically, every email and calendar client on the planet, ehh, it’s probably not true. Many of the ones that you would use, like the Gmail client, many people have used that, Outlook, which is a desktop application that you can use. It’s now a web app, but it basically has the same concept. These systems, they’re actually, I would argue, offline-first. They’re designed to do most of their operations on an offline version of the data, and they continually are synchronizing that offline version of the data with the online version.

So they’re talking to the server to be able to say, Hey, do you have any new emails? Okay, great. You have some new emails. Let’s bring it down to our offline version. But when you open up Outlook, for example, in Microsoft Windows, you open up your Outlook email client, it doesn’t It doesn’t matter if you have internet or not. It’s going to show you the last view you saw before you closed Outlook, whether that is an email, whether that is an inbox, you’re going to see that. And then it’s going to asynchronously update itself. If there’s internet as fast as it can, but in a very asynchronous way. And we’re used to using these types of apps. These apps work really remarkably well, and they add a lot of reliability to the system. Similarly, you send an email, it goes in your outbox, right? It doesn’t right away go to the destination. It’s cued for send, and as soon as it has internet, it’s going to send. Some more modern apps that are approaching this problem space have learned, and this is where the offline-first data model becomes really interesting, have learned that they can actually build collaborative experiences by adopting a offline-first data model.

So in their email and calendar apps, it’s a single user system. Only generally one of us is interacting with the system. So there’s no need for real-time collaboration. But if you look at, for example, like Figma, if you’re familiar with the design tool named Figma, or Linear, which is a task management tool, both of these applications have adopted a offline-first data model where the data is designed to just like… The data is designed to synchronize from the server to the client, and then the client can actually take operations directly on that local data, that local version of the data. And then those operations can be synchronized asynchronously, which is an important piece of this architecture, with the server when there’s Internet. And you can think of this as how email works. To a certain extent, the way in which linear works can be thought of as how email works. On the client, we have some data that we’ve cached locally, which is our emails that we’ve cached locally. We have an outbox of things that we want to do, essentially in the linear model, maybe you’re trying to add a task to the system, you’re trying to do whatever.

And we have a… We’ve separated the application concept from the network concept. The network can happen asynchronously and the application can continue to work. So we’re starting to see this trend towards collaboration as a first-class citizen. And interestingly, if you look at a lot of ways in which people are building collaboration, they are building it in a way that is also offline-first. In one of the talks I gave earlier this year at KubeCon, New York City, I mentioned this concept that offline-first is a forcing function. And what I meant by that is that by building your data model, the relationship between your application and the data, by building it in a way that supports offline-first, you could vaguely think of it by building your data model similar to how we think about email and how email clients work. And that is a very abstraction on the topic. The topic is much more complex than this, but I just wanted to give you a good metaphor, which is emails. We can all think about how emails work. By building a data model that is similar to that, we can gain collaboration because collaboration, it’s important to be able to work as fast as users work.

Collaboration is frustrating and it’s slow whenever a single operation you execute has to be synchronously committed to the server and back, i.e. There’s an API request in the collaboration. When you’re editing Figma file, it would be really frustrating if you moved an object for that entire move operation to wait on asynchronous request and response to the server. Building a model where we can actually do that move operation entirely locally with no network connection at all allows us to have super low latency on our operations and thus build a more real-time experience. And then collaboration is enabled by essentially building the collaboration part, like the way of merging data. By building that directly to the data model, we can gain that collaboration. So this is a little bit of a wide topic that I’m trying to address here. But going back to the question, which is, is the industry adopting it? I think absolutely yes. And going back to the question of what about offline-first helps the system? It’s by thinking about offline-first, even if offline-first isn’t necessarily a product feature you’re trying to enable, by just simply making that part of your design space, you can more simply build a collaborative app that gives you that real-time experience.

Kate: Right. And I love that idea of the Figma file just moving one pixel at a time as it pings the server for every little movement. So yeah, I think that really helps me to envision how it’s already working. And yeah, I mean, this is the obvious solution, but it is this mental disconnect of In order to collaborate better, I need to go offline. It takes an adjustment in how we think about this, but you’re absolutely right. I mean, it’s almost like that you need a little time to, I guess, be quiet with the work that you’re doing, and then you can collaborate better because we actually have something that’s a little more finished before we dig into it as a team and actually try to get this project accomplished.

Carl: That’s another aspect of it as well. There is some other interesting aspects of offline-first, which I think you’re highlighting, which is that by building the data model, you also gain the ability to choose to go fully offline, to work offline as a feature. And this feature, like when you’re writing a doc, I’ve seen a couple of interesting startups. Off the top of my head, I can’t remember their names, but there’s been a couple of startups recently that have started to experiment with different ways of doing document collaboration, where rather than the Google Doc style, where everything is online all the time, where you always see the user’s changes all the time. Switching to a model where it’s more like, okay, I’m going to check out this document into my own version. I’m going to edit that version. No one can see those changes. And then when I’m ready, I’m going to merge back in. So maybe more of a Git style, like source control management style document workflow. And that’s also an offline-first workload. Even just Git as a version control system is an offline-first system. We have the database which is designed to run entirely on the client. We can run operations locally on the client, and we can synchronize asynchronously.

So offline-first is everywhere. It’s just being able to use it effectively is the hard part.

Kate: Yeah. I love that example because I got to admit, anytime I’m on a Google Doc writing with other people, I’m just sitting there reading what they’re writing. I could definitely be getting more done if I wasn’t spying on my coworkers. Okay. So it seems like SQLSync fits into this larger conversation that’s going on in the front-end development space that includes caching, hydration, and state. So a number of front-end frameworks are coming up with innovative solutions to these problems. So I’m thinking of like with the React server components and islands architecture, which is used by frameworks like ASTRO and Fresh. So I guess I’m just curious, why are these solutions not sufficient? And if they do have a place, where is SQLSync a better solution?

Carl: It’s a really good question. So when we’re talking about front-end applications, we can think about them as maybe from a couple of different perspectives. So the first perspective is front-end apps, especially like web apps, specifically, where React server components and islands are predominantly being used. Web apps have this interesting distribution model where you go to a URL and you effectively download the app. There’s a different distribution model to how we distribute native applications. Native applications, you go to maybe the App Store, you click Install. It’s installing the app. You have this clear concept of I’m getting a thing on my computer here, and then you then open the app and then you can use it. It has its own internal navigation system. The web is interesting. We’ve correlated the navigation system with the distribution system, and that is one of the key innovations of the web that made the web grow so quickly. Okay, so why do I bring this up? Because React server components and islands are directly related to this new paradigm of a distribution system. With this new distribution system, we need to be able to essentially load an application from zero.

There’s nothing on the user’s device to a working app as fast as possible. When a user goes to a URL, we don’t want them to have to wait for the entire application to install all at once. So a lot of these features that have shown up in the web platform are really tightly coupled to that web distribution architecture, this idea that we can go to any app at any point, whether or not the device has ever seen the app before. And because of that, being able to do some server-side rendering, like React server components, being able to free-initialize some of the state that they’re going to need to be able to render that application, that’s a form of caching. It’s a form of accelerating that time to first bite for that user. And these are great ideas. And these ideas don’t necessarily go away if you adopt a offline-first data model. You can actually have both. And so this is like I think a good way to think about it. It’s like, well, why do we need these things? Well, because we want to be able to do incremental rendering, we want to be able to accelerate that time to first bite.

We want that user experience really good. That’s one reason. Another reason is that some frameworks have started to adopt this idea that we can do partial server-side page re-rendering, where you click on a button and we’re going to essentially just use an island, for example, to just reset, to just update a portion of the page to a new content. Different frameworks do this in different ways. Some frameworks will actually send down HTML fragments that were rendered on the server that can be loaded by a JavaScript into that component inside the server, and that’s where you get, inside the client, that’s where you get things like islands, which is interesting. And then other frameworks will work really tightly with the React layer, for example, or other front-end framework layer to turn those… When I click a button, turn that into an API request that would maybe retrieve some JSON, and then that JSON will end up being processed by the client and then hydrating into a new client component state. Okay, so that’s all just preliminary, getting up to the point where I have an application that’s running in my client. Those tools that we have, like ReactServer components and islands, are important mechanisms to incrementally turn a blank page into a working application.

They’re very important. But at some point, if you’re building a rich client experience, you’re building an experience where the application is a rich experience. What is a rich client experience? A rich client experience is Figma and linear and Airtable and Google Sheets and Google Docs. These are experiences where you want low latency interactions with the data, essentially. If you had to do long, asynchronous server request responses for data operation, the application would be less nice to use, not as good. So this is how I define in my mind this rich client experience where you want to be able to use low latency interactions. And how do you do low latency interactions? In order to do low latency interactions, you predominantly have to adopt a concept called optimistic mutations. So you have to be able to essentially optimistically change the data that is locally cached to be able to bypass waiting for latency. So you have to basically say, I’m pretty sure I know what’s going to happen when the user clicks this button. And so what I want to do is optimistically just do that thing that I think is going to happen on the client before the server gets back to me.

And if we can do this really well, suddenly we have a rich client experience. We have a client experience that feels maybe more like a native application where the user is using data on their device. This is really, really hard. It’s hard to to build optimistic mutations. And I wrote a whole blog post about why it’s hard and all of the things that you have to now consider once you start to build optimistic mutations. And I’ll say one key thing that’s, I would say, pretty easy to rationalize why this is difficult. An optimistic mutation requires basically reproducing API logic that exists on the server in the client. If we want to be able to do something optimistically, we have to be able to predict the outcome of that optimistic thing. How do we do that? Well, we have to reproduce the logic that produces that output, which traditionally sits on the server. We have to reproduce that logic in the client. Software engineers don’t like to repeat ourselves. We like to keep things simple. But the idea of reproducing API logic on the back-end in the front-end defeats that. It makes our apps more complex, makes them more fragile.

If they become out of sync, suddenly you have this problem where, oh, my gosh, we show the user one optimistic result, but then the server just writes all over that with some different outcome, and that’s a bad experience for the user. So all of this is to say that React server components, Islands, these are great solutions to incrementally streaming that initial experience, to creating a relationship between the server and the client that is a little bit more sophisticated, allows us to do essentially less work to get more benefit. We only need to string down a subset of the application visual model to be able to update state. That’s all great stuff. But when we want to add rich client features to our app, when we want to have that ultra-low latency relationship with the user, we have to have a system that builds around the idea of optimistic mutation, the idea around vocal queries, where we can reshape data quickly without waiting for the API response. When we start doing that, we add complexity to the system, and ultimately, we end up with these very complex rich client applications.

Why did I build SQLSync?

Because I wanted to build a database that runs inside the browser, and I wanted to be able to build an API layer that I can run both in the back-end like a traditional API. And I can also copy that entire API down to the client to be able to run optimistic mutations directly inside of the client application, such that the developer no longer needs to think about copying and reproducing logic. SQLSync is able to run the exact same API running in the server. It’s able to run it locally. It’s able to manage the data relationship. It’s able to track query subscription, so it knows which portion of your app is based on what portion of the data. And it’s able to do that without the developer having to think about it. That’s the approach that I want to start to lean towards.

SQLSync is currently well-suited for I think of it as two kinds of applications. It hopefully in the future, it’ll be suited for a lot more apps. But right now, it’s really designed well for these two concepts. So the first concept is what I call user-oriented data models. So a really simple example, we’ve been using the metaphor of email. And email is like a user-oriented data model. It’s designed pretty much for one user. We want to have cross-device synchronization. We want all of the users’ devices to see the same state. So that’s a nice aspect of the user-oriented data model. And we’re predominantly working with the data of one user.

The nice thing about the data of one user is that the data of one user is relatively small, and we want to synchronize it to every single one user’s devices. We want to have that cohesive experience. Another application that’s really good within that same category, the user-oriented data model, would be a personal knowledge management app. So think about apps like Obsidian, which is a personal knowledge base. Personal wikis, for example, are also good. Personal task managers or even personal financial solutions, where it’s about one user’s data. So even a banking app, having SQLSync manage the local side of your data. Now, it’s not going to be able to run financial transactions for you, but it can at least show you your accounts, show you all your transaction history. You can provide a really nice local low latency experience over analyzing that data, which is really ideal because it’s a single user data. So any app that’s really optimized around single user data can really effectively use SQL. The second app is what I call document oriented data. Document oriented data, you can think of as like Google Docs, Google Sheets, Figma, Photoshop, Airtable.

When you’re using these apps, you’re basically working with a single document. In Google Docs, it’s literally like a text document. But in Figma, it’s a design document. It’s some image that you’re collaborating on with other people. In the case of Airtable, it’s a table, right? So these apps have a documented oriented model. When a user has opened a document, they only care about that document. So you can think of as like browser tabs. Each browser tab might only care about a single document. And we want to be able to optimize for low latency interactions with that one document. And so SQLSync works really well for document oriented models as well as per user data models. The other kinds of data models you might say, okay, what is that compared to? A really common data model in a lot of web apps is multi-user collaboration models, where the size of data is larger than what you can put on a device. So a really easy example of that is something like Jira. Jira, which is a huge enterprise task management solution, if you went to most enterprise companies, the amount of tasks inside of their Jira instance is probably more data than you’d want to put on every single device.

You don’t want to have that huge data set completely offline-first. And because you don’t want to do it, you have to build this really interesting system where only a very small subset is on each device. And that does work. People have built it. Linear is a good example of a task management tool which has this offline-first thing where they figure out what is the set of data that the user has cash, and we can do local operations on that data. The downside of this approach is that you’re not completely offline-first. You’re not able to look at all of the data. You’re only able to look at the data that your device happens to have cached the last time it communicated with the server. And so it’s a different data model, and it’s different data scale. But ultimately, SQLSync is really well optimized for document-oriented data models and user-oriented data models.

Kate: All right. I think that that summary is an excellent point for us to wrap up. So I especially want to thank you for coming on here today and even using my country line dance metaphor with the Client/Server two-step warms my heart. So before we go, would you tell folks how they can hear more from you, Carl? What are your preferred social channels? Are you planning on speaking at any event in 2024? How can we keep up on SQLSync?

Carl: Absolutely. So SQLSync, you can learn more about SQLSync on sqlsync.dev which is the main website for SQLSync. There you’re going to be able to see the blog post I wrote, which talks about a lot of these concepts. And then I’ll be able to release more content there and hopefully start to build a community around this idea that we can make simpler apps by building around offline-first and building around a front-end optimized database stack. So that’s sqlsync.dev.

You can also, for me personally, you can follow me on carlsverre.com, so my just full name. Com. There I have all my social channels. I have my own personal blog, which is just starting. I’m hoping to do more, but yeah, that’s the big thing.

And then as another point is I am planning on attending conferences in 2024. And if you want to start meeting in person, some conferences that I recommend checking out is Handmade Seattle, which I’m not sure when it’s going to be yet, but that’s a really cool conference. Software You Can Love, which might be in Italy, which would be really interesting. KubeCon and WASMCon.

Kate: I want to thank Carl for chatting with me about his open source SQLSync project and weighing in on the front-end data management issue. So my name is Kate Holterhoff, analyst with RedMonk. If you enjoyed this conversation, please like, subscribe, and review the MonkCast on your podcast platform of choice. If you are watching us on YouTube, please like, subscribe, and engage with us in the comments. With that, thank you.