In this conversation, John O’Hara, inventor of the Advanced Message Queuing Protocol (AMQP), discusses this standard’s history and development with Kate Holterhoff, senior analyst at RedMonk. O’Hara shares insights about the complexities of middleware, the importance of standards, and the evolution of AMQP 0-9-1 and 1-0. He reflects on the AMQP working group, the role of hyperscalers in shaping standards, the limitations of HTTP, and the importance of standards bodies like ISO, OASIS, and IEC. O’Hara emphasizes the need for middleware to become as ubiquitous as the web and shares insights on the future of AMQP and the standards process.

Links

• LinkedIn: John O’Hara

Transcript

Kate Holterhoff (00:13)
Hello and welcome to this RedMonk Conversation. My name is Kate Holterhoff, Senior Analyst at RedMonk, and with me today is John O’Hara, the inventor of AMQP, which is an acronym for the Advanced Message Queuing Protocol, of course. And John has an extensive experience in FinTech and is alumni of JP Morgan, Bank of America, and NatWest. He is also the co-founder and CEO of Taskize. John, thanks so much for joining me on the MonkCast.

John O’Hara (00:40)
Thank you very much for inviting me on. I’ve seen your work in the area and it’s very impressive so I’m very pleased that you’ve brought me in here.

Kate Holterhoff (00:47)
that means a lot to me. All right, so I’m so pleased to welcome John to the MonkCast to continue my series on messaging and message queues. I’d like to start by setting the table in terms of why this topic matters. So at a high level, first, how do you define messaging? And second, what do you tell folks that ask you why it is important for distributed systems in 2025?

John O’Hara (01:06)
How do you define messaging? I mean it’s an enormous subject area and I think your work has uncovered that. At the same time it can seem tremendously simple. You see people say, I built a messaging system in an afternoon and that can be done. I think quite simple. To me it’s something that’s as important as an operating system or a database or a file system. And we kind of have, know, Linux is kind of the operating system and the file system wars have kind of settled down. It’s ext4.

Open ZFS, stuff like that. The database, people are starting to sort of, there’s a meme, just use Postgres, and that’s not wrong. And that gives you, and we’ve got fantastic technologies to build applications with, but we really haven’t kind of settled the business of moving data between machines. And that’s for all sorts of reasons. Just in how you make an application, building an application using middleware as opposed to micro…

HTTP calls is a really good way and the banks have been doing it for years. That’s a nice way of building applications. But more on the necessary side, communicating between different corporations or different political bodies. If you want to have your organization talk to another organization, something has to shuttle that data across the middle. And if we’re always sitting down and the first argument is, how will we do this? That’s kind of awkward. We don’t yet have a kind of postgres of messaging.

And I guess the other part of it for me is I spent ages coming across messaging over and over and over again during my career. And I did AMQP as a last resort. thought someone has got to do this. It’s not going to be me. Someone’s got to have done it already or will do it. It’s not going to be me. then darn, no one’s doing it. OK. So it’s fiendishly complicated.

people don’t realize how complicated it is. So I’d love us to be focusing our energies as an industry, like we did on Linux, like we did on Postgres, like we did on file systems, into making state of the art, load bearing, awesome technology to do the business of messaging. And that’s why it matters, and that’s why it matters to me.

Kate Holterhoff (03:08)
Great, so what I’m hearing is it’s an absolutely essential part of computing of distributed systems and that it’s an unsettled question that we’re still grappling with how to do this as well as possible.

John O’Hara (03:18)
Yeah, we’ve got lots of shallow answers.

Kate Holterhoff (03:19)
Well, we’ll try to dig in a little bit and see if we can flesh them out a bit. So next thing I’d love to talk about is your background here. I’m always fascinated about how folks get interested in messaging to begin with. So how did you become interested in technology and computer science at a young age?

John O’Hara (03:35)
I’m unfortunately getting quite old. I remember the very day I became properly interested in computing. I was 11 years old and I’m from Ireland originally and it was a rainy day and I’d just come out of rugby practice and it was properly miserable and I lived in the middle of the countryside. was a long time to the next bus service was running so I walked up into the town. The rain fell harder. I stepped into a news agent and on the shelf there was among all the different magazines they had there was just one.

Computing Today. Computing Today had a very unattractive cover. It had a picture of a green screen with VisiCalc. They just invented VisiCalc, the world’s first spreadsheet. And I picked that up and that got me hooked. I’d always been interested in science and technology, but that got me really hooked on computers. And within a year, I had the fourth VIC-20 sold in Northern Ireland, a great expense. And it just went on from there. And I kind of really got a passion for it.

Ended up doing electronic engineering and computer science at Aston University. I got sponsored to do that, which is a rare thing. You don’t hear of companies paying people to go to university these days, which is just… But I had that. had Ford Motor Company sponsor me through university. And that had the side effect of getting me into Ford Motor Company here in where I live today, Essex. The headquarters, the design center. And you’re in there looking at the CAD CAM systems, all the different supply systems they have.

so much technology, so many different types and they were a great company. They rotated their graduates around different things so the amount of experience was just fantastic and that kind of got me really going. How did I end up crossing over into banking? In 1993, the motor industry is cyclical. early 1990s, there’s a fall crash in the automotive sector and

The company said for the next two weeks anybody who wants to can take Voluntary Redundancy. And I took Voluntary Redundancy off a graduate training program, which is bonkers. I had nothing to go to. I started looking for things in London. A friend had told me there’s some sort of event going on at the Waldorf Hotel. I finished interviewing somewhere, went along with the Waldorf, walked in. Couldn’t believe what I’d seen. I had three interviews as I walked through the door. And this company I’d never heard of, some little outfit called JP Morgan.

I ended up with a job. I walked out with a job. And that was, I would never have thought of working for banks, but it was a fantastic time to do that.

Kate Holterhoff (05:54)
wow, that’s not the kind of story that I hear from folks who are looking for positions today.

John O’Hara (05:59)
Yeah, it’s a very unusual story. and they were were JP Morgan that’s before Chase before all the other mergers they’ve done for Bear Stearns. They were just when the Glass Steagall was about to be repealed and they were rebuilding an investment bank from scratch and they had sucked in this massive number of people to do that. And it was incredibly exciting and incredibly well funded exercise. And that was when I really started. I’d seen different computer systems at Ford’s experienced everything.

Kate Holterhoff (06:01)
Yeah.

John O’Hara (06:27)
lots of arcane things from the really old PDP-11s and stuff like that. Sitting in front of a clay model of a car, scanning into a PDP-11, you know. But moving across, so seeing all this variety and moving into the bank made me see the need for these things to communicate with each other. So it’s a sort of story that builds over time. We see heterogeneous things, a need to communicate between complex systems and do that well in many different ways. Do it quickly, do it reliably.

whatever, which flavor is it gonna be?

Kate Holterhoff (06:59)
Right, and so you moved to the financial sector after completing your bachelor’s science at Aston University, and you also majored in electronic engineering in addition to computer science. So did that factor into your time in the automotive industry?

John O’Hara (07:07)
Yeah.

Yeah.

No, I think I always like people who done electronic engineering. They come from a sort of different way of thinking. And I did that because computing was too easy. I wanted to find out. I wanted to make it hard, learn something properly, a proper hard skill, physics. And also, we really want to understand how to build a computer from the chips all the way up. And I’ve been the guy at school, the kid at school who’d been given access to the school computers two years earlier. I was helping people in there with their O levels and A levels when I wasn’t even doing them.

So it’s like I was just so, I found it so easy. I thought, well, I’m going to do degree. I need to make the degree hard. And electronic engineering is really hard. The computer science bit was the easy bit. But I think the fundamentals there. It turns out that to make software fast or cheaper, you just need to move fewer electrons. Because moving the electrons is what uses the energy. So if you move fewer electrons,

it happens faster and it happens more efficiently. And when you understand sort of all those different levels and how they play together, which is something I think folk don’t see very often. They don’t get the opportunity to really span the layers that people do with things like Raspberry Pi. And I met the guy who founded Raspberry Pi. I think that’s a wonderful thing that he did there to help people really understand how it all works at the ground level up. I think it really changed your perspective.

Kate Holterhoff (08:36)
Yeah, I didn’t realize that you were such a hardware geek. think that’s exciting. that is, unfortunately, kind of a dying art here as more developers even move into using SaaS tools and things that are hosted in the cloud, and they never actually have to touch the server.

John O’Hara (08:40)
Yeah.

Yeah,

we’re having this conversation now. And that’s true. The industry has turned the whole thing into assembling abstract systems in the cloud. Cloud vendor gives you stuff, you plug it together. You can almost say, well, in the advent of AI, does any of this really matter? We’ll just ask a computer to build something. So I think the jury’s out on that one. Still very much so, but it’s a pregnant question. And one of the things that I’ll go into, I do venture partner work as well.

I’ll go into startups and I’ll consult with them. And it’s quite easy usually to take 20 % out of someone’s cloud costs. That could be hundreds of thousands of pounds or dollars out of their cloud costs and make a meaningful difference, like five or 10 % difference to the bottom line of a startup just by knowing how things work. I think I did a presentation at QCon last year, which is all about

be aware of what the hardware can do for you. The hardware these days is frankly amazing and most engineers have no idea how mind-blowingly powerful it is and they just keep plugging stuff together and throwing that performance away.

Kate Holterhoff (09:54)
Yeah, that’s significant. Yeah, and you mentioned AI here. When I attended the Supercomputing Conference, which was in Atlanta, where I’m based, so it was very convenient, I learned that a lot of the chips are really not being used at their full capacity. And so there’s all these ways to try to make them more efficient. But it sounds like it really is a hardware problem in a lot of ways, trying to make them work better and move those electrons, like you say.

John O’Hara (10:17)
Yeah, yeah, but we can come into that sort of later because it’s like if you want something to go fast and I’m a director of Adaptive Financial and they have Aeron now with Martin Thompson and his work and he’s always been into the whole idea of making things play well with the hardware and it’s almost like thinking about technology like you’re a Formula One team and the secret to it is to not move data around, ironically.

The closer you can keep things… You know when you change tasks as a human being, you context switch and so you get interrupted and you get out of the zone, out of the flow state. Hardware is the same. And every time you distract hardware, it slows down. And that’s where the main issue is. It’s where we’re distracting hardware and every time you make a network call or anything like that, you’ve distracted the hardware.

Kate Holterhoff (11:05)
Huh, I like that metaphor. All right, so let’s go back to your history a little bit. So you’re working at JPMorgan. Yeah, the job fell into your lap, it sounds like. It was a really, I’m sure you did, I’m sure you did. But you, know, once you got in, you stuck around 16 years at JPMorgan. So obviously something about this domain really interested you.

John O’Hara (11:15)
Well, hopefully the interview’s okay. There were just good… coming everybody.

Kate Holterhoff (11:31)
So talk to me about what it was about the financial industry that you found so compelling.

John O’Hara (11:36)
I think it was JP Morgan in particular in that time in particular. The people I was working with were absolutely phenomenal. And when you’re working with phenomenal people solving phenomenal problems that are intellectually challenging, you’ve got very sizable budget to do it. And that lasted for a long, time. I think that the time I left was after one of the mergers, was the Bear Stearns merger.

And the firm kind of changed its shape. I ended up working for Bear Stearns guy and I wasn’t so happy anymore. And and Merrill Lynch just called me one day and said, hello. And I went, yeah, sure. But the run up until that point was absolutely fantastic. The things like FpML, was derivatives processing. Back when financial innovation was how firms made money, banks made money. And financial innovation is a bad word these days.

because if you’re innovating in finance, you probably create a complexity that your customer doesn’t understand, and that’s not good. But back then, we could manufacture any cash flow structure you wanted for any client. And we actually did some standardization work there, the financial product market language. And the work through managing and modeling derivatives, which were really complex, and then building the trading systems for thousands of traders, in fact, this is part of where AMQP came from, was we had a system that’s supporting 5,000 equities traders.

back before you had high frequency auto trading, when people were sitting at screens deciding whether or not to buy or sell programs, trades, and they all had to have real time information distributed to them as fast as possible from one point in the world to another point in the world. And the message around that was very, very challenging because the hardware was worse. Fast Ethernet back then was 10, was Fast Ethernet 100 megabits? Or 10 megabits? I mean, it was really slow.

So the software had to be incredibly well written to make up for that. the challenges of working in a global context with sophisticated stuff, with clever people trying to beat the competition, it was just a fantastic place to be. And I remember there were two things that sort of stood out me. One of the things to say about JPM in its early days was more PhDs than NASA, more software engineers than Microsoft.

Kate Holterhoff (13:48)
You

John O’Hara (13:49)
And I remember standing

up at a recruiting event in UCL in London. And I went on to stage after Google. And I said, you don’t want work for them. Come work for us. We’ve got all this other stuff. Of course, that’s changed very much since then. Google has absolutely wiped the floor with everybody. But it was funny back then to be able to do that.

Kate Holterhoff (14:09)
I had no idea. That’s a really good quote, a good way to get folks interested in your company, my goodness.

John O’Hara (14:15)
Yeah, so it’s a fantastic place to work. And I think everybody who passed through there, if you find people who through there in those two decades, they all have fond memories of it.

Kate Holterhoff (14:25)
Wow, okay.

John O’Hara (14:26)
and you’ll

find them all over the place. They’re running all the different banks in different places now.

Kate Holterhoff (14:29)
Well, yeah, if there were that many, what, software engineers and, and what did you say?

John O’Hara (14:33)
PhDs than NASA.

Kate Holterhoff (14:34)
PhDs, that was it.

All right. So talk to me about what was happening in the days before AMQP. You’ve alluded to this, but what made creating AMQP so important at that time?

John O’Hara (14:44)
Yeah.

I kept coming across the same thing. So I wrote my first middleware in like 1998, then I wrote another one in 1999, then wrote another one in the year 2001. I just said, why do we have to keep writing these things? And you go to the market to try and buy them. And there were a handful of stuff that were good. And the rest of it didn’t really work. The stuff that was good was insanely expensive. We were talking like 25,000 dollars per CPU core.

20 years ago. And this is something really you wanted everywhere. So it was entirely non-trivial to think about just introducing this stuff. You couldn’t create an architectural dependency on it. And because there were no standards, you didn’t know if the vendor was going to withdraw support or change it or turn around. You had no second source. Nothing was compatible with anything else. So the temptation to write your own was really, really high. And indeed, every bank wrote its own middleware.

Kate Holterhoff (15:17)
Wow.

John O’Hara (15:44)
that I think came across Paul Weiss and Nirvana. Nirvana middleware was out of Deutsche Bank. even JP Morgan, there was an internal system called the Morgan Messaging Library, which is really, really good. And one of the members of staff had asked to buy the source code and commercialize it. And so the firm thought, yeah, sure, even a firm like JPM. And this is what you know us, middleware is really complicated.

And it’s very hard to keep smart people interested in looking after it. So the first three years is fine. Then the year four, five, six, no one wants to take care of it. It’s super, super complex. So they will take it, go out, see if you can make a fist of that out commercially and off you go. And this chap took the software, had to go commercially. It kind of half worked. And then they turned around and thought, I know what, we’ll just extort the…

the mothership for, I know they’re using it, I’ll just ask them for lots and lots of money and they built a license manager into it by this point. And they basically pointed the gun at the firm’s head and said, pay us this amount of or we’ll turn it off. The 5,000 traders, we’ll turn them off. And that particular episode, that kind of shows why standards exist, right? But that particular episode created the business case for AMQP and created the funding for JP Morgan’s involvement in AMQP.

Because even if you look at Intel AMD, AMD exists because the US government didn’t want to have just a single supplier. They wanted to be able to go to another supplier. Because the dangers of having a single supplier is just too high. When you’ve got a certain amount of it, the temptation for the supplier to come back and gouge you goes off the charts. And we’re actually seeing that. You’ll know what firm I’m referring to. There’s a certain firm makes virtualization software and the economics of that virtualization software change recently.

So, you know, it’s a big deal. You want standardization. So that’s kind of where I kept seeing these middlewares everywhere. And there’s so many of them, they’re incompatible, and they were flaky and they were too expensive. And that’s the point where I thought, right, OK. And the CIO of JP Morgan at the time blessed the project and said, yep, off you go. See what’s the worst that can happen, what’s the best that can happen. Let’s do this.

Kate Holterhoff (17:54)
All right, and so you sat down to write this protocol. How long did it take you?

John O’Hara (17:59)
So that’s a really interesting one. So wind your mind back to the early 2000s. Patent litigation, software patents were a big thing. And you were sued for making. If you made something, you were sued for making it. So we couldn’t make it. The danger of making middleware was too high

for the firm to take. So what we did is we specified it really precisely. And we found a partner in Brussels to come work with us after we specified. We brought all this knowledge that we had inside the firm together. And we collected the brains trust that we put it into a big specification. And then we went looking for someone who’d be good enough and qualified enough to build that thing. And also could bring something to the table as well. They could be good communicators and help give it some legs.

And we knew if we were serious about making it an international standard, we needed to have more than one implementation. You have to have more than one. So, well, we’ll bootstrap this by making the first one. We’ll pay for the first one, and we’ll make the hardest one, the one written in the C language, because writing things in the C program language is insanely hard. So we thought, we’ll take that on the chin. We’ll pay someone to write it in C. And the guys at iMatix came in, Pieter Hintjens and Co. And that’s where the journey started.

And this was being funded by this other need to get off the legacy middleware that we already had. So we had this running thing that we knew really well that we corporately invented that we had to replace. And there was a team of 25 people. So five of them were working in AMQP. There was the external guys in Brussels and there’s a team of 20 in India and London working on re-platforming 5,000 traders onto a new system we were writing as we were going along.

So that was implementation number one. And then we made a decision. There were a few decisions we had early on. One was to make the protocol asymmetric. It was meant to be easier to make a client for it than to make a server. That way it’ll be easy. The brokers are hard to build. So make it easy to make clients. Therefore, it’ll spread by itself. And the second one was to document it really, really well, which we did.

And also we made the documentation published in XML, is just very, very painful. But people realize if you open the XML document up inside the XML, we’re instructions that would tell a code generator how to write half of the client code for you. So if you had your head screwed on, you could accelerate, make it really, really easy to write a client to go against an AMQP broker. But we had Matt Arrott involved at that point as well. Matt had come in and said,

He was working with TWIST, which is an industry standards body to streamline invoice processing of all things. And Matt had worked for Autodesk and he had worked for all sorts of things before it, but he was associated with TWIST and came in and said, we’ve seen your protocol, we want to standardize on it. the interest started to bubble as to what was going on here. And I should talk about the AMQP Working Group.

But then the best thing ever was when Alexis Richardson came in one day to JP Morgan London and said, we’ve built your protocol. I just was, he hadn’t, they’d given us no indication they were doing it. They just appeared one day and said, we have a message broker. We’ve done it. And it made me immensely happy because that was implementation number two. That means you’re on your way to an actual international standard.

And the working group itself is funny because I talked about patents. We didn’t make it because of the danger of patents. And it’s funny, the Derek Collison interview you did was very interesting. And Derek said something, if TIBCO hadn’t been so expensive, AMQP might never happened. He’s right. If the economics hadn’t been eye watering, know, we wouldn’t have built it. We were looking for any. In fact, before we built AMQP, we went for another vendor and tried to use it.

Kate Holterhoff (21:32)
You

John O’Hara (21:42)
And it failed, under stress under production thought, shit, we’ve just wasted a million dollars on another vendor. Throw that away, let’s write our own. We’re done, let’s put it out there as open source. But the other part of the standard, we needed to build a coalition. We knew that even a big firm on its own couldn’t do middleware. JP Morgan couldn’t do middleware on its own. Deutsche can’t do middleware on its own. We can’t do it on our own. So we had to get everyone together.

and we created this structure called the AMQP Working Group and I worked with the Chief Legal Counsel, the IP Counsel at JP Morgan. We crafted this contract

The contract essentially said that if you participate in this, any technology you introduce, you license the patents you have in that area to everybody else

that was in the AMQP working group license. It a big deal to sign it. It meant something. And we had Cisco and Red Hat and JP Morgan and who else? Credit Suisse, Bank of America. We had TWIST. had IONA Technologies. They used to, you know, back when they were around, they all signed up to this thing. And we gradually built more and more people into this constellation where they all signed up to this contract that said, we licensed you our patents to make this.

And there’s a sting in the tail. If you assert any of your patents against any user of AMQP, the patent licenses from all other licensees are withdrawn from you.

Yeah, because the patent part was the biggest worry for us. And back to Derek’s point, another thing about TIBCO, and this makes the point beautifully to me, they have this thing called subject-based addressing. know, x.y.z, you’ve got a little string that sort of, they patented that. So no one else could make subject-based addressing because they had the patent on it. And the patent expired two years ago, I notice, and now NATS is using subject-based addressing because the patent has expired.

But back then, patent well and truly enforced. So we had to tiptoe around all of this stuff. And I remember I really did not want to do AMQP. I looked everywhere. I’m an expert in so many internet protocols now. I thought, what else can we abuse to do messaging? Can we use, can the email approach to the call be changed to do messaging? Can the news protocol be changed to messaging? Can NFS be changed to, you know.

Kate Holterhoff (23:46)
you

John O’Hara (24:00)
So many different, I learned all these protocols to see what they felt like and to get a sense of has anyone, can any of these be repurposed? And the answer was no. So, you know, I really didn’t want to do it because it was such a Herculean effort to even start the process. And I remember meeting in the early days, getting all the banks are really good at getting each other together in a room. It’s something we do really well.

And I got about 10 banks into the old JP Morgan headquarters in London, which is the old city of London boys school. And it’s right by the river. It’s right by the Unilever building. It’s a beautiful building. The great hall, the headmaster’s office, the library. And in the library, I assembled all these guys from different banks in a room and talked about, can we commoditize middleware? And they all went, yeah, yeah, that’s a great idea. We really want you to do this. That’s fantastic.

And that’s where the next thing comes from. And people who do open source today know this. Many, many people will say yes. Very few people will put the effort in. as we started the whole process, because we had this production, we had three years before the trading system would be turned off.

Right, three years. The die had been cast. We had to get it done. And the teams that worked on that, mean, Steve Connolly, Mahesh, Jeez, Lazar, the guys in that part of the project, inside JP Morgan, making that project a success, which they did incredibly well. And by the time we finished, we had a slider. We could actually slide across and say, new messaging, old messaging, AMQP, other thing. I could just literally slide across.

different parts of the world and slide them back again if it wasn’t working. We just slid it across and moved the whole thing across. But that was an entirely non-trivial effort to make that happen. And they succeeded. And whenever we were putting the… So this is all happening in the background. That’s my day job, right? And I’m chief architect for global equities at JP Morgan. Non-trivial thing. And I’m driving AMQP. And one of the things about driving a protocol is we had…

don’t know where got the idea. I had participated in the FpML protocol beforehand, so I knew how hard this was. It’s so hard to bring diverse views together and competing commercial factors and competing visions and emotions. It’s a huge thing. And I resolved that every week, every Wednesday at five o’clock, I would have the AMQP working group call. Come hell or high water for seven years, that call happened. And the come on.

We’d open the conference bridge and people would come onto the conference bridge, maybe two people, three people, six people, 14 people, 25 people, varied wildly from time to time. But they needed this constant, you knew that on Wednesday at five o’clock there would be always an AMQP working group call. And on that call, sometimes I was the only person talking for the whole way through. Status update, how the projects were going, how their event, how parties work was going, how the protocol development work was working.

what edits have been done this week, and you just tell people. And the very fact that they’re there listening to you, they’re adding their support and it matters. They don’t, that is its own participation. I think people underestimate just how valuable it is having supporters listen. And it becomes quite grinding down because you think, wish someone would help me. But they don’t, and that’s just the nature of it. And I think people who drive open source projects see this as well. It’s like, hurrah, go on.

Do the work. Make it happen. Let me know when it’s ready.

Kate Holterhoff (27:30)
So in 2011, the AMQP Working Group announced its reorganization into an OASIS member section. And in 2014, OASIS AMQP was approved for release as an ISO and IEC international standard. Do you have any thoughts about this period in AMQP’s history?

John O’Hara (27:47)
Yes, I think OASIS, the standard frameworks are out there, OASIS and others are very, very useful. We should have gone to OASIS earlier. The work we did was, I mentioned being paranoid about patents, but I think going to OASIS earlier would have been good. And OASIS makes life very straightforward. The ISO standards themselves I have been involved with ISO 2022.

The SWIFT standard, which banks use, I’ve had people work for me on SWIFT. And working on ISO is very, it’s a very tough environment to work in. OASIS is a nice in-between point. So I thought that period at OASIS was fantastic. And in fact, we’re still there, we’re still at OASIS. OASIS is kind of where the standard nurtures and then it goes across to ISO.

And 2006 was a huge year because we put the thing live. We actually moved from AMQP working group to OASIS. And OASIS were lovely. We sat down one day and we thought, will we?

do this ourselves this way or would we move to one of the formal standards bodies like OASIS or IETF and we had to pick which one. We chose OASIS because they were really nice people and it wasn’t an overly bureaucratic process and they were affiliated with ISO. And we kind of had this view, maybe one day we can get to an ISO standard, you maybe, which is a bit insulting to OASIS because it’s like they were, they’re a cracking organization, but we thought, can we get to the gold standard, you know? And we moved it there and that worked really well.

building and building and building. And it went on. But the project went live successfully. There’s some stories around that as well, you know. Because there’s standards and there’s implementation. A standard is worth nothing without a useful implementation. And there are standards out there. And I remember I built an implementation of CORBA Event Service once. I thought, why is this so hard? And I bumped into some else’s. Yeah, it’s really a strange thing. And it’s very hard to make it work.

We bumped into one of the standards authors one time and said, oh yeah, no one ever built one until you, know, no one had ever built it before we wrote the standard. And the idea of that never even occurred to me that people could be writing stuff down that had never been tried. It turns out this is quite common. We had resolved not to ever do that. But one of the, so back to the sort of the plot, this going live bit was a real crunch point because I had a team in Glasgow.

And we needed some Java implementation. So got the guys in Brussels building C. But they didn’t do Java. So I’d actually got my own team in Glasgow, a guy called Robert Greig, writing a JMS client for AMQP. And he did a great job. Very talented guy. But the main C project was running into trouble. We had this rock, this really, really crisp drop dead dead.

that would have had a massive tens of millions of dollars impact if we’d been up against the wall having to pay a license at gunpoint. It had to go live and the guys writing the C code were having some performance issues. They decided to do it multi-threaded and that was kind of difficult and writing multi-threaded code in the 90s and that back then was incredibly, it’s hard today, it was really hard then and it wasn’t quite working and I really wanted to go live because we said that the deal was

If we go live on what you write for us, we will give you the source code. We will give you copyright ownership of the code if you go live. That was the incentive. We’ll pay you to build it. And if it works, we’ll give it to you. And we’ll buy support. We’ll make it really work for you commercially. So I needed this thing to go live. I wanted to go live on the C implementation. Because there’s so many

assumed personal contracts. People have have given their all You know, they’ve worked really hard if it doesn’t get used and they’ve the guys find that the investment banking pressure I’ve hear about working investment banks. The pressure is totally real and It’s in the technology divisions everywhere And I think that they’ve never seen anything at that level of pressure and they’re coming into it

from outside with the right hands and then suddenly the full weight of it was on them and they felt that. they made a few tweaks, we changed a few configuration settings and we were able to go live. We went live with it and it worked. But this is where it gets really interesting. Coming up to that point, coming up to that point, we weren’t sure it was gonna work.

We needed a Plan B. So we activated Plan B. And coming out on Christmas, year of, I think, can’t remember which year it was, 2005 or 2006, I talked to Robert, who’d written the JMS client, and said, Robert, can you turn that into a full implementation of the messaging system over a weekend? He went, I’ll have a go. And in the course of one week, he wrote a complete implementation of it in Java that would be enough for our needs. So Plan B was sitting there, and it worked.

Now, I’ve got to say, this sort of says the difference between working in C and working in Java. C is so much harder, so much harder than Java. It was so much easier to do this in a modern programming language than was to do it in C. So there’s no denigration of anybody here at all. But we were doing it C because we wanted it to be, why did we want it C? We wanted it be something that could go into microcontrollers. We wanted it to something that could be shipped with the Linux operating system.

because Linux was all kind of C back then. Things must be in C to be part of the ship with us. The Apache web server was written in C. So we wanted to have that kind of genesis to it. And then Red Hat came along. And Red Hat had been part of the working group for a long time. And I met Carl Trieloff at IONA when he was there, and he had moved across to Red Hat and wanted to do middleware in Red Hat. And they’d come along, joined in, and I thought, this is great, fantastic.

At some point along this journey, got Microsoft involved as well. The web traffic went off the charts as Microsoft got involved publicly. Carl wanted to have more expertise in his organization. I think he had talked to iMatix about could they join to do something together, and that didn’t go anywhere. I had this problem that Carl was upset they had no

expertise on hand and the iMatix guys said they didn’t want to it so I said well have some of my guys and with JP Morgan’s permission some of my staff rotated into Red Hat and took the Java implementation with them and that became Apache Qpid. So Apache Qpid was originally written inside JP Morgan which is that thing I said I didn’t want to do and we

hand it across to Red Hat, but it looks like Red Hat products in every way, and Red Hat will have all the patent licenses and stuff that they need. And now they’re owned by IBM, ironically, which is interesting. So yeah, it was very long journey. And it kept going from there. So we should stop. So that’s kind of the crunch point. When we got to that point, the number of sessions we were having in San Diego, New York, in Boston, in Brussels, we were…

meeting in London, we were meeting regularly everywhere, we had the calls going, we had this wonderful core of people working really hard driving it forward, we had great debates between the guys who wrote Rabbit and the guys who wrote OpenAMQ and the guys who were working inside JP Morgan and the amount of learning that was done in the whole area of middleware was off the charts, everybody learned so much.

And that was a wonderful, wonderful time, wonderful crew. I’ve got some pictures from San Diego, the event we had in La Jolla there at the Scripps Institute. And just wonderful, you know, seeing this thing properly take off. And I think Rabbit sold to VMware around about that time. And it was great watching. I should go say about Rabbit, watching Alexis grow that company was astonishing.

They did such great work in making the product, in educating the market. The educational materials they produced were awesome. The developer evangelism they did was off the charts. They worked with you guys as well. And to see him put in the hard yards, he put in so much work to make that thing take off. And it’s inspirational. In fact, watching Alexis do that is what inspired me to start my own company.

Kate Holterhoff (35:42)
Mm-hmm.

John O’Hara (35:58)
yeah it also taught me how how hard it is to do it because i could see it was grinding him down and so whenever he sold to VMware that was fantastic

Kate Holterhoff (35:57)
Really? that, yeah, that really is a…

Talk to me about the AMQP 1-0. Why was it so significant?

John O’Hara (36:12)
Ah, the ISO standard it was significant. 1-0. There’s two parts to this. Getting the ISO standard was a massive thing. Governments buy ISO. ISO is attached to the World Trade Organization in some way. If you want to do something at governmental level or Department of Defense level, it’s going to be much easier to buy things that adhere to an ISO standard.

That was huge and Microsoft helping us get there and Red Hat helping us get there was very important.

and now this this is you wonder why there’s two protocols right because people watch me go what do mean two protocols there’s zero nine one and one zero and they look like two protocols and there’s i’ve been sort of going around people asking what happened there just recently because i i had

This is also at the time I moved from JP Morgan to Merrill Lynch and Bank of America joined the working group as well and standardized AMQP much to the entrenched vendors annoyance because this was a multi-billion dollar business for the mainframe company who shall remain unnamed and they were a bit annoyed by that. We thought we were done. Not we, I thought we were done. In fact, we thought we were done. It’s a wonderful.

really great protocol. It was just the right amount and everybody had had some input into it and people trusted me to lead it and they probably thought I had more control over it than I did and I thought I had more control over it than I did. There’s a point, if you start your own company, there’s a point you realize your company doesn’t need you. At some point, AMQP reached the point where it was moving without me and I hadn’t noticed it.

So we thought we were done. We thought 0-9-1 was it, and 0-9-1 would become 1-0 and would be blessed by ISO and all this sort of stuff. But it had missed something. We had all the things that users needed in a protocol. Users want one tool that does the job really well. Vendors want flexibility, optionality, backwards compatibility. They want the ability to support

different features and different products. And that’s something that I’d fought quite hard against when I was a chair. And that was a mistake. If I had made it easier to accommodate variation in the protocol, it wouldn’t have fractured the way that it did. There’s two protocols there. 0-9-1, which is what’s at the heart of Rabbit, is a fantastic user. Really, it’s incredibly useful. It’s an incredibly useful thing. It does everything that business really needs.

and then 1-0 is easier to retrofit. It can be retrofitted onto other existing things. So that means that a user using it can’t quite get an expectation of consistent behavior. The way I talk about this these days is like, You pick up a USB cable and you don’t know what it’s capable of. Yeah, this is just becoming a problem right now. Some of them can do video.

Some of them can do power delivery. Some of them can just do, you know, read your USB stick. But you have no idea when you pick the cable up which one it is. So that’s really good for vendors and not so good for users. And that’s what had happened. And we have, so we have actually kind of two protocol families there. And a pretty good understanding of why they are the way they are. But it did confuse everybody. I thought we were done.

went across the Bank of America where I had to put my roots down, had to start proving myself, know, chief architect for Merrill Lynch’s global equities division, non-trivial job. And I kind of stepped back, Matt Arrott took over the chair of the AMQP working group and the thing kept moving. You know, the interested voices moved it the way they wanted it to move. And that’s what happened. And I think so many things to learn from the whole experience.

I was talking to Alexis about this recently, it’s like, I thought having done seven years that was enough. Alexis would be no you need to do ten. So yeah, so I still think, I mean the places AMQP is today are utterly amazing. Rabbit’s version is in the Indian government national ID scheme. know, a million new identities a day come out of RabbitMQ in India.

Some of the versions of 1-0 are in the railroad system in North America runs AMQP and its crash protection system. The EZPass toll system is AMQP. The Deutsche Börse Stock Exchange, their protocol is AMQP. The R3, which is a crypto blockchain company, the R3 Corda blockchain in the middle of it, AMQP. It’s everywhere.

It’s everywhere. I think PayPal were running a version of it at one point, but they’ve moved on from that since then. But why is a standard important? A standard is important because…

Software, a messaging system that’s defined by itself it’s not dependable because the author can change their mind. If something is defined by a standard, then you know the products that support it are not just defined by themselves. They’re defined by this external thing that is the Rosetta Stone, the point of truth. Does your thing adhere to the standard? Yes or no. You can manage that. And that’s worth a ton on its own.

I remember talking to…

James Strachan. James Strachan was the inventor of ActiveMQ. He also, I think, invented the Groovy programming language. A very talented chap, this James. He used to work at JP Morgan. He was involved. I found a commit in the system I was upgrading from James. And James had gone out, because again, I said I didn’t want to write messaging. I’d gone out James and said, James, you know,

Are you interested in supporting AMQP? And he went, no, just use my product. And I’d gone to IBM and I’d asked Andy Stanford-Clark a long time earlier in a meeting in JP Morgan, said, Andy, would you open the protocol for MQ series? No. You know, I really didn’t want to do it. I really didn’t want to do it, but it needs to be done. And it’s like, if you look at all the things that really matter, know, POSIX.

is an ISO standard. C is an ISO standard. C++ is an ISO standard. SQL, SQL is an ISO standard. If it really matters internationally, it’s an ISO standard. And SWIFT, Payments Network, ISO standard. these things are all incredibly painful to make. the standards processes typically are underfunded. They’re made as labors of love by people who really care.

And but it’s not easy. And I would love to see that we kind of get to a Postgres of messaging, you know, where there’s an Apache web server of message. That’s what AMQP stood for originally. my code name was Apache Message Queuing Protocol, right? And I changed that to a message. I can’t do that. We’re not there yet. So I changed it to a message queuing protocol. So the original name was a message queuing protocol.

And Pieter from iMatix came in and said, no, no, no, no, no, no, It must be more interesting. That must be advanced message queue protocol. So I had this thing where I wanted each contributor to leave their mark. They were giving of themselves in the process. I wanted them to be able to leave a lasting mark and his is in the name.

And it’s also, you know, the exchange concept was a nice way of bringing together that we had three messaging domains. There’s queuing, PubSub, and there’s this other one that we needed for JP Morgan. We needed full content-based routing, the headers exchange. Everyone goes, what’s that for? It was for us, it was necessary for the business case. It’s also the thing that prevents backwards compatibility with third-party messaging systems. Because no one else did it. No one else did full message.

routing messages by just saying, I want all messages that contain the temperature. Okay, the message system goes, yeah, I will arrange that to happen for you. And it goes off around the world and gathers everywhere that’s appearing and funnels them all towards you, which is magical. And in fact, you see why we had it, it’s the essence of a global order routing system for equities trading. Bring me stuff here. So that’s kind of where that came from. And everybody got to put something into the protocol.

And actually some of the discussions I had recently said that that was not a good idea. know, that that was one of the weaknesses. We should have thrashed things out in a different way. Yeah, so it’s it’s a long journey. I still deeply care about middleware. It’s an unfinished journey. I look at Postgres as being an interesting example because, you know, Postgres was rubbish in version nine back in, you know, Postgres is 40 years old. And if you go back 20 years, it was utter rubbish.

And now it’s world class. Now it’s world class. Standards take a very long time to work their way through, but they’re totally worth it. They just take an awful lot of

Kate Holterhoff (44:48)
All right, and you mentioned IBM. I want to hear a little bit more about the hyperscalers and how they play into this story. So before AMQP, IBM dominated business messaging, but Microsoft’s involvement is perhaps more well known, yeah.

John O’Hara (44:55)
my.

They still do. Microsoft

were great. Microsoft, the guys there. David, And Sam, the guy who knew the ISO standard process. They kind of shepherd it through that process, they shepherd it through the vendor-friendly version. that meant a huge amount to see Microsoft legitimize this. And we had an amazing event.

We had an interoperability event in 2009 at JP Morgan’s headquarters on Park Avenue, 270 Park, the one they’re rebuilding right now. And I was in Bank of America at that point, so they invited me back in for this event. And they had all the vendors who were participating who built implementations. had six different vendors on the stage who had built a version of the AMQP, and they demonstrated them all working together.

Microsoft, Red Hat, IONA were there, IONA have been bought by someone else now. There was, SwiftMQ, Andreas Müller from SwiftMQ, excellent guy there. And we demonstrated interoperability. And we had pledges of allegiance from companies like SAP, know, they said, we’re in, we’re in. And to see that demonstrated on stage was quite something.

Kate Holterhoff (46:06)
I’m interested in the fact that AMQP is the primary protocol for Azure Service Bus and Azure Event Hubs. But yeah, IBM was the impetus.

John O’Hara (46:08)
IBM.

Yeah, so Microsoft have

really understood the value of open standards. The fact that they reluctantly embraced ODF for open documents. They have embraced Linux like you wouldn’t believe now. They embraced email, SMTP. So they embraced HTTP. And they really understand the value.

And this is where FpML came from, I remember whenever JP Morgan did FpML, and I was part of the team that did that. But the business guy who invented it, I think a guy called Philippe Kwong Hu, one of the traders, and I said, why are we doing this? know, because I was young, why are we doing this? And they said, well, look, there’s two elements here. One, there’s legal risk. If our clients don’t understand what they’re selling them, there’s a legal risk. And number two,

If we tell everybody how to do it, the pie gets bigger. We’ve got the biggest slice of the pie. Our slice gets even bigger. Standards have this effect of growing the pie. And if you’re a really good implementation of a standard, you get even more businesses with all of that. So Microsoft’s support for AMQP, invaluable. It being a service bus, invaluable. It’s very, very hard to make things stick.

like SMB, the file system for filers now, which is Microsoft’s networking protocol for file systems. That’s taken over the world. I think it’s quite open now. But people have realized that that’s important. And then you see that the cloud vendors kind of holding on vestigially. And there’s a whole other set of discussions here. Because I mean, if you look at what I do, it’s like Chief Architect for

Big banks thinking of IT strategy and stuff is really important. And you think, well, what are motivations? And they really want to make you sticky. They want to become sticky. Only recently has Amazon put AMQP implementations up there. Amazon followed Microsoft. Microsoft had Service Bus, it was AMQP. Then Amazon had their own SQS, SNS, Simple Queuing System Notification System, entirely proprietary.

at some point they blinked. They’ve got two implementations now that support AMQP, Rabbit and ActiveMQ. And ActiveMQ, that’s James’s product. James Strachan, he said, just use my software. His software now supports AMQP. He invented, he added STOMP, the simple text message protocol to, there’s so many projects that started to try and compete with or kill AMQP, which is really interesting. I would say STOMP is definitely one. OpenWire is one. I think MQTT.

was made public by IBM to confuse the market a little bit and also follow the same standardization process as AMQP. I think that was deliberate. And the funny thing of that was, and then MQTT is a lovely product by the way. I’m not dissing anything here, but with AMQP we out to make comprehensive industrial grade middleware. It was, you couldn’t use it without logging in. It was secure by default.

It supported transactions and stuff like that. was everything you needed to run your business on. And the other ones that came out didn’t support those things. And that’s enough to make you have to go back to the expensive product. We were trying to standardize the core of the thing. And even ZeroMQ. ZeroMQ, came out of the sort of iMatix got upset with us after the Red Hat thing that kind of went.

bad, a lot of upset, and ZeroMQ is born out of that upset. Another thing comes out of this is lightweight messaging versus heavyweight messaging, which I find tremendously funny. Lightweight messaging is not a way to denigrate anyone. People think in technology that lightweight is good. No. You run Linux. Linux is anything but lightweight.

Right? You run Postgres. Postgres is the one most heavyweight database as you can imagine. Right? If you’re doing something lightweight, you’re just leaving the power tools in the cupboard. Why would you take the little saw when you can take the big saw? Right? And this is the thing. So one thing that really gets technical guys and

It’s funny to me with Aeron because Aeron is lightweight messaging and it’s a scalpel for a very particular job. It’s high frequency trading. has to be, it doesn’t care about being global. It cares about being really fast to stock exchange as fast as possible. But for everybody else, that’s not the normal use case, right? If you wanted to get a package to a friend as fast as possible in an adjacent city,

The fastest way to do it might be to get in your car and drive the package to them. That is a lightweight messaging system. You got in your car and you drove to them and you handed it to them, right, and they were expecting you. Nobody does that in business. They all give it to UPS or DHL or FedEx. Yeah. And that’s the heavyweight system. But the heavyweight system is more convenient to use than the lightweight alternative. And it gives you a much better quality. look, I can watch where it is. I can see it being…

And if they lose it, I can claim insurance on them. I get tracking. Even the diamond industry, the jewelry industry, sends its stuff using FedEx and DHL. They’re not going to, you know, this is so valuable. No, they just stick it. They give it to someone else whose job it is. so I think that they, but it’s so seductive to technology people to try and make something, myself included, to try and make something really fast. It’s so seductive.

and you do that point case and it works okay. And this is the thing, anyone can build a messaging system in a weekend. You think, great, I’ve got a queue, fantastic done. great, got PubSub, fantastic done. It’s just the tip of an iceberg. And that iceberg is enormous. Undeliverable message, security. Back pressure control, credit control, network admission, know. Fragmented messages, ordering.

Ordering the presence of model consumers, high availability. The amount of stuff inside messaging is absolutely huge. And I’ll say to people, when they say queuing is slow, I’ll say your operating system is made of queues. The thing you’re using right now is full of queues. An operating system spends its entire time queuing things. What they’re doing is making sure the queue’s in the best way possible so you experience this incredibly

seamless flow of activity, but it’s doing it by managing queues. There’s this other irony of messaging systems, and that is when you go into a management console, say for RabbitMQ or something, and you look at a messaging system that’s maybe processing a thousand messages a second, which isn’t very much these days. You could do a million messages a second, which is enough to serve the entire population of your country every day. People forget how fast this stuff is.

I remember actually one of early demonstrations that Alexis team told me he had years ago, was they were doing pub sub video. They were publishing video as messaging, as messages. And each frame was a, each frame of the video was a message. And you could just watch a movie on it. And people think, my goodness, that’s so fast. Actually that’s slow compared to what can really happen. That reminds me, Shrek 2 was animated using AMQP, the render farm, the jobs were allocated over AMQP.

It’s everywhere. So yeah, lightweight messaging. It just gives you a half-assed solution. And If you look at a queuing system that’s running really fast, and you look at how many messages are in the queue? It’ll say zero or one.

Kate Holterhoff (53:26)
I didn’t know that.

John O’Hara (53:43)
zero, one, 10, zero. What’s happening? A thousand or a million messages are passing through every second, but everyone going in is being immediately consumed. Yeah? But the, and you go, why do I even need a queue? The benefit is that everyone going in can be being consumed by 1000 consumers like the render farm or everyone going in, they all disappear, if your processing agents all stop and disappear, they can be buffered up. And when they come back, they can be pulled out again. And the sender never knew.

You it’s a bit like having that you don’t care about a road traffic accident where you’ve sent your piece, your delivery via UPS. You do care if you’re in the car and you see the road traffic accident. You’re in it. You’re delayed. If you’re if you’re using a service, they’re taking care of this for you. You know, they’re working on how to route around it to whatever. And so people think queuing is slow and it’s not. And that if I could if I could make people understand one thing, it’s that queues are not slow.

But of course everyone takes their experience of queuing for a stadium or queuing for a car park and they think queues are bad. But actually, no, they’re not. They’re the way that that even works. So there’s so much stuff to be standardized still. If you look at database SQL and Postgres and the capabilities it has, it are immense. If you look at what file systems are doing and how they are.

you know, allocate the storage across things and make that work quickly. Immensely complex stuff, instantaneous snapshot backups, things like that. And these are now just taken for granted. I want that to happen for messaging.

and there’s little fragments out of there in different products but every product’s different. You don’t know if someone’s going to be bought by a PE firm and the price is jacked up. It’s just things need to be defined by the standard not by themselves.

And there’s so much, I’d love people to be taught messaging in university. I’d love us to get rid of HTTP for microservices because it’s stupid. If you go into a bank, a trading system, they’re not using HTTP. They’re using high velocity pub sub messaging. And everything’s done that way. TCP sockets are for, no, don’t do it. You’re just asking for a world of pain.

Kate Holterhoff (55:56)
You mentioned HTTP last time when we spoke about messaging. Can you talk more about that? Why so many folks use that as the standard?

John O’Hara (56:05)
That’s so funny. People forget, why things are the way they are. Like, HTTP was invented by Tim Berners-Lee to read web pages, to read documents, and to publish them back. So get me some documentation to see, you know, and post, that’s actually the verb, post something back to update it, and delete it if I don’t want it anymore. Yeah, document-centric, right?

And everyone started putting web pages out there to have their presence on this new, final internet thing. And so it’s the one thing in a company’s network you could guarantee was accessible was their web page. people thought, engineers got the point, well, we need to be able to do something between companies. And the developers thought, well, I can’t.

talk to them to make their network engineers unlock the door. But I can see that port 80 is open, so just do that. And that’s why people did HTTP, was because it was the one, they could have used any other one of thousands of other network ports. The time protocols, they all run on different ports. But no, all had to go through, everything in the world is going through these two ports, 80 and 443, and using a protocol designed to fetch documents.

because it was the one door that had been left open by the network engineers. And no one wanted to talk to the network engineers. It’s a political problem because the guys who take care of the low-level network are entirely separate to development teams, are entirely separate to the business teams. So if you wanted to make something happen between companies, no one even knew how to talk to the network guys, which is really funny. so consequently, and indeed there’s another protocol, WebSockets. WebSockets is the same. Well, gee, how can I get…

my stream of data into the other company. know what I’ll encrypt at first. can’t see what it is. They’ll go through the HTTPS web port for web pages. And then what I’ll do is I’ll hide, I’ll send a special magic bite sequence that says right now, I’m just going to send stuff to you, right? And that’s WebSockets. It’s essentially a tunneling over everything else is bypassing all the existing security.

So it’s interesting, it’s a bit like today, the word is our roads are the width they are because that’s the width of an oxen dragging a cart in Roman times. We kind of got that in technology everywhere. But I very much like to see people, yeah, that’s troublesome. I mean, I won’t go into detail. It then becomes self-limiting in its own way.

Kate Holterhoff (58:25)
I love that story.

John O’Hara (58:33)
But yeah, I’d like to see us sort a few things out. But of course you can’t go backwards, you only can go forwards.

Kate Holterhoff (58:39)
That is true. All right. So AMQP has been around for 22 years. Do you anticipate it sticking around for another 22?

John O’Hara (58:42)
So, hey.

I can’t believe that.

Standards are… It’s very easy for it to outlive companies.

The fact it’s in the railway system, it’s in bank systems, it’s in payment systems for roads, it’s inside other protocols now. It was very, very carefully thought, both versions of it. And Carl at Red Hat said that the key difference in the two versions is that the 0-9-1 he sees as being asymmetric. One side is fat, the other side is thin. That was a deliberate choice, it was user friendly.

The one the vendors wanted was symmetric. They wanted something that could be brokerless and could retrofit on other products and could be tunneled through proxy things. They had the specific requirements they wanted to meet. And that’s the key difference. But both of them are valuable. both of them, know, 0-9-1 is an OASIS published standard. It went through the entire process to get there. There’s so much well articulated intellectual property in those standards.

They’re really well described and the companies that built up around them, you you see on the internet, we talked about cloud vendors a minute ago, but you see on the internet some people do queuing as a service. A lot of them are using RabbitMQ or RabbitMQ derivatives. It’s a great product. It’s a great protocol. So I think it’s got legs. I would like to see the standards process reopen. Standards process is supposed to reopen every 10 or 15 years and for folk to come together with the benefit of hindsight and turn the wheel.

And if I could add to what’s been done there, would be to the biggest mistake I made was not embracing extension points or profiles they are sometimes called. And I had a terrible reaction to the word profile because Bluetooth headphones. 20 years ago, Bluetooth, none of it worked. It was all rubbish, right? And the profiles were what was getting in the way.

Bluetooth works fantastic. Another great example. Now Bluetooth is awesome. Back then it was rubbish. These things need to be handled. Turn FpML, the version I have worked on was not great. And the ones in between were very challenging. Version five is a fantastic piece of work. You know, it took 10 years to get there. The process of bringing the people, the industry experts to get together again to melt in the learnings and

turn the wheel on it one more time. I’d love to see that happen.

Kate Holterhoff (1:01:08)
I love the Bluetooth headphone example because yeah, complaining about Bluetooth not working in my house is something that we do pretty often and it still works great. You know, earbuds, everything’s Bluetooth now. You can’t get away from it.

John O’Hara (1:01:15)
you

Yeah, and I think that’s a great, that is why standards are worth persevering with. And they also outlive companies. Companies, you talked about hyperscalers. The hyperscaler, and I talked about the fact that they don’t embrace standards until they’re forced to, because they don’t want you to be able to move to another hyperscaler. They want you be paying for value added. They want you gradually raise the prices, hypothetically, perhaps. And they want you to forget that there are alternatives. They want you to forget you can do it yourself.

You know, so yeah, it’s it’s and this this then crosses over into open source, which I won’t touch on now, but the the monetization of open source by hyperscalers without revenue flowing back to the source communities is deeply problematic. Non-trivial to fix. I haven’t got a solution for that. The same sort of true with standards. It’s like

And again, the example is, we talked about this, this is an old AirPod. This has got a lightning connector, right? Who else but Apple could get away with having non-USB charging connectors, right? If you are dominant and have an amazing product, and Cisco used to this with video conferencing, telepresence used to be, they’re full-size, life-size telepresence rooms, which are amazing. They were entirely proprietary.

When you’ve got something awesome, you don’t want to standardize it. But then you look at your TV at home, and you look at that HDMI cable. There’s another great example, HDMI cable. Over time, people eventually realize that growing the pie is worth more. It just takes them a long time to realize you’ve got to grow the pie. If everything plugs into an HDMI cable, then you’re going to have more people to sell to. They’re going to understand what they’re buying. You don’t have to educate the market. If people know it’s an AMQP message broker,

You don’t have to educate them. They know what it is. If they got this and this with say, AMQP on them, they should be able to just plug them together. And Rabbit did a massive amount work on that with the fact that people loved Rabbit so much. So many people wrote open source clients for RabbitMQ. Of course they’re writing clients for AMQP 0-9-1 as well. You know, they worked with the Apache Qpid, fine. And that was the benefit of it. It’s like plug and play. It’s definitely worth it. So yeah.

But I also discovered in the process of making this, I talk about standards being underfunded, I discovered very late in the day that some people were paying for themselves to come to some of our events. You know, their company wasn’t sending them. They were coming themselves. And you think, you know, I want that person’s input, but why is, and you can see why the business is going to look at just immediate direct next three month profit. don’t usually care about the next time horizon.

And maybe they’re conflicted, like the cloud providers, if you’re so dominant, you don’t want to create standards, but actually, they do advance us as humankind. That sounds very noble, it’s not meant to. And as I say, I never wanted to do it. It was such a hard thing to do. It’s such a hard thing to do. so.

Kate Holterhoff (1:04:12)
You

Well,

you’re among friends here and I appreciate your candid answers and I don’t know, it sounds like you kind of are a hero.

John O’Hara (1:04:23)
The thing is, retrospect, you realise all the mistakes you made. And I’m a lot older now. And having run my own company in between, that changed a lot of ways how I think. The world wasn’t quite so black and white at the end of that. When I was younger, I was blind and deaf to certain things. Which maybe I wouldn’t be today.

Kate Holterhoff (1:04:45)
Well, you’re not alone in benefiting from hindsight. Nobody can see the whole thing while it’s happening. And a lot of good came out of it in the end. And AMQP, as you said, is still being used. The implementations Rabbit, specifically. it’s still extremely dominant and extremely important.

John O’Hara (1:04:54)
Yeah.

And the one thing that made me smile the most was I mean to see the main all the main cloud providers have a AMQP implementation now all of them maybe not Google might have with Rabbit now can’t remember using Rabbit or not. Maybe they’re not sure but The one that made me smile when I discovered that IBM MQ series had a new chapter in their reference guide a AMQP compatibility or interoperability and that made me smile because

Kate Holterhoff (1:05:29)
I can see why.

John O’Hara (1:05:30)
A long long time ago, whenever we decided to do this, a, JP Morgan, we’d done one those big corporate trips down to Hursley. There’s something about middleware, all done, so much of it is made in the UK. So much of it. Hursley is near Winchester. It’s a real stately home. IBM have a lot of mainframe stuff down there, and MQ series is down there as well. And we went down there as a corporate delegation, and we’re doing the chit chat, know, 10 people from their

organization down one side of this big table, 10 people from us down one side of the table. And my boss says to me, John, tell them what we’re going to do. I just looked at it, was Mark, Mark Etherington. I said, what do mean, tell them what we’re going to do? Tell them what we’re going to do. I said, we’re going to commoditize middleware. And the looks that came back across the table were, because we had this, they had no forewarning. So they all reacted viscerally immediately. And some of them,

Kate Holterhoff (1:06:13)
You

John O’Hara (1:06:24)
almost burst out laughing, and others looked deadly afraid.

So that was funny. But yeah.

Kate Holterhoff (1:06:30)
That is funny, and it sounds like with good reason.

John O’Hara (1:06:32)
So grow the pie. There’s so much space. it’s one thing to sign off with. There’s so, I noticed when I was building applications that roughly at least a third of every application could be done just by a good messaging system. That sounds, when you get close to the code, about a third of every application can just be done by a good messaging system. And to have something everywhere.

in every device on the planet, in every computer, in every phone, to have that capability, much like SQLite is, SQLite is the exception to light things. It would just transform the opportunity for different types of commerce and business and some degree of interoperability and work out ways of differentiating. It’s not easy to solve the commercial part of this. In fact, one of the things we did with AMQP is try and make sure everybody had a reason to be there, commercial reason to be there, a good how they were gonna win.

Everybody had to win. Yeah, I think that it would really help us if we could have middleware be as pervasive as the web itself.

We’re all running Chrome or Firefox but Chrome. And we’re all running HTTP and we’re all running those web servers. I would love to see the same thing go into message queuing.

Kate Holterhoff (1:07:41)
And I think that call to action is a good place for us to wrap up here. So before we go, do you have a social media presence where you record your musings on this subject and others? Is there a place where folks can follow you?

John O’Hara (1:07:54)
I should.

It’s funny, I’m getting to the point now where I’m thinking I should just write more down because, you know, it’s not made people like writing software. don’t like writing things down. And Gregor Hohpe did a fantastic work with his Enterprise Integration Patterns book. But there’s there’s a whole bunch of stuff. And I’ve been asked by a whole bunch of people to write down the stuff that I learned across my career.

Just to make it pithy and accessible and easy I might do it. I’ve not got it yet So I’m just on LinkedIn so John O’Hara at LinkedIn. If I launch anything it’ll be off there

Kate Holterhoff (1:08:26)
Wonderful. So we’ll look for your blog or your forthcoming book, it sounds like. I don’t know. It sounds like you committed to me. I don’t know. This is what I’m hearing.

John O’Hara (1:08:31)
No, pressure. You’re gonna have covered all the ground, Kate.

But

you’ll have covered an awful lot of ground yourself.

Kate Holterhoff (1:08:42)
I’m working on it. I’ve really enjoyed speaking with you, John. Again, my name is Kate Holterhoff, Senior Analyst at RedMonk. If you enjoyed this conversation, please like, subscribe, and review the MonkCast on your podcast platform of choice. If you are watching us on RedMonk’s YouTube channel, please like, subscribe, and engage with us in the comments.