My Stephen colleague yesterday published an interesting post The Difference Between SOA and Microservices Isn’t Size. In it he argues that developer led adoption of technology – the new kingmaker thesis – is why microservices are different from SOA.
Apart from being made to feel old – we now live in a world where SOA can be described as “archaic” I thought it was worth adding some colour about why and how things are different this time around, and why and how microservices are indeed different. There is of course more context we could have pointed to – SOA was of course a rehash of, and reaction to, concepts we’d attempted to make stick during the Object Oriented technology era and the delights of Corba. We could call out 4GLs and the idea of accelerated development through reuse.
Technology adoption always has a context, and when it comes to application development, we’re not in Kansas any more.
SOA was an artefact of its time. When SOA was codified open source was not yet the commonly adopted way of developing software, the Internet was in its infancy, and Waterfall was the commonly accepted approach to developing software. Application development, testing, QA and operations were effectively separate disciplines with their own deployment environments and budgets. There was no social coding. Search first development wasn’t a thing. Web companies were not making open source contributions as a side effect of their business operations. SOA celebrated reuse, and while there was an idea we might swap out components and services comprising a whole, we certainly weren’t aiming to make code disposable. Nobody expected to use SOA to make multiple code changes to production systems per day. There was no concept of continuous deployment. Netflix was still a hairball. Apps were stateful. Web Scale was barely a thing. Web programming meant HTML not Go. Forking was considered harmful. People were still buying and deploying on premises Enterprise Applications. Teams were fricking enormous. Containers was some weird crap on mainframes, and maybe Solaris if you were l33t. In his post Stephen mentions that Amazon was the only successful SOA. Indeed. Back then there was no Amazon Web Services, no universal permissionless deployment mechanism.
Fast forward to today. We’re cloud native. Open source has won. Agile has won. DevOps has won. CI/CD is so obviously the right approach. Git and distributed version control systems is very much the new normal. The idea of building services based on black boxes seems faintly ludicrous, although with the rise of serverless we’re getting a taste for it again. We build with an an expectation of disposability. The state of the art in testing, for systems that run on the internet, is light years ahead. I would probably go as far as to argue that testing itself is what is different. Better testing approaches, that better suit our deployment models, is why microservices are going to be broadly applicable. This excellent slide deck by Toby Clemson, with input from Martin Fowler, lays out effective microservices testing.
With SOA, on the other hand, we really hadn’t thought through how we needed to change testing at a fundamental level to make everything work in distributed systems engineering. We were still trying to make faster horses. With SOA, testing was something that other people did. With microservices, testing is foundation to the application development process. Good developers take pride in writing good tests. Testing is what you do when do you software engineering. And think of the organisations we had. Huge great unwieldy things.
“Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization’s communication structure.”
Conway’s Law is another reason SOA was so problematical. We had monolithic organisations trying to build distributed software. Today teams are small, they fit around a tapas table, and they invite the product owner to give feedback as they go along, and pay for the wine.
We’re building with an expectation of disposability. With the Docker Pattern of course you’re going to throw stuff away. Enterprises are now getting the message about minimum viable architecture. We don’t expect stuff to last the month, let alone be in production for 5 years.
With SOA and the XML Deathstar we tried to preordain every possible use case, edge case, approach. But that is not how evolution works. The new way is better, and that’s why developers are so enthusiastic about microservices.
As I sign off I should say that I haven’t mentioned any of the downsides of microservices. Distributed computing is hard. Writing good APIs and understanding service boundaries, and bottlenecks, and what you’re going to need to optimise for is hard. A monolith can be the most effective way to establish product market fit. Having come to bury SOA it’s also important to point out that a lot of the people involved in driving microservices lived through SOA. They learned the lessons and moved the state of the art forward. Progress isn’t linear, but a lot of the idea behind SOA were good ones. We just weren’t ready for them yet, technically or organisationally. We had trains and canals, not roads to drive on.
For now yes, microservices are on a saturation curve in terms of developer interest.