Minimum Viable Architecture – good enough is good enough in an enterprise

“Architecture is like the opening of a chess game, you can’t win in the opening, but you can lose.” – Gunnar Peterson

Randy Shoup recently shared this great set of slides, which summarises a bunch of experience in scaling Web companies. My only quibble would be the title – in that the lessons are just as relevant to modern enterprise IT. One of the biggest mistakes we’ve made over the last 30 years, with our focus on “enterprise scale”, is constant premature optimisation – architecting and budgeting as if all systems had to have mainframe class reliability and scalability. The irony of course is that IT spent some much time and money trying to achieve mainframe characteristics, having decided mainframes weren’t flexible enough to support the business. IT hasn’t spent enough time considering engineering trade offs as they relate to the business, with a careful focus on types of workload. I am not a believer in bimodal IT, but rather taking an approach which is more initially experimental, and then iterative, for new business services in all areas.

One of the core messages of Randy’s presentation is about disposability. The code that proves product market fit is not going to be the same code that later scales to 20m users. Organisations are constantly tasked to do more with less. One way to achieve that is with minimum viable architecture. Spend money maintaining systems that justify it. It’s not only silos that we need to break down, in becoming more agile, but also huge IT contracts. Today enterprises should adopt approaches more like Web companies. Bring a small team development team together with a product owner, spin up an AWS instance, let the engineers choose the open source stack they feel will deliver a working model fastest, and get started. It’s important to stress here that product market fit can be applicable to internal, not just consumer facing apps. Disposability is definitely not the comfort zone for most enterprise IT organisations, but they should be helping these small teams to scale, in architecture and personnel only once they are ready. Before that it’s their job to get out of the way.

In truth there is nothing new in any of this thinking. Shadow IT has always chosen technology that “didn’t scale”. Until it did. There was a time when Oracle, now a paragon of “enterprise IT” was more like MongoDB as it exploded in popularity – super accessible and low enough cost for a department to adopt. If it fell over every now and again, that was a small price to pay for development velocity. Today we see major banks claiming they won’t run their business on MongoDB even though they already are. The CIO is the last to know.

This post serves as a follow up to my question – how are we going to achieve escape velocity for modern software development? If enterprises are going to drive a successful digital transformation, and develop a culture that supports agile development and devops, then they need less architecture, not more of it.

Bonus Update – I got some great feedback overnight on Twitter so I thought I should include it.

Evan Weaver, founder of FaunaDB was sceptical about the technical debt implications when you build systems with an expectation of disposability.

“This model makes sense for a random walk through the market. If you actually have a strategy, commit to that strategy. There’s a difference between building optionality and building abstractions, too. Often conflated. In the end, Twitter barely made it, and product progress was slow for years due to post facto infrastructure investment.”

Given Evan worked at Twitter, and lived the pain points we’re talking about here, we have to give his views credence, but they also reflect the fact he is now building a database infrastructure product. I would argue the MVA pattern described stands however, and Randy Shoup, who also commented, certainly does.

“Great extension of the idea, James. I like to say that a new project in an enterprise is just a startup by another name :-). Same lessons. Assuming success from the start and architecting for it is a great way to prevent success from happening, in my experience. Almost every leading company you can think of started with a monolith that did not scale, and *when appropriate* rearchitected. eBay, Twitter, Amazon, Netflix to name a few. We’ve never heard of the competitors who architected the distributed system first. Guess why?”

James Nugent of Hashicorp also had some useful insights.

“Furthermore it is almost impossible to build a well structured SOA before the structures emerge from production-facing prototypes.”

We need to understand the patterns and underlying structure of the service calls- that is a really crucial aspect here. Where are the core structural bottlenecks? What are we optimising for, exactly? Starting with a complex set of distributed microservices likely doesn’t make sense, either though certainly in an an enterprise context building applications, rather than infrastructure. Scott Feinberg lays down some microservice antipatterns in this funny and insightful Monki Gras talk from 2016 about the value of monoliths for product market fit.

It also struck me it might be worth mentioning my maxim When Web Companies Grow Up They Turn into Java Shops, or more properly when web companies grow up they start using Java. The company starts by saying Java sucks, we’re going to use the current popular language of the day, runs into scaling issues, says Java still sucks, but the JVM has some value, so let’s use Scala. Then, overtime, needs to start growing the team, Scala skills are hard to find, and it turns out there are a lot more Java programmers out there, some with the relevant distributed systems expertise.

Good architecture should be emergent, and the skills you need to start a project are not necessarily the same ones you need to manage it at scale.

Oracle and MongoDB are both clients.