So what even is a Service Mesh? Hot take on Istio and Linkerd

Remember that time, when we used to blithely use Docker and microservices interchangeably?

Docker has proved to be excellent in simplifying dev and test workflows, while Kubernetes has emerged as a strong alternative center of gravity for deployment of container-based apps. But it’s still early days in terms of production deployment of container based apps, especially of microservices-based ones apps.

What’s a service mesh? It’s the thing that focuses on solving all of those problems. Routing, rerouting for graceful degradation as services fail, secure inter service communications. Abstracting the networking and messaging in microservices based deployments.

You never want to be the only person that turns up at a party. If there is only one of something its pretty hard for people to understand as a market category.

With that in mind, the launch last week of ist.io is notable because it joins linkerd, another service mesh platform previously accepted by as a project by the Cloud Native Computing Foundation. And then there were two. Or three, counting Docker routing mesh.

Istio is described as

“an open platform to connect, manage, and secure microservices. Istio provides an easy way to create a network of deployed services with load balancing, service-to-service authentication, monitoring, and more, without requiring any changes in service code. You add Istio support to services by deploying a special sidecar proxy throughout your environment that intercepts all network communication between microservices, configured and managed using Istio’s control plane functionality”

Backers of the project are shall we say rather notable – IBM, Google and Lyft. At this point, unlike linkerd, istio is only based on production lessons from a single company – Lyft. IBM and Google are coming at the problem from a vendor platform perspective.

Istio reflects IBM increasing investments into the Kubernetes ecosystem. By working with Google it is making a statement about regaining some industry engineering leadership.

As IBM’s Jason McGee said:

“IBM needs an engineering opinion [on building microservices], and we worked with partners that show that.”

That said – one interesting tension likely to emerge is that while initially integration between Kubernetes and Istio is very crisp, there plans in future to support serverless, Cloud Foundry and even bare metal deployments. Pivotal certainly expects to see Cloud Foundry support in the near future. Dormain Drewitz’s post on istio is really good, breaking it down into 3 key benefits – security (TLS for service to service authentication, intelligent traffic management (proxy, deployed as a sidecar to the relevant service), and visibility (monitoring and tracing for troubleshooting and debugging)

So what about linkerd, which is led by a single vendor, Bouyant, building a classic VC-funded commercial open source business around the technology, based on lessons learned while the founders worked at Twitter. The project’s key design principle is simple – that in a microservices world failures are more often found in the interactions between services. This should not surprise us – it’s pretty much distributed computing 101. But the complexity of the microservices implementations at Web scale natives is staggering enough to require a new generation of dedicated tooling. Microservices is hard – as the originator of the term Martin Fowler makes clear in this post it is not for everybody, because of the overheads created in building and maintaining applications composed of independent services with strong module boundaries.

The chart of the Twitter topology above is one of favourite visualisations of the fact.

Linkerd is described as “a transparent proxy that adds service discovery, routing, failure handling, and visibility to modern software applications.”

“By providing a consistent, uniform layer of instrumentation and control across services, linkerd frees service owners to choose whichever language is most appropriate for their service… Linkerd takes care of the difficult, error-prone parts of cross-service communication—including latency-aware load balancing, connection pooling, TLS, instrumentation, and request-level routing.”

So a service mesh is in effect a domain specific router for microservices. It sits above TCP/IP and assumes Level 3 and 4 networking. Not surprisingly, William Morgan does a bang up job of explaining why you need one. Not surprising because Morgan is cofounder of Bouyant, which wants to win paying customers customers building synchronous transaction-based systems with requirements for minimal latency composed of microservices – likely to be fintech and telco initially.

Istio seems more agnostic about supporting asynchronous microservices patterns, which could become a point of distinction between the two platforms going forward. 

Service discovery and routing are two of the microservices questions that have yet to be comprehensively answered by either Docker Swarm or Kubernetes. In a distributed world services are by definition and design going to fail – so how do you route around them when they do? Linkerd is already used in production at Ticketmaster and the UK challenger bank Monzo (originally called Mondo). At least one other major cloud company is doing some very interesting refactoring around Linkered, but that’s NDA for now. Perhaps ironically the smallest vendor Bouyant seems to have the most service mesh deployments at scale.

So we now have a new center of gravity, a couple of interesting new projects to consider when looking at how to build, monitor and manage microservices based apps, alongside open source tools like Prometheus and vendors like Datadog and Weaveworks. Service mesh – it’s a thing now.

Docker, Google, IBM, and Pivotal are clients. Weaveworks is in my coworking space.