{"id":4955,"date":"2018-08-06T12:41:53","date_gmt":"2018-08-06T12:41:53","guid":{"rendered":"https:\/\/redmonk.com\/jgovernor\/?p=4955"},"modified":"2019-12-10T15:57:25","modified_gmt":"2019-12-10T15:57:25","slug":"towards-progressive-delivery","status":"publish","type":"post","link":"https:\/\/redmonk.com\/jgovernor\/towards-progressive-delivery\/","title":{"rendered":"Towards Progressive Delivery"},"content":{"rendered":"

\"Related<\/p>\n

At RedMonk we generally try to avoid coming up with new terms for technologies and trends – after all, where there is an available term in common use why not just adopt it?. The pragmatic approach means we often end up using terms that seem kind of silly (Ajax, noSQL, Serverless, Cloud even) , but it also means we avoid coming up with catchy little numbers like \u201chigh performance application platform as a service\u201d (hpaPaaS).<\/p>\n

The business of naming has changed a lot since we launched the firm, as the shape of the industry has. Tech today is a lot more playful than it was when it was driven by Enterprise Technology vendors. The Internet has changed everything, and it has certainly changed how names for things grow and spread. Developers are the new kingmakers. As one community complains vociferously that a new term is dumb, another is adopting and propagating it with glee.<\/p>\n

Which bring me to Progressive Delivery. I have been waiting for a term to emerge to describe a new basket of skills and technologies concerned with modern software development, testing and deployment. I am thinking of Canarying, Feature Flags, A\/B testing at scale. Advances in approaches to application and service Observability. On the technology side, Kubernetes and Istio bring new management challenges, but also opportunities – service mesh approaches can enable a lot more sophistication in routing of new application functions to particular user communities.<\/p>\n

What is Istio for? \u201cIstio lets you connect, secure, control, and observe services\u201d. But what\u2019s it for? At Google Next recently Istio was definitely falling into dessert topping and floor wax territory. But routing services and deployments, observing them, and rolling out changes to particular populations, sounds like a good basis for (another) revolution in software delivery. GitOps<\/a> -management by pull request, described by Weaveworks, is a related concept. Here is Stefan Prodan writing about GitOps Workflows for Istio Canary Deployments<\/a>.<\/p>\n

A couple of months ago I was talking to Sam Guckenheimer, product owner on the Visual Studio Team Services team. He was describing the Microsoft approach to testing, which considered the \u201cblast radius\u201d when delivery application changes across different communities – how many users would be affected, in what kinds of ways. Canaries were therefore policy-based – with a roll out to a particular subset of users, for example internal users in a particular geography first, before gradually widening the blast radius, eventually cutting over all users to the new version. It struck me that Sam was describing something useful, and when used the term \u201cprogressive experimentation\u201d I had a bit of an epiphany.<\/p>\n

Later that day, I casually slipped the term Progressive Delivery into a conversation with Adam Zimman, vp of product at LaunchDarkly, the feature flagging service, and it clicked. He really liked the idea, which is hardly surprising given LaunchDarkly is about the management of new features, where organisations can deploy subsets of functionality to some users, before gathering feedback and delivering them more widely. Adam has been working on a post about the idea as well – Progressive Delivery, a History\u2026. Condensed<\/a>.<\/p>\n

Continuous Integration\/Continuous Delivery has been extremely useful in taking us forward as an industry. CI\/CD is the basis of everything good in modern software development. But I do feel there are some new aspects and practices that we don\u2019t currently have a name for.<\/p>\n

Folks like Jez Humble have done an incredible job of pushing the state of the art forward, codifying Continuous Delivery<\/a>. Arguably some of the ideas I\u2019m considering when I think about Progressive Delivery have been around in CD thinking for a long time – Here\u2019s Martin Fowler on blue-green deployment<\/a>, writing in 2010. One of the interesting points in the post is the consideration in the post that the two target environments should be \u201cdifferent but as identical as possible\u201d.<\/p>\n

A great deal of our thinking in application delivery has been about consistency between development and deployment targets – see the promise of Docker and then Kubernetes. A core aspect of cattle vs pets as an analogy is that the cattle are all the same. The fleet is homogeneous.<\/p>\n

But we\u2019re moving into a multicloud, multiplatform, hybrid world, where deployment targets are vary and we may want to route deployment and make it more well, progressive.<\/p>\n

A recent post<\/a> by the Target engineering team is very interesting in this respect. Target runs Kubernetes in the cloud, but also in every retail store. It built a tool called Unimatrix to handle deployments.<\/p>\n

\u201cThe various store development teams have different requirements in terms of what stores they deploy to first and what capabilities they are targeting. So, the problem of enabling continuous delivery to the stores wasn\u2019t quite as simple as \u201cgive it to Unimatrix and a minute or two later you\u2019re deployed everywhere\u201d. We needed a way to differentiate stores with distinct capabilities so teams could build their pipelines specific to their needs.<\/p>\n

Within Unimatrix, we implemented a facade for Kubernetes namespaces as a way of grouping stores by their unique facets. Some of these groupings include things like the types of registers that are in the stores, the regional markets these stores are in, whether the store has a certain IoT capabilities, and so forth. To date, Unimatrix exposes 27 different \u201cnamespaces\u201d for the entire fleet, and teams can choose which group of stores they deploy to first, depending on what they\u2019re doing.\u201d<\/p>\n

I was talking to IBM Distinguished Engineer Jason McGee recently and he agreed that progressive delivery made sense. IBM runs tens of thousands of Kubernetes clusters for clients, geographically distributed, with quite different target configurations based on, for example, whether the cluster is running on prem our off.<\/p>\n

This detailed post<\/a> explains more about how IBM relies on feature flagging technology to manage its Kubernetes deployments.<\/p>\n

\u201cSo LaunchDarkly is actually what does the magic for us, which allows us to manage these thousands of deployments across many, many clusters. So for every service that we have, we have a feature flag. So in this case, I\u2019m going to pick on armada-billing because they actually do things the right way. And so armada-billing has a set of rules. And we\u2019ll scroll in here. So for every rule that you\u2019ll see, for example, we have a rule that says if the cluster name starts with dev, roll out this version. Or if the cluster name is stage of south roll out this version.<\/p>\n

And the cool thing about this is that since everything is decentralized and we don\u2019t have like a single central chain server, central deployment server pushing code out, we could theoretically push out new code to every single one of our clusters within 60 seconds because each of these Cluster Updaters is running independently. And every 60 seconds they\u2019re checking for updates. So if we wanted to we could actually push out new code within 60 seconds to every single one of our clusters. So we could update 1,500 deployments all at once. Should we? No. Could we? Yes. But what we\u2019ve chosen instead is kind of, and this is kind of what we settled on, is that we\u2019ll roll out by region.<\/p>\n

So well actually pick on the Australians first. We\u2019ll roll out stuff to Sydney. Make sure it doesn\u2019t break there first. That\u2019s kind of our canary test. And so we\u2019ll roll out stuff to Sydney. Make sure nothing breaks in the carts. This is after we\u2019ve already gone to our existing stage environment. But we do find is that, you know, there\u2019s no place like production home to test stuff in. Because we\u2019ll post off the stage. It works great. We\u2019ll put into pre-prod. It works great. And will push out a production. Bam. \u201cOh, we forgot about this or did this.\u201d So what this allows us to do is to just easily \u2026 We test stuff in Sydney. The cool thing about the process here is doesn\u2019t really \u2026 We can go either way. So we can go forward or backward releases.\u201d<\/p>\n

This architecture allows for more social autonomy across the distributed teams.<\/p>\n

\u201cWe kind of talked about the squad autonomy and letting them do their own thing. It\u2019s really up to the squads how they actually want to roll to code out. We, again, we\u2019re kind of setting best practices. \u201cLet\u2019s do it by region. Let\u2019s automatically push out the dev when the stuff is built.\u201d<\/p>\n

Other reasons people might want progressive deployment might be business policy or compliance-based. We might only want to roll out functionality to a particular geography because of regulations for example. This might not just be for testing reasons, but driven by business requirements. Progressive delivery potentially allows for a different discussion with the customer – taking us beyond simple alpha, beta and GA thinking.<\/p>\n

Some folks working on Progressive Delivery related technologies\u2013 IBM, Launch Darkly, Microsoft, and Target. Turbine Labs is also notable, building a platform for \u201cshaving the monolith\u201d, using Envoy as a service router:\u201cStand up new infrastructure, split traffic, and compare performance in one place.\u201d Observability is crucial in progressive delivery, and Honeycomb is doing really interesting work there.<\/p>\n

RedMonk has some history with the ideal of progressive changes. Back in 2008 we had our friends at Crowd Favorite build us a plugin for WordPress, which allows for progressive licensing<\/a>, where the license a piece of work is published becomes more permissive over time.<\/p>\n

In conclusion, it may be that the last thing the industry needs is a new term for software delivery. It\u2019s probably a bit mad to publish this the day before I go on holiday. My mentions may blow up with people telling me I am an idiot and just described Continuous Delivery. But it does seem we\u2019re unearthing a new set of problems and a new set of opportunities, and I don\u2019t feel like anyone has given it a name yet. So for now i will be using it, and seeing what kinds of reactions I get.<\/p>\n

 <\/p>\n

disclosure: IBM and Microsoft are both clients.<\/p>\n","protected":false},"excerpt":{"rendered":"

At RedMonk we generally try to avoid coming up with new terms for technologies and trends – after all, where there is an available term in common use why not just adopt it?. The pragmatic approach means we often end up using terms that seem kind of silly (Ajax, noSQL, Serverless, Cloud even) , but<\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false},"categories":[1],"tags":[455,548,464],"class_list":["post-4955","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-cicd","tag-istio","tag-kubernetes"],"jetpack_featured_media_url":"","jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p9wfjh-1hV","_links":{"self":[{"href":"https:\/\/redmonk.com\/jgovernor\/wp-json\/wp\/v2\/posts\/4955","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/redmonk.com\/jgovernor\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/redmonk.com\/jgovernor\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/redmonk.com\/jgovernor\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/redmonk.com\/jgovernor\/wp-json\/wp\/v2\/comments?post=4955"}],"version-history":[{"count":0,"href":"https:\/\/redmonk.com\/jgovernor\/wp-json\/wp\/v2\/posts\/4955\/revisions"}],"wp:attachment":[{"href":"https:\/\/redmonk.com\/jgovernor\/wp-json\/wp\/v2\/media?parent=4955"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/redmonk.com\/jgovernor\/wp-json\/wp\/v2\/categories?post=4955"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/redmonk.com\/jgovernor\/wp-json\/wp\/v2\/tags?post=4955"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}