A Swing of the Pendulum: Are Fragmentation’s Days Numbered?

Share via Twitter Share via Facebook Share via Linkedin Share via Reddit

Foucault's Pendulum

One of the lessons that has stayed with me all these years removed from my History major is the pendulum theory. In short, it asserts that history typically moves within a pendulum’s arc: first swinging in one direction, then returning towards the other. I’ve been thinking about this quite a bit in recent months as the predictable result of widespread developer empowerment becomes more and more visible in virtually all of the metrics we track. Unsurprisingly, when you have two populations making decisions, the larger one leads to a wider array of outcomes. CIOs, as an example, were long content to consolidate on a limited number of runtimes – Java, .NET and a few others. All of the data we see, however, suggests that as the New Kingmakers have begun to rise up and act on their own initiative, the distribution of runtimes employed has exploded. The pendulum, quite obviously, had swung from centralized to fragmented, driven by a fundamental shift in the way that technologies were selected.
The question I’ve been pondering is simple: when does it begin to swing back in the other direction?

If there is any reversal here, it will come from developers. Even the large, CIO-centric incumbents are aware today that developers are in charge, so there’s no evidence to suggest that CIOs have a plausible strategy for putting developers back under thumb. But while over the last few years newly empowered developers have shown an insatiable appetite for new technologies, it hasn’t been clear that this trajectory was sustainable longer term.

Which is I’ve been paying attention, looking for evidence that the pendulum swing might be slowing – even reversing. The data is inconclusive. As Donnie has noted, there have only been five languages that really mattered on a volume basis on Github: JavaScript, Ruby, Java, PHP, and Python. And yet our rankings indicate that while they do indeed represent the fat part of the tail, there is substantial, ongoing volume usage of maybe twenty to thirty on top of that.

What the data won’t say, however, developers themselves will. Witness this piece from Tim Bray:

There is a re­al cost to this con­tin­u­ous widen­ing of the base of knowl­edge a de­vel­op­er has to have to re­main rel­e­van­t. One of today’s buz­zwords is “full-stack developer”. Which sounds good, but there’s a lit­tle guy in the back of my mind scream­ing “You mean I have to know Gra­dle in­ter­nals and ListView fail­ure modes and NSMan­agedOb­ject quirks and Em­ber con­tain­ers and the Ac­tor mod­el and what in­ter­face{} means in Go and Dock­er sup­port vari­a­tion in Cloud provider­s? Color me sus­pi­cious.

Which links to this piece by Ed Finkler:

My tolerance for learning curves grows smaller every day. New technologies, once exciting for the sake of newness, now seem like hassles. I’m less and less tolerant of hokey marketing filled with superlatives. I value stability and clarity.

Which elicited this reponse from Marco Arment:

I feel the same way, and it’s one of the reasons I’ve lost almost all interest in being a web developer. The client-side app world is much more stable, favoring deep knowledge of infrequent changes over the constant barrage of new, not necessarily better but at least different technologies, libraries, frameworks, techniques, and methodologies that burden professional web development.

Which in turn prompted a response from Matt Gemmell entitled “Confessions of an Ex-Developer”:

I’m glad there are no compilers (visible) in my life. I’m also glad that I can view the WWDC keynote as a tourist, without any approaching tension headache as I think about what I’ll need to add, or change, or remove. I can drift languidly along on the slow-moving current of the everyday web, indulging an old habit when a rainy evening comes by.

It’s a profoundly relaxing thing to be able to observe the technology industry without being invested in it. I’m glad I’m not making software anymore.

To be clear, these are merely four developers. Four experienced developers, more importantly. It may very well be that their experiences are nothing more than a natural and understandable change in priorities that comes with age.

But their experience seems to mirror a logical reaction to a very rapid set of transformations in this industry. Given the hypothesis that the furious rate of change and creation in technology will at some point hit a point of diminishing returns, then become actively counterproductive, it follows that these could merely be the bleeding edge of a more active backlash against complexity. Developers have historically had an insatiable appetite for new technology, but it could be that we’re approaching the too-much-of-a-good-thing stage. In which case, the logical outcome will be a gradual slowing of fragmentation followed by gradual consolidation. Market outcomes would be dependent on individual differences between rates of change, the negative impacts of fragmentation and so on.

It may be difficult to conceive of a return to a more simple environment, but remember that the Cambrian explosion the current rate of innovation is often compared to was itself very brief – in geologic terms, at least. Unnatural rates of change are by definition unnatural, and therefore difficult to sustain over time. It is doubtful that we’ll ever see a return to the radically more simple environment created by the early software giants, but it’s likely that we’ll see dramatically fewer popular options per category.

Whether we’re reaching apex of the swing towards fragmentation is debatable, less so is the fact that the pendulum will swing the other way eventually. It’s not a matter of if, but when.


  1. I have always been suspicious of the idea that developers are the new philosopher kings, because other people (like CIOs) typically pay their salaries and in the end value the investments that developers are making. As you say, there is a lot of innovation at the moment and it is still hard to evaluate the economics and pick winners. This is a common situation in unsettled markets, which is what we have today in a lot of areas of IT. That always gives the advantage to people who can adapt quickly to chaotic change. In this case that’s developers, i.e., labor. When the backlash comes it will be driven by employers. That’s usually how things have played out over history.

  2. […] Redmonk researcher Stephen O’Grady, reading a developer blog tea leaves, tentatively concurs: […]

  3. I think you’re seeing the impact of virtualization. If things are composed of small services running in a virtual machine that’s going to work and behave consistently without a traditional admin to babysit it, you can have one team using Java, another using Ruby and another using NodeJS, that’s perfectly viable.

    In other words, virtualization has increased the carrying-capacity of organizations for platforms.

  4. […] indeed are already starting to retrench in areas such as programming languages as Stephen O’Grady notes. These technologists are fast approaching a point of being unable to truly understand each […]

  5. […] カプラン=モスの考えを踏襲すれば、必要とされるためには超優秀にならなければいけないという思考の新しい開発者世代を抱えていることを私は懸念している。一つにはこれによって「フルスタック・エンジニア」を理想とするあり方が推進されているのだが、幸運にも、この現象は衰退を始めている。Redmonkのアナリスト、スティーブン・オグレディは以下のように述べている。 […]

  6. […] 除了这两个语言,前十的语言几乎是静态的。除了一些小的改变,事实上这些年都差不多是静态的。同时发现这是一定周期支持一种特定的语言或者是一种风格的语言,简单的来说,最受欢迎的语言几乎没什么改变,对未来也没什么倾向性的变化,这是不是意味着语言的采用和语言的分化已经达到了顶峰? […]

Leave a Reply

Your email address will not be published. Required fields are marked *