Industry Analysis: It’s About Audience

Share via Twitter Share via Facebook Share via Linkedin Share via Reddit

My own experience with analysts is mixed. Analysts tend to be great at predicting the past, but far less adept at predicting the future.” – Matt Asay

I submit that the above is true. Well, generally true. Analysts, as a rule, are better at predicting certain subsets of the past; the lack of reliable adoption and deployment metrics for open source, as an example, is a recurring problem for measurement and metrics firms. But it is more or less accurate to state that the traditional industry analyst industry is more comfortable by far predicting the past than the future. Who wants to author another “Windows NT will wipe out Linux” paper, after all?

I further submit that the above is true because that’s what the typical analyst customer wants. This is, as far as I’m concerned, nothing more than the obvious conclusion to be drawn from the financial performance of the market leaders. Matt goes on to assert that analyst customers are, in fact, looking for quite the opposite, saying “[the future] is actually what customers expect from them.” I do not agree.

It would be difficult, after all, to imagine that all of the industry giants were simultaneously failing to serve that same need, that same craving for predictions of the future. It stands to reason that, in the face of demand, one of them would emerge, differentiated on their ability to service these requests. That this has not occurred – though some might dispute this – tells me what I already believed: that analyst customers, particularly the larger ones, are thinking less about the future than the present, and less about the present than the past.

And, it must be said, less about the past than their jobs.

There has rarely been more pressure, after all, on IT departments. Budgets are being cut, resources being let go, enterprise pricing is going up, and on top of it all the technology landscape is increasingly fluid. The trio of open source, Software-as-a-Service and consumer technologies are putting IT staffers in the awkward position of explaining why they can’t deliver the same technology that their kids use to chat at home. In such an environment, is it any wonder that conservativism is the rule?

The problem, then, with traditional analyst coverage is one of audience – if indeed it’s a problem at all.

The audience for major industry analyst coverage wants to know not about what’s new, not about what’s exciting, not about what’s cutting edge, but rather what is safe. What will keep them in the CIO’s office longer than the oft-cited 18 months figure. Which the larger firms are, frankly, excellent at delivering. Not to mention monetizing.

And while the temptation might be to lament a status quo that favors a geological pace for technology development, such tears are entirely unnecessary. Because while the available evidence indicates that the powers that be within industry analyst customers are indeed looking backwards rather than forwards with respect to their analyst research, it also demonstrates quite effectively that the folks in the trenches are exceedingly effective at routing around such obstacles. With, or more likely without, the help of industry analysis and research.

As I’ve been telling audiences for years now – see this presentation from 2005, or this one from 2006 for examples – the power within the enterprise has shifted. As proof, I point you to the success of Linux, Apache, MySQL and PHP – even Skype or WordPress: technologies that were successful not because of analysts, or CIOs, or – with the odd exception – vendors, but rather because of the audience that did not and does not make deployment decisions based on analyst reports. Just what works.

rPath CEO Billy Marshall’s piece the “The CIO is the Last to Know” says it better than I can. But I still love telling the story of the CIOs and IT managers from broker/dealer firms I spoke with years ago, who told me that because they hadn’t issued instant messaging software none of their people were using it. The punchline, I’m sure, you can guess.

It is these on-the-ground, in-the-trenches audiences – who are neither the primary target audience for traditional research nor the ones funding it – that are the foundation of RedMonk. We listen, of course, to CIOs and vendors alike, but the bulk of our intelligence derives from the developer masses. These good folks tell us what works, and what does not. Which sometimes aligns with what we hear from vendors, and sometimes does not.

When we say “Analysis for the people, by the people,” we’re not trying to be cute…well, actually, yes. Yes we are. But it is at the same time a simple reflection of the truth as we see it, that we happen to be more forward looking because that’s where our readers – who double as our researchers – are pointed. The technology trends that I flatter myself to believe that we have correctly identified and championed well ahead of the curve – from the cloud to database developments to dynamic languages to mashups to REST to SaaS and so on – were not, themselves, RedMonk conclusions. They were the conclusions of our community; we merely researched and extrapolated from them.

None of this is to say that one approach or the other is entirely correct: it is, as always, a fundamental question of audience. CIOs ask for one type of research, and there are firms – many of them – that deliver on those expectations in fine fashion. Early adopters, on the other hand, ask for something quite different, and we and others try and deliver that. Whether you’re looking to forestall future criticism or read about the latest trends in web scale out, after all, you deserve research that suits your needs.

What’s more, a tension between the conservative and the radical is not only necessary in the context of technology adoption, but desirable.

If you’re looking for industry analysis, then, I would tell you to remember your elementary school History lessons: consider the source, and perhaps more importantly, the source’s source. Whether you’re conservative or bleeding edge, there is research available. You just have to know where to look.


  1. Frankly, I think it’s this way because it’s easier. The less conclusions people have to draw from data, the less they have to think and the less time they can spend on it. It seems to me more like an optimization of the level of detail in analysis than an optimization of its level of currency.

    I say this because it’s not just drawing conclusions about the future that’s hard — it’s equally hard to draw general conclusions from specific data. The two are parallel.

  2. A good post. I met with a CIO the other day and he does not understand technology. I think your analysis has highlighted to me the growing conservatism of on-premise IT people.

    In my opinion, what we need to do though is provide Analysis, with hard real tangible data, back to these on-premise guys, such that they understand and buy these off-premise services. This in turn will help to fund the whole “ahead of curve” ecosystem.

  3. @Donnie Berkholz: i think to some extent the Paradox of Choice concept is applicable, but at the same time the problem seems to me rooted more fundamentally in the way that corporations think, and outsource risk.

    it is indeed difficult to predict the future: impossibly so, in many cases. but often we’re not actually talking about the future, as William Gibson reminds, us, we’re talking about the present, which is far more predictable. if not provable to the satisfaction of conservative enterprises.

  4. @sogrady: Oh, that’s not the way I meant it when I was talking about fewer conclusions. What I meant was that shallow analysis was easier than deep, and people often don’t want anything beyond the shallow.

  5. SOG: you can’t have it both ways. You say CIOs look to backward-looking analysts to preserve their jobs, but then you also point out that their shelf-life hasn’t been helped by this strategy. Reading between the lines, you’re implying that CIOs are idiots for continuing with a strategy that demonstrably does not work. Analysts might be wise to continue bilking CIOs for such “analysis” but it’s not the sort of wisdom with which I’d be comfortable buying my groceries.

  6. @Matt Asay: well, the citation of the short-shelf life bit was actually intended to be somewhat tongue in cheek, as that figure is disputed. and it is true that i do not agree with a strictly or even primarily backward looking approach for CIOs, because that’s neither adaptive nor innovative.

    all of that said, i understand the fact that many CIOs are by circumstance forced to focus on the present, and that they’re often unable to embrace risk (read: innovation) because of an inability to sell it to a conservative CEO or board.

    more, i’m of the belief that the history provided is important, and also that balance is desirable. it is unfortunately the case that backward looking analysis currently dominates the research world, but optimally, technologists will have research available that indicates where things have been, where things are, and where things are going. meaning that we need all of the research, even if i don’t subscribe to some of it.

  7. […] O’Grady has an interesting post on his blog talking about analysts and their audience. A secondary point in his discussion, though, […]

  8. I wonder how much of the industry analysis that goes on is really intended for the benefit of the “end user” at all.

    Obviously this does not apply to the reputable sources of analysis like Redmonk. But aren’t there a heck of a lock of analysts that are merely actors that read a vendor’s script for the right price? It often seems like popularity contests where the analysts define the status quo of industry segments based on which contestants pay them the most money.

    From reports that seem to more prominently feature the vendors that have paid relationships, to the conferences held where only speaker submissions from paid clients are considered – to the analysts who rarely even talk to vendors that aren’t paying clients. It’s impossible to know just how pervasive all of these conditions are, but if you are a vendor who has held audience with a lot of industry analysts, you see a lot of subversive tactics and get cynical about it very quickly.

    More in line with the context of the original thread, I’ve always wondered why the analysts who make predictions don’t more publicly revisit those predictions down the line to demonstrate how true / untrue they wound up being. As annoying and elitist as I found The Industry Standard of old … I think their new content focus on predictions is actually quite innovative and should be embraced in similar fashion by industry analysts that make market forecasts. Then we could start to have more of a focus on the individual analysts who have great track records, and a more transparent way to analyze the analysts.

Leave a Reply

Your email address will not be published. Required fields are marked *