I’m not quite sure what to make it of it – and maybe you all can help me parse this, but yesterday’s post on Duncan’s post seems to have struck quite a nerve. It’s not too often that I become embroiled in controversy, but Duncan Chapple, whom I’ve never spoken with before, is quite distressed with my usage and interpretation of his material and left a comment expressing this yesterday. I reproduce it here in full:
Stephen,
I don’t believe we have had any contact before, but I am surprised that you should plagarise, and misrepresent, my blog article about this research as you have. You have selected only the elements that fit your story, and distored my views and the data in your favour. This is terrible form for any blogger, but quite exceptional for one who aspires to be an influential analyst.
The chart you have copied belongs to HFN. You have not sought their permission, or even acknowledged that the research is theirs. It is certainly not a quality or influence ranking of analyst firms. It shows only the web traffic hitting websites of a sample of firms. The list does not claim to be exhaustive. In no way can this chart, nor my comments on my blog [www.analystequity.com], be used to conclude that the 13 firms are the most important. I have not said, as you suggest I have, that the twelve firms you cite are those that are more important than your firm. Our research shows that there are very many firms more influential than yours.
Readership is quite different from influence. It is self-evident that Ovum is more influential than Aberdeen; and IDC more influential than Jupiter – despite web traffic scores that point in opposite directions. Our surveys of CIOs show, for example, that free research meets the needs of technology buyers much less well.
Seeing the way in which your post has distored the research discussed, I can start to appreciate why some buyers consider free research to worth exactly what it costs.
Duncan.
Personally, I was taken aback, first because I didn’t believe that I’d misinterpreted his data to any significant degree, second because he seemed quite offended, and lastly on account of the fact that he leveled some fairly serious accusations at me. In the interest of giving his comments fair hearing, I thought I’d take a look at them in sequence and respond as best I can, then let you all judge for yourselves.
- “I am surprised that you should plagarise“:
In many industries, these words are probably not a big deal. Much of my work as a systems integrator, in fact, involved plagiarizing the deliverables of past projects so as to avoid reinventing the wheel. And indeed, it’s certainly true that even in the industry analysis business much of the analysis is recycled and repackaged from other materials. But in this case, when it involves allegations of plagiarization of a specific piece of research, I think it’s a very serious charge to be throwing around. And it’s one, frankly, that I’ve never faced before.So the question before me is: is it a fair assertion? I guess it depends on what you consider plagiarism. What I can be faulted for here, I’d agree, is not linking back to the original source of Duncan’s chart, which I’ll do here (it’s courtesy of HFN) and have already done in the original piece.
But in no way, shape or form did I claim or imply that chart to be my own creation, or my data. Quite the opposite, in fact; I discussed instead how this data validated my own beliefs, which obviously implies that it’s not my own. Further, I linked to both Duncan and James’ pieces which discuss the source of the original chart. While I might not be the smartest tack in the box, it’s highly unlikely that if I intended to plagiarize something, I’d then go ahead and point you to the original source.
Duncan’s charge of plagiarism, I’d thus contend, is unfounded.
- “I am surprised that you should…misrepresent…my blog article about this research as you have. You have selected only the elements that fit your story, and distored my views and the data in your favour. This is terrible form for any blogger, but quite exceptional for one who aspires to be an influential analyst.”
This assertion is less of a concern: I used the data to make a point of my own, and I don’t have a problem with that. But have I indeed misrepresented his views? Let’s review what I said that he said:
If Duncan Chapple is to be believed, however, they [large enterprises] don’t care much about what folks like us have to say
This assertion seems fairly defensible, given that I then quoted him directly from his piece as saying:
Larger organisations tend to pay more attention to industry analysts than do smaller organisations, partly because they have larger budgets for buying technology. Furthermore, larger businesses pay most attention to the three or four most respected analyst firms.
I’m guessing he wouldn’t argue that I’ve misrepresented him in that context. It’s instead, I suspect, my argument that his piece validates my long held assumption that the internet and our open content is helping us achieve visibility and authority in a fashion that was not possible previously. Here’s what I said:
I’ve known for a while, then, that at least as far as the great big interweb is concerned, RedMonk is worth paying attention to. [1] What I didn’t know was whether or not that matter in the context of enterprise buyers. According to Duncan, however, it matters at least to the less conservative midmarket buyers. As his chart (inset) breaks it down, the 13 firms more important than RedMonk are…
Over on James’ post, Duncan made the following comment:
Please take care not to misrepresent this research. The notion that this shows “influence” is yours, and is not the interpretation that either I nor the researchers at HFN have of the data.
His assertion is that James and I are implying a conflation of traffic and analyst influence that he did not intend; indeed, that we’re deliberately misrepresenting his words in that respect. But let’s look at the title to his piece:
“Internet underpins analyst influence on mid-market.”
Does that not imply something of a conflation? He went on to say in the piece:
Last week I saw some research by HFN which assesses the importance of IT research companies by making use of publicly available data…
.. This analysis is based on the visits to their external websites. Since almost all client access of research is online, and since the research of many non-clients also runs through analysts’ websites, this measure is an excellent start to any analysis.
Given these two pieces of information, is it unreasonable for me to infer some importance to the numbers in terms of influecne, and a correlation between the two? I don’t think so, nor do I believe that I’ve “distored [his] views and the data in [my] favour.”
- “The chart you have copied belongs to HFN. You have not sought their permission, or even acknowledged that the research is theirs.”
Acknowledged above, and fixed. If you prefer, I’ll embed a direct link to your website so as not to copy it directly (James stored it in Flickr for the sake of convenience), and if HFN has any objections they should feel free to contact me ([email protected] / 866.733.6665) and we’ll take it down. - “[The chart] is certainly not a quality or influence ranking of analyst firms. It shows only the web traffic hitting websites of a sample of firms. It shows only the web traffic hitting websites of a sample of firms. ”
I’d agree that it’s absolutely not a measure of quality, and didn’t make that assertion. In fact, if you do a quick scan on the page in question, you’ll find that the first appearance of the word “quality” is in the comments, as it’s mentioned nowhere in the piece. In terms of influence, it depends on how you define that. As James noted in the comments over on his blog, RedMonk would no doubt do poorly in a report measuring CIO influence, but that’s something we readily admit. I’m far more concerned about our reputation with developers and architects than I am with CIOs, for better or worse. Internet traffic is one way to gain insight into that. - “In no way can this chart, nor my comments on my blog [www.analystequity.com], be used to conclude that the 13 firms are the most important. I have not said, as you suggest I have, that the twelve firms you cite are those that are more important than your firm.”
In terms of my previous post, I was reflecting on the available data – those included in the chart, and my remarks should be viewed in that context. Of the 38 firms on the sample, some are ranked higher than others – that’s not in dispute. If one uses the web traffic lens as a metric to gauge importance, my statements are pretty straightforward. Is this the once and future ranking of importance? Nope. But that wasn’t what I was arguing. - “Our research shows that there are very many firms more influential than yours.”
This strikes me as fairly obvious. Let me quote from yesterday: “let me remind those of you following along at home that we are three people. Three.” Of course there are firms more influential than RedMonk (though not, I’d argue, as fun to work with ;). - “Readership is quite different from influence.”
Here’s how I qualified my remarks yesterday: “It’s important to note, of course, that not only do we not know the methodology, this graph is based on Alexa data which is an approximation.” That, I take it, was not enough. - “It is self-evident that Ovum is more influential than Aberdeen; and IDC more influential than Jupiter – despite web traffic scores that point in opposite directions.”
I would not presume to know how the other analyst firms are perceived, but I am not surprised that traffic is not a direct 1:1 correlation to influence. That said, I don’t believe it’s irrelevant, either. - “Our surveys of CIOs show, for example, that free research meets the needs of technology buyers much less well.”
It also surprises me not at all that CIOs prefer paid research; these are, after all, the folks paying big money to Forrester, Gartner, IDC, et al. What might be more of a surprise is that I’m not particularly concerned about that fact; we’ve found plenty of enterprise architects and developers that appreciate our free research, and that’s enough for me. - “Seeing the way in which your post has distored the research discussed, I can start to appreciate why some buyers consider free research to worth exactly what it costs.”
It seems silly – not to mention petty – to judge my free research by one post, but it’s absurd to make judgements on the overall idea of free research on the basis of a single post by a single analyst.
Ultimately, I’m still not sure just what Duncan’s complaint is. My post, boiled down, essentially amounted to:
1. HFN’s numbers indicate RedMonk is above the middle of the pack in traffic
2. Traffic – IMO, anyway – has a relationship to influence (e.g. it’s difficult to influence someone if they don’t read you)
3. Add 1 & 2, and RedMonk’s more influential than our size would lead one to believe
And despite his comments, Duncan clearly does perceive some relationship between the two – here’s how he ended his piece:
These data show how prepared people are to keep on hunting for information online. Many users will only look at Gartner, the biggest IT research company, Forrester, Jupiter Research and IDC. However, in the dozens of mid-sized analysts firms under them, the race gets very varied. There are almost two dozen firms with scores that are at least two-thirds of Gartner’s score.
HFN’s data show that the Internet is a great leveller. Only Gartner and Forrester have started to really turn their influence into shareholder value. However, it is clear that there are many other analyst firms that have a substantial readership, which is heavily concentrated in the mid-market.
How else could the Internet be a “great leveller” if it didn’t accord any influence? Thanks much.
In any event, I’m curious as to what all of you think of this? Am I offbase? Did I plagiarize or grossly misinterpret Duncan’s post? For those of you who read to the end, I’d be grateful for your impressions – whether you agree or disagree with what I’ve written here. Thanks much.
P.S. One thing I’d ask Duncan, in case he reads this: would you be comfortable disclosing which of the parties on HFN’s charts are your clients, as is RedMonk’s policy when we discuss software?
