tecosystems

Responding to Duncan Chapple

Share via Twitter Share via Facebook Share via Linkedin Share via Reddit

I’m not quite sure what to make it of it – and maybe you all can help me parse this, but yesterday’s post on Duncan’s post seems to have struck quite a nerve. It’s not too often that I become embroiled in controversy, but Duncan Chapple, whom I’ve never spoken with before, is quite distressed with my usage and interpretation of his material and left a comment expressing this yesterday. I reproduce it here in full:

Stephen,

I don’t believe we have had any contact before, but I am surprised that you should plagarise, and misrepresent, my blog article about this research as you have. You have selected only the elements that fit your story, and distored my views and the data in your favour. This is terrible form for any blogger, but quite exceptional for one who aspires to be an influential analyst.

The chart you have copied belongs to HFN. You have not sought their permission, or even acknowledged that the research is theirs. It is certainly not a quality or influence ranking of analyst firms. It shows only the web traffic hitting websites of a sample of firms. The list does not claim to be exhaustive. In no way can this chart, nor my comments on my blog [www.analystequity.com], be used to conclude that the 13 firms are the most important. I have not said, as you suggest I have, that the twelve firms you cite are those that are more important than your firm. Our research shows that there are very many firms more influential than yours.

Readership is quite different from influence. It is self-evident that Ovum is more influential than Aberdeen; and IDC more influential than Jupiter – despite web traffic scores that point in opposite directions. Our surveys of CIOs show, for example, that free research meets the needs of technology buyers much less well.

Seeing the way in which your post has distored the research discussed, I can start to appreciate why some buyers consider free research to worth exactly what it costs.

Duncan.

Personally, I was taken aback, first because I didn’t believe that I’d misinterpreted his data to any significant degree, second because he seemed quite offended, and lastly on account of the fact that he leveled some fairly serious accusations at me. In the interest of giving his comments fair hearing, I thought I’d take a look at them in sequence and respond as best I can, then let you all judge for yourselves.

  1. I am surprised that you should plagarise“:
    In many industries, these words are probably not a big deal. Much of my work as a systems integrator, in fact, involved plagiarizing the deliverables of past projects so as to avoid reinventing the wheel. And indeed, it’s certainly true that even in the industry analysis business much of the analysis is recycled and repackaged from other materials. But in this case, when it involves allegations of plagiarization of a specific piece of research, I think it’s a very serious charge to be throwing around. And it’s one, frankly, that I’ve never faced before.

    So the question before me is: is it a fair assertion? I guess it depends on what you consider plagiarism. What I can be faulted for here, I’d agree, is not linking back to the original source of Duncan’s chart, which I’ll do here (it’s courtesy of HFN) and have already done in the original piece.

    But in no way, shape or form did I claim or imply that chart to be my own creation, or my data. Quite the opposite, in fact; I discussed instead how this data validated my own beliefs, which obviously implies that it’s not my own. Further, I linked to both Duncan and James’ pieces which discuss the source of the original chart. While I might not be the smartest tack in the box, it’s highly unlikely that if I intended to plagiarize something, I’d then go ahead and point you to the original source.

    Duncan’s charge of plagiarism, I’d thus contend, is unfounded.

  2. I am surprised that you should…misrepresent…my blog article about this research as you have. You have selected only the elements that fit your story, and distored my views and the data in your favour. This is terrible form for any blogger, but quite exceptional for one who aspires to be an influential analyst.”

    This assertion is less of a concern: I used the data to make a point of my own, and I don’t have a problem with that. But have I indeed misrepresented his views? Let’s review what I said that he said:

    If Duncan Chapple is to be believed, however, they [large enterprises] don’t care much about what folks like us have to say

    This assertion seems fairly defensible, given that I then quoted him directly from his piece as saying:

    Larger organisations tend to pay more attention to industry analysts than do smaller organisations, partly because they have larger budgets for buying technology. Furthermore, larger businesses pay most attention to the three or four most respected analyst firms.

    I’m guessing he wouldn’t argue that I’ve misrepresented him in that context. It’s instead, I suspect, my argument that his piece validates my long held assumption that the internet and our open content is helping us achieve visibility and authority in a fashion that was not possible previously. Here’s what I said:

    I’ve known for a while, then, that at least as far as the great big interweb is concerned, RedMonk is worth paying attention to. [1] What I didn’t know was whether or not that matter in the context of enterprise buyers. According to Duncan, however, it matters at least to the less conservative midmarket buyers. As his chart (inset) breaks it down, the 13 firms more important than RedMonk are…

    Over on James’ post, Duncan made the following comment:

    Please take care not to misrepresent this research. The notion that this shows “influence” is yours, and is not the interpretation that either I nor the researchers at HFN have of the data.

    His assertion is that James and I are implying a conflation of traffic and analyst influence that he did not intend; indeed, that we’re deliberately misrepresenting his words in that respect. But let’s look at the title to his piece:

    “Internet underpins analyst influence on mid-market.”

    Does that not imply something of a conflation? He went on to say in the piece:

    Last week I saw some research by HFN which assesses the importance of IT research companies by making use of publicly available data…

    .. This analysis is based on the visits to their external websites. Since almost all client access of research is online, and since the research of many non-clients also runs through analysts’ websites, this measure is an excellent start to any analysis.

    Given these two pieces of information, is it unreasonable for me to infer some importance to the numbers in terms of influecne, and a correlation between the two? I don’t think so, nor do I believe that I’ve “distored [his] views and the data in [my] favour.”

  3. The chart you have copied belongs to HFN. You have not sought their permission, or even acknowledged that the research is theirs.”
    Acknowledged above, and fixed. If you prefer, I’ll embed a direct link to your website so as not to copy it directly (James stored it in Flickr for the sake of convenience), and if HFN has any objections they should feel free to contact me ([email protected] / 866.733.6665) and we’ll take it down.

  4. [The chart] is certainly not a quality or influence ranking of analyst firms. It shows only the web traffic hitting websites of a sample of firms. It shows only the web traffic hitting websites of a sample of firms.
    I’d agree that it’s absolutely not a measure of quality, and didn’t make that assertion. In fact, if you do a quick scan on the page in question, you’ll find that the first appearance of the word “quality” is in the comments, as it’s mentioned nowhere in the piece. In terms of influence, it depends on how you define that. As James noted in the comments over on his blog, RedMonk would no doubt do poorly in a report measuring CIO influence, but that’s something we readily admit. I’m far more concerned about our reputation with developers and architects than I am with CIOs, for better or worse. Internet traffic is one way to gain insight into that.

  5. In no way can this chart, nor my comments on my blog [www.analystequity.com], be used to conclude that the 13 firms are the most important. I have not said, as you suggest I have, that the twelve firms you cite are those that are more important than your firm.”
    In terms of my previous post, I was reflecting on the available data – those included in the chart, and my remarks should be viewed in that context. Of the 38 firms on the sample, some are ranked higher than others – that’s not in dispute. If one uses the web traffic lens as a metric to gauge importance, my statements are pretty straightforward. Is this the once and future ranking of importance? Nope. But that wasn’t what I was arguing.

  6. Our research shows that there are very many firms more influential than yours.”
    This strikes me as fairly obvious. Let me quote from yesterday: “let me remind those of you following along at home that we are three people. Three.” Of course there are firms more influential than RedMonk (though not, I’d argue, as fun to work with ;).

  7. Readership is quite different from influence.
    Here’s how I qualified my remarks yesterday: “It’s important to note, of course, that not only do we not know the methodology, this graph is based on Alexa data which is an approximation.” That, I take it, was not enough.

  8. It is self-evident that Ovum is more influential than Aberdeen; and IDC more influential than Jupiter – despite web traffic scores that point in opposite directions.”
    I would not presume to know how the other analyst firms are perceived, but I am not surprised that traffic is not a direct 1:1 correlation to influence. That said, I don’t believe it’s irrelevant, either.

  9. Our surveys of CIOs show, for example, that free research meets the needs of technology buyers much less well.”
    It also surprises me not at all that CIOs prefer paid research; these are, after all, the folks paying big money to Forrester, Gartner, IDC, et al. What might be more of a surprise is that I’m not particularly concerned about that fact; we’ve found plenty of enterprise architects and developers that appreciate our free research, and that’s enough for me.

  10. Seeing the way in which your post has distored the research discussed, I can start to appreciate why some buyers consider free research to worth exactly what it costs.”
    It seems silly – not to mention petty – to judge my free research by one post, but it’s absurd to make judgements on the overall idea of free research on the basis of a single post by a single analyst.

Ultimately, I’m still not sure just what Duncan’s complaint is. My post, boiled down, essentially amounted to:

1. HFN’s numbers indicate RedMonk is above the middle of the pack in traffic
2. Traffic – IMO, anyway – has a relationship to influence (e.g. it’s difficult to influence someone if they don’t read you)
3. Add 1 & 2, and RedMonk’s more influential than our size would lead one to believe

And despite his comments, Duncan clearly does perceive some relationship between the two – here’s how he ended his piece:

These data show how prepared people are to keep on hunting for information online. Many users will only look at Gartner, the biggest IT research company, Forrester, Jupiter Research and IDC. However, in the dozens of mid-sized analysts firms under them, the race gets very varied. There are almost two dozen firms with scores that are at least two-thirds of Gartner’s score.

HFN’s data show that the Internet is a great leveller. Only Gartner and Forrester have started to really turn their influence into shareholder value. However, it is clear that there are many other analyst firms that have a substantial readership, which is heavily concentrated in the mid-market.

How else could the Internet be a “great leveller” if it didn’t accord any influence? Thanks much.

In any event, I’m curious as to what all of you think of this? Am I offbase? Did I plagiarize or grossly misinterpret Duncan’s post? For those of you who read to the end, I’d be grateful for your impressions – whether you agree or disagree with what I’ve written here. Thanks much.

P.S. One thing I’d ask Duncan, in case he reads this: would you be comfortable disclosing which of the parties on HFN’s charts are your clients, as is RedMonk’s policy when we discuss software?

15 comments

  1. A Whole Bunch of Little Stuff

    Today’s proved to be an exciting day so far: I caught up with an old friend from the Mesa/FX days, John Arley Burns. It turns out he’s had a lot of experience in his current job testing out open source…

  2. Definition of plagiarism “To steal and pass off(the ideas or words of another)as one’s own without crediting the source.” From an academic or legal sense your Blog didn’t do that. Is correct spelling a requirement in Blogs-if not it should be. SOGSR

  3. My first thought is unprintable.

    My second thought is that it is dangerous to write one’s first thoughts in such a public forum. Perhaps this is what Duncan did here, we’ll have to wait and see what his response is.

    My third thought is that this requires more thought.

  4. Hi Stephen,

    James told me that you’d posted this. Just to follow up my email to you, let me just answer your question. Lighthouse has no clients that are analyst firms, nor do we prospect for business amongst them. In the past we have sold one-off tabulations of our pre-existing data to two analyst firms, both of whom approached us. We do so for a token fee: this amounts far less than one percent of our revenue. We do not have their permission to cite them as clients. If you agree to hold their names in confidence, just call me and I will tell you.

    I’ve updated my post with a small clarification and, as I have mentioned, will write more later.

    Thanks for the question,

    Duncan.

  5. Hi Stuart,

    It’s fascinating what some people get upset ovet, from what I gather there doesn’t seem to be any foundation in Duncan’s complaints. It’ll be interesting to read his reply to you clarifications.

    Keep up the good work!

    regards,
    martin

  6. Stuart? What the [expletive] was I thinking? Stephen, of course. Sorry!

    Have I been reading your blog too much when I get you and your father’s name mixed? 🙂

    regards,
    martin

  7. they are all “soggy”…

    happy birthday for the other day pops!

  8. Holy Moly –who p-ssed in Duncan’s Cheeros?!

  9. Hi Stephen,

    Okay, time for me to eat some humble pie. Thanks for your email in reply to mine, and for James’ passionate call earlier today. I reacted wrongly, and can only apologise.

    As Jon speculated, this is what can happen when we jump to conclusions. As the risk of reopening wounds, let me explain my comments. I was alarmed by the original post, especially after clients of mine assumed that this was a ranking of mine. The post also seemed to mistakenly attribute to me the notions that RedMonk matters to midmarket buyers, that midmarket buyers are less conservative, that the 13 firms on the chart are those more important than RedMonk and that the chart is mine. In particular, presenting HFN’s work unacknowledged seemed bad form, and presenting it as mine.was inaccurate. I called this ‘plagiarism by proxy’ in an email to James yesterday, in that HFN’s work was presented as mine. Of course I now see that my use of that word was incomprehensible and too harsh – especially since you have corrected that omission.

    Before I take the time to clarify my views on analystequity.com, I wanted first to make a public apology and, by explaining myself, hope that you accept the apology I emailed to you.

    Best,

    Duncan.

  10. Hey, someone that says he was in error and apologises?!?! This is becomming hard to find this days, …

  11. One thing bothers me about Duncan’s reply on analystrelations.blogspot.com. It’s the looseness of language that refers to scooping a double handful of metrics out of Alexa as _research_, referring to a simple monotonic operation on those numbers as a _score_, categorizing the top level of traffic as _exceptional_, and so forth.

    Dressing up simple, analytically suspect work in fancy terminology is exactly what makes folks suspicious of industry analysts. Doubly so as mashup culture allows us to do these sorts of views before breakfast, in real time, across all industries, and so forth.

  12. Hi Chris,

    Actually, the chart and the labels on the chart are HFN’s, not mine. Personally, I don’t find ‘score’ to be especially fancy terminology and your alternative (monotonic operation) can perhaps be bettered. I am sure they are open to suggestions. ‘Number’ seems good.

    However, on the activity itself, I think we will disagree. I think these data are of interest. HFN found then, tabulated them, wrote an analysis of it and held a conference call with leading German AR professionals to discuss them — and to discuss the weaknesses in them and to set them in context. That seems quite worthy of the word research. If you are unfamiliar with the use of score as a word for a number that has been operated on (z-score, t-score) then I can see that you might confuse the word score for the sort of score you get in a football competiton. HFN’s analysis was pretty clear about what the number are and are not. I also tried to make that point and, indeed, reacted strongly when I thought the data were made out to be more than they were.

    I would be open to suggestions on alternative methods to process these data. However, I can see much to support the idea that we are better off not knowing and showing these data, with all the caveats and discussion that we surrounded them with.

    Duncan.

  13. Steve- I applaud you and James for the ‘openness’ of this conversation with Duncan. I imagine many people would have thought twice about publishing his initial comment, the opening salvo as it were. Shows true integrity, critical to influence. -brenda

  14. Ooops. Of course, I meant to say:
    “I **can’t** see much to support the idea that we are better off not knowing and showing these data”

  15. all: i appreciate and have accepted Duncan’s apology here, but in the meantime i just wanted to say thank you to all of the individual commenters here for weighing in with their opinions. it was much appreciated.

Leave a Reply to pops Cancel reply

Your email address will not be published. Required fields are marked *