James Governor’s MonkChips: On Sample Sizes for Magic Quadrants
Its about time, I guess. Last Thursday I received an email, strangely enough from Gartner’s Vendor Relations department, asking me to remove a link from monkchips, because it apparently constituted “unauthorised use”.
So wait a second. I can’t post a URL to a Gartner site? What exactly would be the penalty for doing so?
Vendor relations? I am not a vendor.
Shouldn’t it be the responsibility of Gartner, and in this case SAS, to protect their intellectual property? I mean that’s what firewalls and registration mechanisms are for, isn’t it?
The irony is that I was going to follow up and say what good work Gartner had done, with a new and improved, and most importantly formally documented, methodology for the Magic Quadrant.
I can’t point you to the useful resource however. Well I can- it would be interesting to see Gartner find a way to punish me for publishing a url. But I think I am doing quite enough of Gartner’s marketing here already.
This isn’t even a Big Co pushes small co around story.. its really too silly for that. I am certainly not looking for any sympathy (although a couple of blog posts wouldn’t go amiss!). I do however suggest SAS and Gartner go and work it out between themselves. As of now the “issue” still remains – the “missing link” is still in the wild. Its really not my problem. Well done Gartner with the new transparency – its just a shame nobody can see it.
Whatever you do don’t go check out SAS.com for this information: who knows what rules you’ll break if you do.
In case you’re wondering – our library is free. Its not extensive, but it has zero barriers to participation. We want people to read our stuff and make their own contributions.
Ric says:
July 11, 2006 at 4:10 pm
Gartner are obviously worried that you’re providing too much competition at Redmonk – they feel threatened, perhaps?
You possibly touched a raw nerve with the comment about their tiny sample sizes … even Redmonk could afford DOZENS of survey participants!
It really is a little feeble.
catherine helzerman says:
July 11, 2006 at 5:27 pm
ahhh… you never forget your first cease and desist.
Nick Patience says:
July 11, 2006 at 5:50 pm
Are you going to publish the letter itself – the ultimate transparency? 😉
James Governor says:
July 11, 2006 at 5:59 pm
i dunno nick. normally i dont post emails online. you think i shd? the tone was snippy and superior…
Thomas Otter says:
July 11, 2006 at 6:00 pm
I use less and less Gartner stuff when talking to customers. The stuff is often really good, but it is such a mission to get hold of it and work out what I can and can’t distribute or quote, it just isn’t worth the effort.
ARonaut says:
July 11, 2006 at 6:08 pm
Gartner are bullies and need to be treated as such… Some of their analysts are okay though (we even know some nice ones) but the whole Borg culture is aggressive.
Nick Patience says:
July 11, 2006 at 6:17 pm
Well James, from a purely personal selfish point of view, I’d love you too, but I can understand why you wouldn’t want to!
/pd says:
July 11, 2006 at 6:23 pm
I think you should reprodcue the C&D as an image.
Warner Crocker says:
July 11, 2006 at 6:29 pm
I’d say post it.
Rob Stevens says:
July 11, 2006 at 7:32 pm
I can’t believe that there are still companies that act this way. Though it’s possible that this was simply an overzealous lawyer thinking he was acting in his client’s own best interest … but considering that a link is really nothing more than a free advertisement, I can’t understand why Gartner would be the least bit concerned about it. I’d kill for links to my company’s website. 🙂
I think I’ll find that information and link to it from my blog. I’m not afraid of a C&D.
mamund says:
July 11, 2006 at 7:49 pm
just reminds me what a pita gartner is. this is ‘pree-internet’ thinking from a company that puports to understand the ‘new media.’ this kinda thing smells of ‘jumping-the-shark’ time for gartner.
too bad, they have some really smart people there (had?)
Anthony Conrad says:
July 11, 2006 at 8:00 pm
I would post the letter and continue showing the links… freedom of the press!
Al says:
July 11, 2006 at 9:26 pm
“First they ignore you, then they laugh at you, then they fight you, then you win.”
😉
Keep up the good work
regards
Al
Edw says:
July 11, 2006 at 9:37 pm
I remember attending my first Gartner conference as a newly minted VP of Marketing. Gartner had never mentioned nor reviewed our company despite many years history and $30 million in licensing revenues. I dutifully set up time during their “free” analyst briefing hours and, before I could smile and offer my hand, the “analyst” handed me their vendor prospectus and began pitching me on the benefits of buying a strategic planning session with them to help us “position the company”.
Unfortunately we had only enough budget to sponsor one conslutant, and AMR had more sway with our client base. Plus their “analyst” preferred wine dinners to golf, which went over better with our execs… boy did we get some nice quotes from AMR. (but, we never did get listed in the Gartner magic quadrant.)
Ian Skerrett says:
July 11, 2006 at 9:59 pm
I think it will be fascinating to see if Gartner actually responds. It looks like your post is getting lots of links. Thanks to Gartner for giving you even more exposure.
stephen ogrady says:
July 11, 2006 at 10:02 pm
dude, post that sucker.
Mark Cathcart says:
July 11, 2006 at 10:26 pm
I wonder if they’ve written same to google, after all the links show up there as well…
http://www.google.com/search?&q=the+magic+quadrant+for+business+intelligence+platforms+site%3Agartner.com&btnG=Search
Dana Gardner says:
July 11, 2006 at 10:33 pm
Put the URL up for sale on eBay and see how little it’s really worth.
Jason Wood says:
July 11, 2006 at 10:50 pm
James,
Absolutely absurd; and just when it looked like Gartner was starting to get the whole blogging thing (or at least attempt to get it).
Best,
Jason
Curt Wehrley says:
July 12, 2006 at 12:25 am
I agree with Ric. A comment from another analyst firm about a small sample size could easily have been interpreted by Gartner as critical and threatening.
Shame, really, considering that you were about to praise them for their work. Why doesn’t the analyst who did the work have a blog? They could have linked to your post, thanked you for pointing out that their sample size got smaller, then explained why it did.
Speaking of sample size…As a statistician, I should point out to all that a smaller size isn’t a bad thing if the quality of your sample is good (a rational and random sample). I don’t know much about Gartner’s approach to sampling for that Magic Quadrant report, but I can make this general statement: Big sample sizes sound impressive, but they’re really not necessary or cost effective in many contexts. Remember why we use statistical analysis: to make inferences about a population based on a sample. Lowering your sample size is fine as long as you can live with the larger margin of error that comes with it.
Dennis Howlett says:
July 12, 2006 at 12:45 am
Piss hard enough and don’t be surprised to see the firehose coming back at you.
Stiennon says:
July 12, 2006 at 12:53 am
Posting C&D’s has become the norm. It is almost unfair though. A stodgy old fashioned company uses strong arm tactics of letters-from-attorneys only to have it blow up in their face with a PR fiasco.
I would do it.
-RS
Gary Slinger says:
July 12, 2006 at 2:10 am
The only value I ever got in access to Gartner’s site was the ability to find a contra-opinion to any “but Gartner says…” comment I got.
Post the C&D. It was sent to you. It’s your property. 1st Amendment and all that.
G.
john simonds says:
July 12, 2006 at 3:57 pm
You now know RedMonk has arrived when you can get this reaction. I think you have ushered in a new era. Reactions like this remind one of other times in history when a person/company tries to protect the old way and the newer ways and methods take over. Many times leaving them in the dust.
I used reaction for the old, yours was an ACTION, a huge difference in progressiveness.
Pamela Dingle says:
July 12, 2006 at 4:44 pm
If they didn’t want the information publicly discussed, then (a) why did the post the information on the internet in the first place, and (b) why didn’t they just move or remove the content you linked to? Sending a C&D seems like nothing more than a lame attempt to deny the reality that once such information is out, it can’t be taken back.
Pam
Ed says:
July 12, 2006 at 9:06 pm
I attended the Gartner Webservices conference in San Diego last month. What’s ironic about this C&D is that all throughout the conference, the Gartner folks are screaming “Times are changing, your company needs to open up the data, mashups and Web 2.0 will affect the enterprise”.
James – Please post the letter. I want to show my boss and everyone else I know in the company. They’ll get a blast out of it.
Dear Gartner,
Please practice what you preach. How do you expect customers to adopt such radical policies such as Web 2.0, Blogs, and Mashups when the Gartner Group won’t even allow links to their website. Please lead by example and not just lip service.
Best Regards,
A Disillusioned Conference Attendee
Duncan Chapple says:
July 13, 2006 at 12:29 am
Gartner has shown poor judgement here. The downside created for Gartner from their letter is much greater than the downside from James’ post, and it could get much worse. Despite it all, analysts’ methods are open to discussion; Gartner is no more likely than King Canute to instruct forces beyond its control.
Duncan.
James Governor says:
July 13, 2006 at 6:20 pm
thanks a lot for all the comments folks. it does seem not the most clueful move ever. stephen and cote say i should just publish the note. i may well do that when i get in tomorrow.
David Rossiter says:
July 13, 2006 at 7:52 pm
James, I have just spent part of the afternoon talking about this with an analyst I know. We looked at it every which way but cannot understand why Gartner would behave in this way. Given your position in the market, it’s remarkable. Gartner’s now probably had more visits to that page than it would ever have done before.
Ian Skerrett says:
July 13, 2006 at 9:21 pm
Please tell me that the comment from ‘Gartner’ is a joke.
catherine helzerman says:
July 13, 2006 at 11:14 pm
Two words: POST IT
Adam says:
July 14, 2006 at 1:39 am
That is pathetic on Gartner’s part. I think they can’t take a little constructive criticism, and that’s the real issue. As someone in the info security industry, I will be thinking a little harder from know on when reading their reports at work.
Tim Marman says:
July 14, 2006 at 3:58 am
I posted about this, including the link.
http://slashstar.com/blogs/tim/archive/2006/07/11/Gartner_sends_a_cease_and_desist_for_a_link.aspx
Gartner needs to get a clue. It doesn’t matter whether this is copyrighted – because a link is simply NOT a reproduction and thus not subject to their exclusionary rights under copyright.
I’m also assuming the above comment is a joke. If it’s not, well, then take the f’ing link down instead of sending out C&Ds. It scales better anyway.
stephen o'grady says:
July 14, 2006 at 4:20 am
ian: just heard from the author – it was a joke. an ill considered one, but a joke. i won’t delete the post, because it’s james’ blog, but i’ll ask him to.
James says:
July 14, 2006 at 11:40 am
Would it help if say your email from Gartner was accidentally forwarded to an enterprise architect blogger who immediately posted it and of course you sent him a cease and desist but it went ignored?
ARonaut says:
July 14, 2006 at 5:02 pm
Now, this thread is certainly getting some interest…
James, I’d love to see the C&D posted -this is the logical conclusion of your post. At least, pls send me a copy.
Re. the motive of the C&D, does anyone know if deep linking is illegal?
HughToppe says:
July 14, 2006 at 6:34 pm
I think this thing is amusing but I’d not make too much out of it. As a brouhaha, it won’t match the scale of the O’Reilly Web 2.0 faux pas (http://radar.oreilly.com/archives/2006/05/controversy_about_our_web_20_s.html)
Large successful companies, no matter whether it is Microsoft, IBM or Gartner (not that Gartner approaches the scale of those vendors) have many people lurking in cubicles in beady eyes with green eyeshades, who are completely absorbed in the very narrow scope mission they have been given. (This is one reason for the company’s original success, the ability to build a workforce that includes both beady-eyed and wide-eyed contributors).
So once in a while an over-zealous functionary steps into turf that is different from their day-to-day chores, and a misfire occurs. Happens. Unless I misread Gartner’s recent writings on Web2, I don’t think this is a strategic salvo, just a misfire (with hopefully no collateral damage to James). I hope non-beady-eyed people at Gartner are made aware of this, and bring clues to the clueless, and perhaps even an apology to James (one can dream). As Tim O’Reilly apologized for a similar misfire. Please keep us posted!
Andrew Smith says:
July 17, 2006 at 5:30 pm
What about:
http://www.hyperion.com/company/news/industry_analyst/