Openness as a Function of Incentive

Share via Twitter Share via Facebook Share via Linkedin Share via Reddit

Why do corporations contribute to open source? What drives open standards? Both of these questions have spawned hours and hours of debate from academia to the blogosphere to the board room. As is obvious to most rational individuals (there are exceptions, as always), the answers are both complex and nuanced, rendering simple answers impossible. Yet I think the key to understanding both is to understand openness as a function of incentive.

For example, let’s take last week’s Office Open XML Formats announcement. As I discussed, I can understand Microsoft’s decision to pursue its own course even if it’s not what I think is in their best interests (and certainly isn’t in their customers). Others, like IBM’s Bob Sutor and Sun’s Simon Phipps were less sanguine. Bob opened his take with the following:

To me, that document formats for “office” applications should be completely open, not hindered by patents, and not owned by a single vendor is just obvious. [emphasis Bob’s]

Simon was similarly adamant:

An open standard is one which, when it changes, no-one is surprised by the changes. Admittedly I’m not surprised when Microsoft repeatedly and apparently arbitrarily changes its interfaces and formats and jerks developers around but I meant “not surprised” in the sense that the change process was open to involvement and contribution by all, not in that way.

For what it’s worth, I agree with both of them. It is obvious that a format should be open and owned by no single vendor. At least, it’s obvious to me, Bob, Simon, lots of folks external to Microsoft and likely the EU. I doubt very much however if it’s all that obvious to Microsoft, and the problem is one of incentive. Scoble tacitly admits this in the video below when he says that this is a whole new Microsoft (though I don’t buy that either; not when the only products mentioned in the interview are Microsoft ones and the Open Document Format isn’t discussed).

As Jonathan Schwartz often says – and I’ve agreed here any number of times – open standards favor the laggards. For all its promise, the Open Document Format is clearly playing that role at the moment, as Microsoft’s Office formats are far and away the dominant players on the planet. Because of this, their incentive to participate in anything that jeopardizes that position is low. While Brian Jones may claim (WMV video link) that the folks from Redmond “were never trying to keep the binary formats locked down, it was just that they were not designed to be easily accessible by other people,” I’m not buying that. There’s some truth there, sure, but could Microsoft have been more open? Absolutely. Why haven’t they? No incentive.

The simple fact is that like any massive, publically held business Microsoft is going to do what’s best for Microsoft; sometimes that will benefit customers, sometimes it won’t. While the benefits of open standards generally and the Open Document Format specifically may be obvious externally, they’re simply not to a division that’s generated absurd margins for several decades.

This simple product decision is hardly the only decision attributable to incentive. Whether it’s Novell buying SuSE and Ximian, IBM investing a billion dollars in Linux, Sun open sourcing Solaris, or MySQL/Sleeypcat/db4objects pursuing an open source centric model, it all comes back to incentive. The incentive might be competitive threats, ammortized development costs, or the opportunity to build marketshare, but they’re all about incentive. Obvious, yes, I know. Yet worth repeating. On a corporate level, decisions around openness often (though not always) come down to a simple cost/benefit equation, much to the chagrin of more altruistic community contributors everywhere. In a perfect world firms would put customer’s needs and wants before their own; wake me up when we get there.

Whatever the open project or standard, I think it’s important to understand the motivations of both potential participants as well as competitors. The shortest distance to achieving victory for openness is via incentive. You want to encourage openness and participation? Find the right carrot, or, if need be, the right stick. Incentive can be manufactured.


  1. I definitely agree with the conclusion – "Qui Bono?" should be what analysts ask about decisions for or against open source, open standards, etc. Rhetoric aside, none of these big companies are doing what they do out of idealistic motives, but out of shrewd calculations of costs and benefits.

    A couple of points you probably don't agree with. First, for most users, forget the incentives and the process, standard is as standard does: It's a "standard" if you can count on it being supported at the other end. MS Word binary format is in that sense a "standard" – One can send essentially anyone a Word document with confidence that they can read it. Bob Sutor used LaTeX as an example of a truly open format that he can surely read forever. Fair enough, but if I sent a random person on the 'net a Word file they could read it, and only geeks would know how to begin to handle a LaTeX document. Which is more "standard" in any pragmatic sense?

    Second, being designed by a committee might make a spec more "open" but it doesn't make it good. It's not at all obvious to me that customers' needs are best served by "open" (in the sense of design-by-committee) specs as opposed to the "open" (in the sense of well-proven, publicly documented, royalty-free …) specs that MS is planning for Office 12. Maybe the "obvious" thing isn't actually the best for anyone. Consider the case of the W3C XML Schema Definition Language. At the beginning of that effort MS submitted its XML Data Reduced schema spec (used in a number of actual products) to W3C but committed to deprecate it in favor of whatever W3C came up with. W3C assembled a group of 50 or so genuine experts , considered several other proposals, drafted and quarreled and compromised for several years, and came up with XSD 1.0 and launched it with a lot of fanfare. THEN the world sat down to implement it and make it work. 4 years later W3C are having a workshop to determine what people think about it, and from what I've seen the campers are NOT happy. This happens over and over when judgments and compromises in committee rooms take precedence over natural selection in technology and marketplaces to shape technical specifications. XSD is finally getting to the point where it can work and interoperate, but it has been painful for all involved. I doubt if many people at MS want to repeat the experience of ceding control to an industry committee.

    Third, it's worth considering the basic value of XML and assess whether the OASIS approach and/or the MS Office approach really deliver that value. I see XML as being for *data* interoperability between applications. It is important that MS Office, OpenOffice, etc. can exchange documents, and the well-documented specifications of their data formats allows third parties to define the translations even if MS or Sun do not. It's not at all obvious that it is important for MS Office, OpenOffice, and other applications all use the same native format (thus constraining their architectures, features, performance, etc.). Both these efforts get their basic data interoperability from their XML heritage and a commitment to document what the markup really means. Efforts to force them to a lowest common denominator format adds no value that I can see for either MS or its customers.

    Finally, on a somewhat snarky note, I see that the IBM and Sun evangelists are asking people to lobby their governments to *mandate* the OASIS format; I for one ask people to choose whichever products/formats best meet their needs and budgets. There's no doubt that the OO/OASIS combination will meet a lot of people's needs, and that's fine. There might even be some people who truly value the fact that it is owned by a consortium rather than a single company, and that's fine too. MS just doesn't think it is sufficiently powerful and compatible enough to meet the evolving needs of the MS Office customer base, and that's who MS really does have to keep happy.

  2. hey mike, thanks for dropping by. as you predicted, i see things a bit differently 😉

    1. what you've essentially just outlined is the difference b/twn an open standard and a standard. i'll readily concede the point that Office is a de facto standard, and i'd imagine any of the Open Document Format folks would as well. the benefits to an open standard are quite different, however, than a de facto standard, primarily in their longevity. as we transfer more and more assets to digital forms, it's crucial that their lifespan meet, at a minimum, the physical asset that they replace. for people with those concerns, open standards are probably the better bet simply b/c they are a.) documented – as is the new MS equivalent – but b.) not subject to the whims and control of a single vendor.

    second, i actually pretty well aligned on this point. design by committee is not the ideal process, in my opinion, but two points to consider: a.) there are times its necessary, and b.) how many people have input into the MS format? more than a few, i'd guess 😉

    third, i see your point but do disagree. as i understand your argument (and correct me if i got it wrong) there are two parts: 1. XML permits a level of interoperation and transformation by its very nature, and therefore 2. it's not hugely important that MS and the Open Document Format coalesce into a single standard. my response would be that just because the underlying technology enables such transformations and interoperability doesn't mean that a.) it will be easy to implement or b.) we'll see widespread implementation. having two formats to work with, in my view, is just another barrier to entry to the next generation of Office document processing layer applications Microsoft and others have been contemplating for a while.

    lastly, i agree that mandating a format for the sake of mandating a format is silly. i don't, however, believe that's the case here. i think the Open Document Format has enough people on board (and will likely bring more) to be increasingly relevant to customers all of over the world, MS customers included.

    so we disagree more than we agree, but thanks for the thoughtful response. i'll chew on some of the points you raised a bit more.

  3. Good point about "de facto standard" and "open standard". My point is simply that most of the value comes from the standardness, not the open-ness (whatever that means!). Obviously you, Simon, and Bob disagree.

    I might concede (I don't really have an opinion) that the world could use a consensus *exchange* format for office documents. I don't understand the position that the world needs a common *native* format for office documents, and reject the notion that the need is so great that the well-known evils of design-by-committee are worth it. It's a fair point that the MS stuff is essentially designed by a committee too, but *ultimately* there is someone responsible who can break deadlocks. The pattern I see over and over at W3C is two camps, one preferring option A, one option B; no consensus is possible on either A or B, so A *and* B is adopted. I see that less inside (effective) companies — somebody makes a decision and takes the consequences.

    On your third point, you restate my position accurately. IMHO the One Size Fits All format (or language, religion, etc.) never ends up fitting all. I could get on my Darwinist high horse and say that *diversity* not uniformity is the source of all progress and that's not really such a great idea in the first place. Sooner or later I'm sure the technology here will stagnate and we'll have a single format. That can't be forced; premature standardization is the root of a lot of evil in the tech world (XSD being my favorite example).

    I really don't know of the Open Document format has enough people on board or reality on the ground or deep support in the customer world to make it a reality to be reckoned with. I personally wouldn't mind if it gets mindshare as the common exchange format (I know and greatly respect many of the people who developed it, so its probably pretty decent as a common denominator. I certainly wouldn't mind if Office 12 supported it as a non-default import/export format. For that matter, they will be both documented XML specs … a cheap or open source Office extension could easily fill the gap if the MS Office people do not.

    I do think that its reality has to be proven in the field, not just the committee room. The tech world is littered with "standards" with no mindshare or economic value. It's impossible for even MS to support all the XML specs even if it had a semi-plausible business case to do so. (BTW, my day job is to own the problem of understanding the technical and business case for supporting or not supporting the various XML core technology specs). This one will just have to earn its mindshare just like the others. I believe in bandwagons when I see them rolling and the parade following after, not when the band on board starts playing 🙂

  4. I am enjoying this post and I think one example that that speaks to Stephen's original point about incentive is, this assumes I understood it correctly

    The implications of a "de facto standard" and how it relates to an open environment ultimately comes down to dollars and how an organization can visualize creating value and therefore revenue.

    As an example, Adobe has been building it's strength as an organization and in doing so gaining huge control over a market segment that most people just aren't watching (that's another discussion). But to get there they made parts of their PDF format available to market. Why, to gain market acceptance and in doing so they become the "de facto standard". Therefore if I want to publish and share in this networked world, using PDF is a standard that ALL application players can choose to support and many do. Sun has native support for PDF in its open office product, why not Microsoft?

    The simple answer to above is because Adobe has been a good partner for Microsoft to drive sales of Adobe’s products and for Microsoft’s, but Adobe hasn't done it to the exclusion of the rest of the industry and platforms, Microsoft has.

    In real life this translates into potential and ultimately how I will spend my IT budget on office and related tool sets.

    Today if I want to share a document with you, I will use Word to create it because it is the "de facto standard", and then convert to PDF (Not necessarily using one of Adobe's products but their licensed technology). Why, could be many reasons from protecting my original document or digital signature I put on that document, to reduce file size when sharing it, to creating a more secure document that my readers will have faith in that it is a secure document. As I look to open office and it's variants, I can eliminate that step and create PDF natively from the toolset, that is worth investing in.

    This is a bit of a leap but as I see it, just because I want to consider alternative’s such as Linux desktop, doesn't mean I am not prepared to or want to use MS office. It could just as easily be Apple Mac, but they built support for that.

    The answer of course is to support an alternative takes money from one pocket and moves it to another, i.e.: windows to office. The underlying answer here is that I believe Microsoft believes that Windows OS is the "de facto standard" worth dying for and they can't do anything to mess with that. Reality is the market is going around them because they fail to see the market need. Perhaps if the anti-trust solutions had broken them up the office group would be willing to play with more partners and the OS division would have to rationalize why they are relevant to my business. That isn't the case and that is why this discussion and adoption by MS is a mute point.

    Imagine if you will if Sun, IBM, and others adopt an open standard that becomes an emerging "de facto standard", then all the potential leverage that MS had could begin to crumble. It is not because Word was bad, it's because we suddenly had choice.

    In closing please consider this, today as "de facto standards", MS products are that because the market lacked good choice that was viable. Ever since my early days with Dos 1.0 (I date myself) MS emerged as a leader because when Visi Calc was the "de facto standard" for spreadsheets and Lotus was emerging with 1-2-3, Microsoft gave us a choice. As more and more people came on board, the product got better because that is what the market demanded. Now we get what we are given because one company makes that decision for us to benefit their pocket book. Choice is emerging once again and by many accounts it appears to be viable, time will tell. You can either watch the bus go by or get on and help set it’s course, Microsoft appears to want to keep making it’s own bus and therefore put at risk sharing the title of “de facto standard” with emerging standards.

    Thanks for listening…

  5. I was researching more into the Microsoft Office XML Reference Schema licensing approach, and for fun I downloaded all of the information on the OASIS OpenDocument specification.

    I find it remarkable that no one seems to be very concerned that the licensing conditions are almost identical! In particular, Sun makes an "Essential Claims" royalty-free limited patent license in the way that is usual for this kind of community-developed specification (e.g., the W3C ones that have similar constraints, like SOAP). Certain kinds of derivative works are permitted for OpenDocument, but that doesn't appear to include derivative works of the schema and specification as part of some repurposing or extension activity.

    I've begun a too-long piece on the Microsoft license at http://orcmid.com/blog/2005/06/office-xmls-ip-inf… and I'll continue when not impaired by the pain-killers following a day surgery that I went through today.

    I note that the OASIS document is 700 pages long and is not yet fully implemented. Arranging a couple of independent interoperable implementations may take a while. The specification uses Relax-NG to present the schema, which may excite some but it doesn't work so well as the Microsoft use of XSD for people who want to do schema assessment with widely available tools.

    Avoiding OpenDocument and choosing to develop specific XML schemas optimized for the three principle Microsoft Office formats and their legacies was maybe not so arbitrary nor capricious on Microsoft's part. This may be a long evolutionary march, building on the work that has already been invested in Office XML formats.

    I think the thoughtful discussion that is happening well in advance of Office 12 presents quite an opportunity to maximize the utility of Microsoft's move. It will be interesting to watch, especially as we can move from speculations to genuine trial use and testing (of both approaches and the ability to make document-fidelity-preserving conversions across schemas).

  6. People, When someone fails your trust a number of times, I get cold feet about them like in this case.
    We all had a protocol to access remote file Systems called NFS, Microsoft came um with SMB ( now CIFS) with it's only goal to hurt interoperability.
    We all had some pretty nice protocols to access our email (POP and IMAP), Microsoft came up with another protocol called MAPI that does nothing they couldn't have done with IMAP but, because it's proprietary, it hurted interoperability and increased the Exchange sales.
    I can concede the Open Office File format really wasn't the best for Microsoft but, I really don't take their word for it. An honest answer would have been to state what can't be done because of all the legacy users.
    Since they always fail in giving technical reasons for their choices, I am asked to believe them on faith and, because of MS's history, I see no reason why I should believe them.

  7. > An honest answer would have been to state what can't be done because of all the legacy users.

    That *is* the MS Office people's basic answer. See http://www.betanews.com/article/Microsoft_Opens_O
    '"We have legacy here," Jean Paoli, Senior Microsoft XML Architect, told BetaNews. "It is our responsibility to our users to provide a full fidelity format. We didn't see any alternative; believe me we thought about it. Without backward compatibility we would have other problems."'

  8. > It is our responsibility to our users to provide a full fidelity format.

    That's a conveniently vague non-answer. And a far cry from Jaime's call for "an honest answer [that states] what can't be done because of all the legacy users."

    What does MS XML do that can't be done with OpenDocument and is necessary for legacy users?


  9. I don't believe the burden of proof about what OpenDocument can do is or should be Microsoft's. That's a pretty high bar. I think it really falls on OpenDocument adherents to demonstrate that full-fidelity round-tripping can be done from MS Office to OASIS OpenDocument and back. The availability of the Office XML Open Format should make it easier to confirm and assess that.

    People who think this is a no-brainer that Microsoft slimed out of need to step up to the plate and find out what the problems really are.

  10. As well as a fine blog entry above, there are some very rational and lucid points in the comments here (as well as some very arguable points, but I'll leave those for others). The problem is not lucidity, it's presuppositions. Mike is arguing from the position of a fait accompli, echoing Jean Paoli's assertion. His unspoken assumption is that Microsoft could not have affected the outcome of the OpenDocument group at OASIS. However, they are a prominent member of OASIS, were aware of the group (Microsoft employees even visited on one occasion I am told) and chose not to participate.

    There are thus two similarly-specified but incompatible formats in existence. One is in the stable hands of OASIS, the other in the unaccountable hands of Microsoft. It may be feasible to construct 100% accurate transforms after the fact but I wish Mike's views about format diversity and Jean's views about support for legacy had been expressed in the OASIS working group instead of being saved for now.

  11. Thanks for clearing my sentence Daniel.
    Orcmid, I wasn't talking about the "Burden of Proof", MS had a career based in locking. Just look at MAPI, CIFS, AD, Office, MS J++ and IE. All made to lock customers and hurt interoperability. Now, what has changed? I think MS's history entitles me to have cold feet about their integration efforts and asking for a better explanation about what they say, after all, it's been a while since I was forced to take "because" as an answer to things.
    Of course I also recognise MS's right to don't care about my opinion. I'm a competitor and not the target MS want's to reach but, still I think it was a fair question.

  12. "There are thus two similarly-specified but incompatible formats in existence. One is in the stable hands of OASIS, the other in the unaccountable hands of Microsoft. It may be feasible to construct 100% accurate transforms after the fact but I wish Mike's views about format diversity and Jean's views about support for legacy had been expressed in the OASIS working group instead of being saved for now."

    A couple of points. "There are thus two similarly-specified but incompatible formats in existence." It's not clear to me that this is a problem. There are actually a lot more than 2 — Word 2003 has by my count 10 different types on the Save As dialog and 16 on the Open menu. Whether that goes up by one or it goes up by two in Office 12 doesn't seem terribly significant for users. (Assuming for the sake of argument that Office 12 supports OASIS O/D as a non-default file type.)

    Simon Phipps and several others as for a detailed technical rationale for why MS did not participate in the OASIS effort to ensured that Office's requirements were met: First, I wasn't an MS employee at the time and I don't work for the Office group, so my opinions are more or less irrelevant to them and my information is sketchy. All I can say is that I do know what some program manager would have to do to make the case for joining the OASIS TC, and for what it's worth (not having any real inside information about Office) I would not have a clue what arguments to make. Why *should* all office software share the same native file format? From what I know in general, it seems that would add a lot of constraint on the MS Office developer's ability to add features, improve performance, etc. We already have HTML+CSS and PDF as common display formats, XSLT as a transformation mechanism, and presumably a lot of 3rd parties who will be happy to fill in any gaps in the N x N format translation matrix. It's probably overly simplistic so say "another office format, another set of stylesheets to write", but that's my basic response for what to do about incompatible data formats. In short, it's just not clear to me what the business or technical case for a One Size Fits All office uber-format is. If there is one, OO will just have to make the case on their own and prove it to the world.

    Finally,the folks creating the MS Office formats are accountable to managers, customers, stockholders, government agencies, etc. and all of them are rather demanding overseers. MS makes a very expensive long term support commitment and takes a large exposure when it ships, changes, or kills off a technology. OASIS or OpenOffice.org can far more easily bury their mistakes without real accountability than MS can, so I hope others don't fall for that "stable hands of OASIS, unnacountable hands of MS" soundbite.

  13. Ahem. Cough, cough. Look what I found. Microsoft OX vs. OASIS OD: Is It Really Open Format vs. Open Standard? (http://orcmid.com/blog/2005/06/microsoft-ox-vs-oasis-od-is-it-really.asp)

Leave a Reply

Your email address will not be published.