The Wisdom of Crowds, Crowdsourcing, and other New Bottles for Decentralized Work

Of late, there’s been a re-emergence of chatter about the idea of using large groups of people to not only work together, but innovate with. This line of thinking goes under names like The Wisdom of Crowds, “Crowdsourcing,” “harnessing collective innovation,” or, more generally, “The Internet.”

The flippancy in the last moniker is meant to highlight that the idea has been kicking around for awhile. Indeed, I tend to fall into simply calling the idea “The Network.” Mr. Wealth of Networks, Yochai Benkler, outlines that sentiment quite well in a summary talk about his book and ideas. As Jon Udell puts it, the new phrase is “commons-based peer production.”

Having not finished reading The Wealth of Networks, I just have the dilettante-take from his talk and other skimming. One of the ideas is a crystalization of what’s implied in other goes at The Network Idea such as Linked, Cluetrain/Gonzo Marketing, and even some much books from the pre-Web days, like Invisible Rendezvous.

The difference with Benkler, it seems, is that he’s positing a fact instead of simply describing the web and Internet, which was a favorite past time of academics, professional and otherwise, throughout the past decade and a half.

That fact is (something like) this: a group of people connected by The Network can work together as effectively, and at times more effectively, than a group of people connected together in more traditional organizations. A “traditional orginization” is either a company, government, or even a more abstract system like “the American legal system.”

Theories from The Network

All this “The Network” talk is interesting, but what’s more interesting is trying to do something with it. I consider myself part of the first generation that, more or less, has spent their entire life connected to The Network. As such, I’ve picked up plenty of ideas about what it means to live a network-connected life and how it works. It’s hard to keep all up with all the citations for ideas from Neuromancer to Benckler, but as we’ll see below, citation isn’t always that big of a deal in The Network anyhow ;>

Identity

Your online identity is as real, and sometimes “more,” as your offline one. Think of people who play World of Warcraft. Or, in the case of bloggers (myself included), think of how strong an online voice can be compared to an offline one.

Of course, the image that pops up first is a depraved, PIB getting a CRT tan in his room playing too many games (if a woman does it, it’s suddenly HOT). On the other hand in the rhetorical words of RU Sirius in an recent episode of his, “is reality just a crutch for people who can’t handle drugs?” While there is a problem with having only email-friends, in an additive mindset, they shouldn’t be dismissed as non-genuine. You just need some meat-world friends too ;>

Free Distribution

Costly distribution is the core problem to spreading ideas and starting conversations around those ideas. Without a The Network, you have to charge for the cost of moving ideas from your head to your readers head, and, thus, you fall down the slippery slope of putting value on ideas apart from what’s done with those ideas.

There’s an interesting idea to analogize in programming. It turns out that design patterns and idioms are often just strategies to fix bugs in languages. That is, what looks like a “fact” or even a good design idea is just a band-aid obscruing an underlying problem. For example, using interfaces really only matters in strongly typed languages. Interfaces are wasted characters in duck type languages.

If you make distribution free, then, how does it change how we value and protect ideas?

Taking it further, because physical distribution is difficult, we assume that you can control the distribution of ideas. Benckler’s Diebold story is a great example of this. Some internal Diebold IP leaked out, so they C&D’ed several people who posted it to websites to stop (read: control) the distribution of that IP. Obviously, Diebold wanted to control the flow of ideas because the IP was a smoking gun that they’d screwed up and would have to spend time and money to fix it.

Of course, as The Church of Scientology found out in the 90’s when they were trying to control the distribution of their IP (yes, The Church has IP, go figure!), The Network makes it impossible to control distribution: there are too many nodes. Diebold, Scientology, and the anti-piracy folks have all found (though probably ignored) that trying to control the distribtion of IP on The Network is a looseling battle.

Once you quell the urge to buck against that fact, the next step is to start asking yourself, how have the constraints of physical distribution leaked back into my understanding of what’s being distributed? That is, distribution, like all things, is a leaky abstraction. More precisly, it’s a leaky service. Once you bilge out that leakage, you end up with new ways of thinking and business models.

A Bucket of Eyeballs

At the core of The Network think of late is the idea that more eyes and “hands” on a problem increasing the chance that you’ll have high quality output. The key, as Anne highlighted recently, isn’t that the group “votes” on the best possible solution, it’s that the they average out all of their individual answer into a collective one. Kind of weird to conceptualize, but a simple example if using Google for spellchecking.

Of course, you can apply a rising tide lifts all boats theory as well: those high quality individuals will help make the rest of the individuals in the group more high quality, and vice-versa if you want to get all warm-and-fuzzy. When I get all excited about diversity, it’s to get this effect: a large group of people will have many different ideas and approaches to an idea or problem. While you could survey all that different thinking and pick out the first, second, and third best idea, salad barring that pool of knowledge to synthesis new approach is where things get exciting.

The problematic thing with the bucket of eyeballs is the same problem that’s existed since the beginning of time: the individual versus the group. It’s too tempting, if you’re one of the “smart” people to go out on your own and get all the credit for your ideas and approaches. More pragmatically, our economic system has no idea how to compensate people who are in a group. The yearly “over-paid CEO” lists are a fantastic pieces of art that explore how inefficient our economy is at rewarding the group, favoring heros over groups. The death of retirement is another example.

Middle-men

As the bucket of eyeballs implies, The Network hates middle-men. So far, in network-think we’ve primarily thought of middle-men in the terms of retail — book and music distributers. But with a more mature understanding and road-test of open source software and wikipedia, we understand that the middle-man concept includes the aptly named middle-management. Indeed, the idea of management apart from the core function of the work at hand seems quaintly antiquated in both open source and Agile software development.

Sure, those functions, and even roles, need to be done: the group needs constant herding and feedback to see how things are going and adjust accordingly. But, with a large group of people working correctly, management is distributed amongst the group instead of being centralized in one person or small group of people.

Now, that line of thinking is half-baked and overly idealistic. Part of the reason that open source and wikipedia don’t need a middle-manager class in the traditional sense is that the middle-managers emerge as needed. What’s more important is that the idea of who’s a middle-manager in The Network is fluid: you can be a committer and “project lead” on one open source project, but you’re just some jack-ass to another one who needs to prove themselves before you can have any effect. And it’s not just across projects that your status is transient, it’s through time as well. You might have done great work and gotten rock-star status during the 1.0 version of the project, but as things evolved, you became dead-weight holding back 3.0 and, thus, lost your status.

Things don’t work like that in companies and large organizations: once you’re in middle-management, it’s as if you can ignore meritocracy and The Peter Principal. Often times, companies actually exists to keep people employed instead of pumping out profitable projects. We easily put government-work in that role, but the fact of the matter is private industry is pretty much the same once you get your Golden Ticket.

Boiling it down, The Network is both (a.) a meritocracy, and, (b.) crappy at transitive reputation. You have to prove yourself to the group you’re working in before they’ll let you do anything, and that proof can’t be used on new groups, you have to re-prove yourself each time.

From another angle, this means you have to start trusting groups and not just individuals. That’s a lesson that current thinking isn’t ready to deal with for many of the same reasons that we want to reward heros instead of groups: it’s just not normal Western thinking. This means that few people and organizations will leap at the chance to profit from situation that require your to trust groups over individuals.

The example, which I’ve been citing a lot recently, is Vista vs. Eclipse. Both are large systems under development by large groups of people, but both take radically different approaches to developing software. While Vista has had a problematic time, Eclipse seems to be doing fine. The excuse is often something along the lines of, “Vista is huge, it’s tackling an extremely difficult problem, there’s all those devices, there’s lots of people and politics behind it, etc.” Indeed, I’m not dismissing all those problems at all.

I complained in a recent RedMonk Radio podcast that I’m tired of hearing people use the excuse, “well, we’re a big company, changes take time.” Here, were getting to why I’m sick of hearing that: the fact that you’re a big, closed, static company is probably part of the problem. As the old joke goes, “The patient says ‘Doctor, it hurts when I do this.’ ‘Then don’t do that!'”

Sometimes forking is for the greater good of the project at hand. Emergent behavior is only good if you nurture it once it emerges, which explains why you can’t just expect your employees to “innovate in their spare time” and then dump the usual work-load on them.

Blinded by Privilege

You have to be able to read and type (the new “write” ;>) to get involved in The Network. More importantly, if you’re worried about feeding yourself, being killed, or worse all this is meaningless.

That’s the The World is Flat critique, and it’s well said. As our American misadventures of late show, Plato’s (later picked up by Christ) idea that once you’re shown truth, you’ll walk lock-step with it is bogus. Part of the problem is that you can’t force someone to join a group if you hope to achieve genuine participation from them. The other part of the problem is thinking that “the truth” is static instead of emergent and fluid. But really, to be frank, membership and access to The Network is expensive, cutting out billions of people.

The other problem, as I mentioned several times above, is that The Network itself hasn’t leaked into the other systems in our society. As I commented on Anne’s post, if you need real health insurance in America, someone in your family needs to nuzzle up to a traditional organization. The Network doesn’t cover giving birth. Nor will it cover taking care of you when you’re 93, have lost your mind, and require 24/7 nursing.

And that, is the problem with The Network: once you unplug from it, it has no power. At it’s core, the ultimate problem with The Network is that it’s too utopic: successful systems are rarely built around amplifying the ideal, and rather revolve around accounting for flaws and inevitable problems.

But, this isn’t to zero-sum all of the positive aspects of The Network, it’s just to remove the chance of it getting pie-in-the-sky fruity. The Network is just a really good tool instead of a complete solution. What’s important, then, is figuring out how it can make the grander system better, instead of whole-sale replacing it.

Disclaimer: Eclipse is a client.

Technorati Tags: crowds, crowdsourcing, groups, thewisdomofcrowds, agile, networking, web, work, management, benckler, opensource