Donnie Berkholz's Story of Data

Defining community health at scale

Share via Twitter Share via Facebook Share via Linkedin Share via Reddit

I was at the Community Leadership Summit over the weekend, and this is part of a series of posts following up on some of the topics I felt were most important.

The community manager for Google Chrome, Jacky Hayward, ran a discussion session at CLS to ask how you can even determine the health of a community when it’s got hundreds of thousands or even millions of members. Clearly this is a tricky problem, and it seems that none of the community managers in attendance had a really great solution to this.

I proposed taking a similar approach that Stephen’s done a fantastic job using here at RedMonk — quantitative research by data mining — but in this case doing so on specific project communities rather than general development forums. Imagine taking IRC logs, mailing-list archives, web forums, or whatever a community has, then tracking activity and sentiment over time. An ideal way to do so might be repurposing system-monitoring software to import community data and track it, set alerts for problems, etc.

A major difficulty that came up in measuring health was getting data on the lurkers — the people who rarely or never participate actively, but just come to learn something or fix a problem. One attendee noted that it’s very hard to tell how introverts are engaging with your content and if they’re even having a positive experience. Another attendee followed up brilliantly that when trying to get at this information, you’re measuring the health of your knowledge base, not the health of your community.

I would posit that what the lurkers are doing is not related to your community at all, and they should be ignored in this context — beyond discussions of how to pull them further down the funnel to become participants. All users are not community members; some of them are just users.

Another topic that came up was basically data versus stories. As you probably know if you’re reading this post, given the title of the blog (The Story of Data), I don’t think the two are mutually exclusive. Especially in the context of community management, we need to remember that data are not the primary goal; the goal is to learn more about the people in your community, and data merely inform your decision-making.


One comment

  1. If you think of a community as a system, where a system’s robustness is often measured by its ability to absorb shocks, we could try to assess community health by asking how would a project respond to a major shock. We might call this the “Oracle Effect” after the effect on several open source projects that the Oracle acquisition of Sun had. The Hudson project had a strong and healthy community as it was able to reconstitute itself as Jenkins where as Open Solaris, despite some minor rearguard efforts, all but died when Oracle took over. You could then run some  hypotheticals (Oracle buying Canonical?) to see what the effect might be.

Leave a Reply

Your email address will not be published. Required fields are marked *