GroupLens Research

I just posted an entry in Wikimedia's blog explaining part of a study I'm working on with some Wikimedians (Wikipedians working at the Wikimedia Foundation). In response to speculation that the English Wikipedia's editor decline could be the result of a general decrease in the quality of newcomers to the site, we performed a hand-coded evaluation of the first few edits performed by editors over time.

Overall, we found that the quality of newcomers has not substantially decreased since 2006. While the rate at which these good newcomers have their contributions reverted or deleted has been rising over time, the survival rate of good new editors has been falling. This supports our working hypothesis that the increased rate of rejection for new editors is causally related to the decline in the survival of new editors.

See the full report here: http://meta.wikimedia.org/wiki/Research:Newcomer_quality

This analysis is part of a larger contribution in submission to a special issue of American Behavioral Scientist on Wikis. Stay tuned.

In the human body, there are two groups of cells that manage the production and refinement of bone. Osteablasts create new bone while osteaclasts break bone down. These cells are constantly working in parallel to manage our bone structure and repair damages. When a fracture takes place, osteoblasts come in to calcify the tissue surrounding the break. They aren't very picky about what or where they calcify so you'll end up with a large mound of bone where the break was. Over time the osteoclasts will trim and refine this bone down until the bone reacquires its original shape.

I feel that this is an excellent metaphor for how work is done in Wikipedia. There is a very large group of editors who do not make many edits on an individual basis, but they contribute the vast majority of content that makes it into the encyclopedia. They behave like osteoblasts in that they contribute large amounts of material but they don't have the experience to know what sort of content is encyclopedic. A smaller group of more active members of the encyclopedia (Wikipedians) perform the role of osteoclasts by trimming unencyclopedic content and refining what is left over into coherent articles.

In order for a human to have a healthy skeletal structure, a balance between bone formation and bone trimming has to be maintained. In the same way, the balance between content contributors and content refiners in Wikipedia must be maintained.

A few days ago, as explained by Udi Manber, Google announced a new service, called Knol, which seems to have approximately the same goal as Wikipedia: to create a more or less comprehensive repository of useful knowledge. Because Google is the super-juggernaut du jour, there is a lot of speculation that Knol will be a Wikipedia killer.

I disagree (not a unique point of view). Frankly, I don't find the Knol idea all that interesting, and if it wasn't Google proposing it, I don't think anyone would have noticed. The basic difference from the Wikipedia collective-authorship approach is that articles are "owned" by a single person. Others may rate, suggest content, etc., but the owner is the sole arbiter of what the article contains.

Here's why I think Knol is uninteresting in 2007:

1. No microcontributions. It's impossible to make a tiny contribution (e.g. fixing a typo). Sure, you can suggest that the typo should be fixed. But there's a lot of value in the immediate gratification: people like to see that the article is better right away due to their (tiny) efforts. In aggregate, microcontributions have lots of value in and of themselves, but they are also a good way to lead people to making macrocontributions.
2. No effort at consensus. It is left to the reader to make sense of the several competing articles on a particular topic. One of the huge benefits of Wikepedia's approach is that this onerous task is more or less done for you.
3. Single point of failure in article maintenance. If an author loses interest in an article, it's difficult or impossible for others to take over and continue work.

These problems are orthogonal to whether Google is able to successfully incentivize authors with money or recognition (things Wikipedia can't do).

I do agree with Manber that many people who have knowledge often don't share it because sharing is hard. But, I do not think the right way to make it easier is to introduce a new Knol-style service. Rather, I think adapting the Wikipedia approach to be friendlier is much more promising, for example by implementing a WYSIWYG editor and making policy less byzantine.

(Particularly welcome in the comments are links to interesting analyses of Knol.)

Wikipedia is going to implement Alfaro et al's algorithm to assign trust levels to individual chunks of text within articles based on the reputation of the author of the chunk. The interface will use color coding to visualize trust levels. 

http://technology.newscientist.com/channel/tech/mg19526226.200-wikipedia-20--now-with-added-trust.html