Say you want to contribute to Wikipedia. You sit down in front of your computer after dinner with a nice cup of coffee and wonder what you can do to help. English Wikipedia has an extensive set of cleanup templates that can help you find articles requiring specific improvements. WikiProjects or a tool like SuggestBot can help you find articles related to your interests. We want to combine these and give you a list of interesting articles while at the same time show whether there’s an opportunity for contribution and specific tasks for improving the article.
Some of the recent research examining what makes articles high quality in Wikipedia has taken an editor-based approach, looking at for instance diversity and coordination (Wilkinson and Huberman, 2007)(Kittur and Kraut, 2008), and editor reputation (Adler and de Alfaro, 2007)(Halfaker et al., 2009). While these models of article quality are informative about work practices, they won’t be able to give you straightforward suggestions of what you can do to help on any particular article. Instead we prefer actionable features, those which easily lend themselves to being acted upon by a contributor. Research has shown that the amount of article content has a strong relationship with article quality (see for instance Blumenstock, 2008). This means that “add more” is a reasonable suggestion for improving quality, but we prefer suggestions to cover a variety of tasks, what specific types of more is this article in need of?
In our paper we evaluated several different models for predicting article quality in Wikipedia. We began with evaluating a well-known model’s (Stvilia et al., 2005) performance on more recent data, then modify that model to incorporate features that are more prominently used now (e.g. number of references) and discard features that do not help predict quality. Our actionable five-feature model performed on par with more complex models on the task of differentiating between articles that are mostly complete and those that are not (86-87% success rate). The features in the model are:
- Article length
- Number of references/Article length
- Number of section headings
- Completeness (a measure of number of links to other articles)
- Informativeness (number of images + information noise)
English Wikipedia has a total of seven article assessment classes ranging from Featured Articles, the best articles Wikipedia has to offer, down to Start-class and Stub-class. Predicting which of these seven classes an article falls into is a difficult problem, our simple model gets 42.5% of these correct, but if we allow for a 1-class error margin it gets 76.9% right. A more complex model scores better (48.3% overall) with gains mostly on Featured and Good Articles, which go through a peer review process that our content model doesn’t appear to capture.
How can this model help contributors find venues to contribute in Wikipedia?
We can create a tool that finds articles that appear to need reassessment, or one that uses these five features to suggest specific tasks for improvement. The latter is already in use in English Wikipedia, where SuggestBot will point out improvement tasks to users who have subscribed to get article suggestions periodically.
While our set of tasks are fairly diverse they might not suit inexperienced editors, for which other tasks might be better like copy edits or adding links to/from other articles. We also experienced first-hand that the concept of article quality differs across languages (Stvilia et al., 2009). When we attempted to use the same model for Norwegian (Bokmål/Riksmål) and Swedish Wikipedia we found that it fails because they do not use footnote citations as extensively as English Wikipedia.
Our research has shown that a simple model containing only actionable features has strong performance in predicting article quality in English Wikipedia. Wikipedia contributors who are looking for venues to contribute can therefore have computer tools help assess the quality of articles as well as suggest specific tasks to improve them, making it easier to discover how to contribute and improve Wikipedia’s quality.
More information
For more information, see our full paper:
Tell Me More: An Actionable Quality Model for Wikipedia
Morten Warncke-Wang, GroupLens Research, University of Minnesota
Dan Cosley, Department of Information Science, Cornell University
John Riedl, GroupLens Research, University of Minnesota
References
D. M. Wilkinson and B. A. Huberman. Cooperation and quality in Wikipedia. In Proceedings of WikiSym, 2007.
A. Kittur and R. E. Kraut. Harnessing the wisdom of crowds in Wikipedia: quality through coordination. In Proceedings of CSCW, 2008.
B. T. Adler and L. De Alfaro. A content-driven reputation system for the Wikipedia. In Proceedings of WWW, 2007.
A. Halfaker, A. Kittur, R. Kraut, and J. Riedl. A jury of your peers: quality, experience and ownership in Wikipedia. In Proceedings of WikiSym, 2009.
J. E. Blumenstock. Size matters: word count as a measure of quality on Wikipedia. In Proceedings of WWW, 2008.
B. Stvilia, M. B. Twidale, L. C. Smith, and L. Gasser. Assessing information quality of a community-based encyclopedia. In Proceedings of ICIQ, 2005.
B. Stvilia, A. Al-Faraj, and Y. J. Yi. Issues of cross-contextual information quality evaluation–The case of Arabic, English, and Korean Wikipedias. Library & Information Science Research, 31(4):232–239, 2009.
