Wikipedia is the encyclopedia anyone can edit. This open, collaborative model is what makes it one of the world’s most popular sources of information. It is also what makes Wikipedia reliable and accurate, as everyone can review changes and additions to its articles. Although vandalism and inaccuracies can occur, its community of volunteer editors has established mechanisms to ensure that the vast majority of inaccurate content is addressed within minutes.
Last week, a study was published in the open-access journal PLOS One: “Content Volatility of Scientific Topics in Wikipedia: A Cautionary Tale.” The study prompted a flurry of discussion around the accuracy of scientific articles on Wikipedia. The Wikimedia community has longstanding support for academic research about Wikipedia. However, the media coverage of this particular study has drawn some questionable conclusions.
According to the study, articles on politically controversial scientific articles on English Wikipedia tend to receive higher edit rates and larger edits than scientific articles considered to be politically uncontroversial. The authors cite three topics they identified as politically controversial—acid rain, global warming, and evolution—and four they identified as politically uncontroversial: heliocentrism, general relativity, continental drift, and the standard model in physics.*
It didn’t surprise us to learn that articles considered to be controversial are frequently edited. The nature of controversy, after all, is that it generates discussion and public attention. For example, while the properties of water (H2O) have been well established, the causes of the Arctic sea ice decline are the subject of ongoing scientific inquiry and political debate.
Unfortunately, the study also jumped to conclusions about what this means for Wikipedia’s reliability, overstating findings and inferring facts not in evidence. Much of the press about the study has repeated the assertion that controversial articles are also more likely to be inaccurate, despite a lack of strong supporting evidence: the study only references a handful of anecdotal examples of inaccuracies. Instead, the study simply seems to confirm that the articles chosen as controversial are, in fact, controversial and thus frequently edited. One of the authors has since responded that they intended no claim about a relationship between higher edit rates and lower accuracy.
In fact, several prior studies have found the opposite to be true, demonstrating that more edits are correlated with higher quality articles. For example, a 2007 study published in the peer-reviewed journal First Monday found “a strong correlation between number of edits, number of distinct editors, and article quality in Wikipedia.” Similarly, in 2013 researchers observed that the number of contributions to high-quality articles is about one order of magnitude higher than that of low-quality articles, according to the book Confidentiality and Integrity in Crowdsourcing Systems.
In addition, the study covered a very small sample size, using just seven of the 35 million articles available across Wikipedia’s many languages.
Wikipedia’s community of volunteer editors take the commitment to accuracy very seriously. Many of them have personal academic or data science interests. In fact, a robust discussion critiquing the methodology of this study has taken place publicly on Wikipedia.
The aim of Wikipedia is to make the sum of all knowledge available to every person in the world. External research and observation are critical to helping Wikipedia grow and improve. But in true Wikipedian spirit, we believe any research should be assessed and reported with rigor and care. It is the same approach Wikipedia editors use to keep building Wikipedia as a reliable, accurate, and neutral resource for all.
*We’ll note that we found the inclusion of heliocentrism in the category of politically uncontroversial amusing. Hundreds of years later, we hope Galileo would appreciate the nod.
Katherine Maher, Chief Communications Officer, Wikimedia Foundation
Juliet Barbara, Senior Communications Manager, Wikimedia Foundation
