Wikipedia’s method for sorting out “good” and “bad” sources is a mess

The digital encyclopedia has no real leadership, doesn’t vote, and has no rules. So how does it govern itself?

Wikipedia’s method for sorting out “good” and “bad” sources is a mess

The digital encyclopedia has no real leadership, doesn’t vote, and has no rules. So how does it govern itself?

In February, The Guardian reported that editors at Wikipedia had “voted to ban the Daily Mail as a source for the website” after deeming it “generally unreliable.”

The Daily Mail, a UK-based daily print and online publication with a daily newsprint circulation of 1.5 million and 238 million unique visitors a month, responded with a series of angry articles, ambushed one editor at his mother's home, and released a statement saying it banned its own reporters from using Wikipedia as a source in 2014.

Except, Wikipedia never truly banned the Daily Mail. Many citations pointing back to the Daily Mail are still live, and new ones have appeared on Wikipedia since the kerfuffle. So what's going on?

H.G. Wells predicted the need for something like Wikipedia back in 1937, saying that “without a world encyclopaedia to hold men's minds together in something like a common interpretation of reality, there is no hope whatever of anything but an accidental and transitory alleviation of any of our world troubles.”

Wikipedia editor Andrew Davidson shared that quote at the beginning of a talk in London earlier this month in which he explained how Wikipedia editors clean up the site. The site's editors, who are volunteers, have always struggled over the essence of facts. Famous battles breaking out over the origin of hummus, when to use Gdansk versus Danzig, and how to spell the word “yoghurt.” This process is decentralized, democratic, and well-documented; these arguments play out on the "talk" pages for individual entries as well as forum threads dedicated to editor discussion, where they are saved forever.

There are no rules on Wikipedia, just guidelines. Of Wikipedia's five “pillars,” the fifth is that there are no firm rules. There is no formal hierarchy either, though the most dedicated volunteers can apply to become administrators with extra powers after being approved by existing admins. But even they don't say what goes on the site. If there's a dispute or a debate, editors post a "request for comment," asking whoever is interested to have their say. The various points are tallied up by an editor and co-signed by four more after a month, but it's not a vote as in a democracy. Instead, the aim is to reach consensus of opinion, and if that's not possible, to weigh the arguments and pick the side that's most compelling. There was no vote to ban the Daily Mail because Wikipedia editors don't vote.

The Daily Mail is known, especially online, for sensationalist content, sloppy reporting, borderline plagiarism, and the occasional fabrication. The paper made up an entire story — with quotes and colorful description — reporting the wrong verdict in the Amanda Knox trial. However, it has won kudos for original reporting and was named newspaper of the year at the latest Press Awards. Wikipedia editors frequently argued about its validity as a source in the discussion section for individual entries. In this case, an editor submitted a broader request for comment about its general reliability. Seventy-seven editors participated in the discussion and two thirds supported prohibiting the Daily Mail as a source, with one editor and four co-signing editors (more than usual) chosen among administrators declaring that a consensus, though further discussion continued on a separate noticeboard, alongside complaints that the debate should have been better advertised.

Though it's discouraged, the Daily Mail can be (and still is) cited. An editor I met at a recent London “Wikimeet” said he'd used the Daily Mail as a source in the last week, as it was the only source available for the subject he was writing about. The site has a link filtering tool that automatically bans spamming sites, text with excessive obscenities, and persistent vandalism (trends such as leaving "your mom" on pages), but it has not been activated for the Daily Mail.

The change is less of a ban and more of a “general rule not to use Daily Mail references when better ones exist,” said John Lubbock, communications coordinator for Wikimedia UK, the charity that helps fund and organise the encyclopedia, but doesn't direct its efforts.

Lubbock noted that the move means editors will replace Daily Mail links with better sources, but with some 10,000 in use, that work may never be fully completed. If there's no more reliable source, editors have to make a judgement call: if only the Daily Mail is saying something, can we trust it? If not, delete the fact. If so, keep the link.

There was no vote to ban the Daily Mail because Wikipedia editors don't vote.

That practice isn't new on the site, and it isn't limited to the Daily Mail. Buzzfeed is generally considered not reliable by Wikipedia editors discussing the issue on the Reliable Sources noticeboard, though such discussion isn't binding and won't be seen by many editors. While its listicles may be of little use to an encyclopedia, it has an investigations team and was shortlisted for a Pulitzer this year.

Meanwhile, less-reputable sources including Russia Today and Breitbart aren't listed as unreliable. However, editors on the site and those I spoke to pointed out that editors shouldn't need reminding that those aren't trustworthy sources.

Debate aside, the Daily Mail itself noted that the "vote" saw 53 editors decide for the millions who use Wikipedia, but the encyclopedia isn't a democracy. The “Request for Comments” pages where such debates happen are rooms for remote debate that anyone can take part in. And there, consensus isn't about tallying votes, but weighing the merit of arguments.

That means a minority could win a dispute by making a better case, though in the case of the Daily Mail, a majority of editors involved in the conversation did back the “ban.” It's a small slice of the the 135,000 people who edit the site each month, though one editor pointed out that the vote was “watched” by more than 2,000 users, more than a usual debate would see.

Editors often do reach consensus. They have to in order to disable open contributions for controversial pages, for example. They recently introduced tighter guidelines for entries on living people to avoid fake death reports and libel. They've also agreed to use systematic reviews rather than individual studies as citations on medical pages.

Enforcement is a different matter. These decisions are typically enforced by editors who revert changes that don't meet the agreed-upon standards. This means the back-and-forth continues on Wikipedia's pages. The Daily Mail decision supported using an automated edit filter, but with it not in place and no apparent plans to do so, there's no reason a person new to the site would even know about the “ban.” And even if an automatic edit filter was used, it wouldn't outright ban the Daily Mail as a source. Though that is technically possible, it would simply show a warning message but then let the editor still click to save the link to the Daily Mail. Remember, there are no firm rules.

As foolish as some Wikipedia battles may seem, eventually consensus is reached, reality is decided upon, and we can feel like we're on solid ground

In the end, there was no vote, there is no ban, and plenty of other newspapers have had similar treatment, with a Wikipedia guide to potentially unreliable sources listing the Sun, Daily Mirror, TMZ, and Forbes.com. Listing the Daily Mail as an unreliable source is merely a trump card for editors to batter each other with during their constant debates about sources. If you want to link to the Daily Mail, be prepared to defend why. If you can't, the link will be replaced.

As foolish as some Wikipedia battles may seem, eventually consensus is reached, reality is decided upon, and we can feel like we're on solid ground. The site's volunteer editors are bickering their way to a “common interpretation of reality,” something we desperately lack here in 2017, with newsroom cuts gutting fact-checking, the rise of fake news, and a president who constantly contradicts himself. “We don't have the certainties we used to… that leaves people unsure what's reliable and who to believe,” one editor told me. “People in politics play off that, to confuse people, to paralyze them.”

Knowledge is power

The Whitehouse.gov reset broke Wikipedia links en masse

Here’s what editors are doing about it.
Read More