Like I need another project.
In my seeming never ending quest to understand how important Wikipedia is to the skeptic community and to the world, I selected Brian Dunning's Skeptoid podcast for an experiment.
I choose Skeptoid for several reasons, first, I have probably listened to every episode at least once, find them informative and know that Wikipedia readers will enjoy them as well. (editing Wikipedia is all about editing pages you enjoy after all). They are available as a podcast as well as in written form which makes it really easy to quote from. Also Dunning is very organized with a podcast every week, and the way he has the episode guide laid out with dates and titles I was able to quickly organize in a Google Document for my own use.
Also when I explained what I was trying to do, Dunning immediately grasped the importance and has allowed me access to his web site stats, without which I could not analyze the impact of Wikipedia posts.
Let me summarize what I'm trying to prove and then get to the numbers.
Skeptoid is a podcast/blog that analyzes many topics that usually correspond with Wikipedia pages. Mostly they are paranormal topics (some very obscure) and a few like Darwinism are not. He even has a few pages that discuss historical topics (which are some of my favorites).
I discovered in a long conversation with other Wikipedia editors that Skeptiod is considered a reputable source for citations (way to go Dunning, not many podcasts can say that) when it comes to fringe topics. But not for more common topics like the "scientist", "raw food" "SUV" and "pitbull" ones. We actually got into it a bit and several Wikipedia editors sprang to his aid (I did not bring attention to the conversation) and championed his podcast. One editor had the nerve to say that podcasts can't be citations because the deaf could not listen to the podcasts. One person jumped on her comment saying how ridiculous that was and there was no policy for that, besides they pointed out Skeptoid has print as well as audio. It was quite obvious that some of his fans are also Wikipedia editors, very heart-warming.
Several Wikipedia pages like the ones mentioned above, I tried to just add an external link to the Skeptoid article which normally was reverted by other editors. Tim Farley explains that you should rarely leave external links, but instead add the article as a citation somehow, which are rarely reverted. External links are notorious for Spam. Well live and learn! There were a few pages that already had external links, I just cleaned up the citation and left it there. Also several of his podcasts do not have a corresponding Wikipedia page to leave a citation on (like several humorous ones).
So I copied his entire episode list into a Google document. On one page it has every episode and date as well as the corresponding Wikipedia page that has a link. Some of the episodes have more than one Wikipedia page, for example this episode "Orbs: The Ghost in the Camera (Skeptoid #29) - Are orbs really ghosts, or a common artifact of photography?" is mentioned on the Ghost Hunting page as well as the Orb (optics) page.
Someone had already left some links before I entered into the picture, looks like 2006 was the last time that editor was active. And while the citations were correct, I didn't think they looked as good as they could and maximize the amount of hits that were possible. So I started cleaning up the links and even used a couple as examples for the Cafe Inquiry workshop to repair.
For example the Bible Code Wikipedia page now has this...
The Bible Code: Enigmas for Dummies
The Bible Code: Enigmas for Dummies
See how much more inviting this looks to the reader? There were many examples of these short edits dating back to 2006, some were even shorter. Not only does this look more appealing, but a reader can click on the link about the Skeptoid article as well as to the Wikipedia page for Skeptoid.
When I started Project Skeptoid in November 2011 there were 206 episodes to link to. By the end of November there were 38 references to the podcast on the corresponding Wikipedia pages. His hit results were 1.4% of the total views to www.skeptoid.com coming from Wikipedia. I'm not going to tell you how many hits 1.4% is because I'm measuring this purely by percentages. Lets just say it is several thousand views.
So I cleaned up a few of the already existing citations and added 10 more Wikipedia pages to the hit count which by the end of January 2012 became 48 Wikipedia pages. He had been adding more episodes all this time bringing his count up to 296 episodes. A look at his stats again and the month of January 2012 hit 1.82%.
Remember that not all of these episodes can have a Wikipedia page, they were humor episodes or were already rejected by editors, and some were student question or listener feedback episodes which I haven't bothered trying to link to.
As you can see, there is potential for improvement. Several thousand readers are following the links from Wikipedia to the episode on Skeptoid.com. These readers probably aren't normal Skeptoid listeners otherwise they would have just gone to his site. The goal is to improve Wikipedia as well as the skeptical/critical thinking exposure to NOT the choir (you and I) but to the general public. In this example I think it shows we are making a difference.
In February 2012 I added a few changes and new links but I'm going to take a break from reporting numbers at least till the March or April stats are available. I want to really make some head-way on the Google document, my goal next time I check the stats I will see 3 or 4%. I could really use some help, please contact me if you can, I will train. susangerbic@yahoo.com
Its a bit of an uphill battle to keep measuring against percentage of total Skeptoid.com views as the site becomes more popular he will naturally have more hits without Wikipeida. If someone can think of a better way of analyzing these numbers I would appreciate your thoughts.
By the way,
Regular Skeptoid fans you might be interested in which Wikipedia page is causing the majority of views back to his website.
HINTS - A very cold topic, this page only has an external link to follow and it has been overwhelmingly number one for both November and January with 5% of the total hits. This Wikipedia page receives about 130K views a month Click for Answer.
In my seeming never ending quest to understand how important Wikipedia is to the skeptic community and to the world, I selected Brian Dunning's Skeptoid podcast for an experiment.
I choose Skeptoid for several reasons, first, I have probably listened to every episode at least once, find them informative and know that Wikipedia readers will enjoy them as well. (editing Wikipedia is all about editing pages you enjoy after all). They are available as a podcast as well as in written form which makes it really easy to quote from. Also Dunning is very organized with a podcast every week, and the way he has the episode guide laid out with dates and titles I was able to quickly organize in a Google Document for my own use.
Also when I explained what I was trying to do, Dunning immediately grasped the importance and has allowed me access to his web site stats, without which I could not analyze the impact of Wikipedia posts.
Let me summarize what I'm trying to prove and then get to the numbers.
Skeptoid is a podcast/blog that analyzes many topics that usually correspond with Wikipedia pages. Mostly they are paranormal topics (some very obscure) and a few like Darwinism are not. He even has a few pages that discuss historical topics (which are some of my favorites).
I discovered in a long conversation with other Wikipedia editors that Skeptiod is considered a reputable source for citations (way to go Dunning, not many podcasts can say that) when it comes to fringe topics. But not for more common topics like the "scientist", "raw food" "SUV" and "pitbull" ones. We actually got into it a bit and several Wikipedia editors sprang to his aid (I did not bring attention to the conversation) and championed his podcast. One editor had the nerve to say that podcasts can't be citations because the deaf could not listen to the podcasts. One person jumped on her comment saying how ridiculous that was and there was no policy for that, besides they pointed out Skeptoid has print as well as audio. It was quite obvious that some of his fans are also Wikipedia editors, very heart-warming.
Several Wikipedia pages like the ones mentioned above, I tried to just add an external link to the Skeptoid article which normally was reverted by other editors. Tim Farley explains that you should rarely leave external links, but instead add the article as a citation somehow, which are rarely reverted. External links are notorious for Spam. Well live and learn! There were a few pages that already had external links, I just cleaned up the citation and left it there. Also several of his podcasts do not have a corresponding Wikipedia page to leave a citation on (like several humorous ones).
So I copied his entire episode list into a Google document. On one page it has every episode and date as well as the corresponding Wikipedia page that has a link. Some of the episodes have more than one Wikipedia page, for example this episode "Orbs: The Ghost in the Camera (Skeptoid #29) - Are orbs really ghosts, or a common artifact of photography?" is mentioned on the Ghost Hunting page as well as the Orb (optics) page.
Someone had already left some links before I entered into the picture, looks like 2006 was the last time that editor was active. And while the citations were correct, I didn't think they looked as good as they could and maximize the amount of hits that were possible. So I started cleaning up the links and even used a couple as examples for the Cafe Inquiry workshop to repair.
For example the Bible Code Wikipedia page now has this...
- The Bible Code: Enigmas for Dummies - Do messages hidden within the Bible really predict the future?Skeptoid: Critical Analysis of Pop Phenomena
The Bible Code: Enigmas for Dummies
See how much more inviting this looks to the reader? There were many examples of these short edits dating back to 2006, some were even shorter. Not only does this look more appealing, but a reader can click on the link about the Skeptoid article as well as to the Wikipedia page for Skeptoid.
When I started Project Skeptoid in November 2011 there were 206 episodes to link to. By the end of November there were 38 references to the podcast on the corresponding Wikipedia pages. His hit results were 1.4% of the total views to www.skeptoid.com coming from Wikipedia. I'm not going to tell you how many hits 1.4% is because I'm measuring this purely by percentages. Lets just say it is several thousand views.
So I cleaned up a few of the already existing citations and added 10 more Wikipedia pages to the hit count which by the end of January 2012 became 48 Wikipedia pages. He had been adding more episodes all this time bringing his count up to 296 episodes. A look at his stats again and the month of January 2012 hit 1.82%.
Remember that not all of these episodes can have a Wikipedia page, they were humor episodes or were already rejected by editors, and some were student question or listener feedback episodes which I haven't bothered trying to link to.
As you can see, there is potential for improvement. Several thousand readers are following the links from Wikipedia to the episode on Skeptoid.com. These readers probably aren't normal Skeptoid listeners otherwise they would have just gone to his site. The goal is to improve Wikipedia as well as the skeptical/critical thinking exposure to NOT the choir (you and I) but to the general public. In this example I think it shows we are making a difference.
In February 2012 I added a few changes and new links but I'm going to take a break from reporting numbers at least till the March or April stats are available. I want to really make some head-way on the Google document, my goal next time I check the stats I will see 3 or 4%. I could really use some help, please contact me if you can, I will train. susangerbic@yahoo.com
Its a bit of an uphill battle to keep measuring against percentage of total Skeptoid.com views as the site becomes more popular he will naturally have more hits without Wikipeida. If someone can think of a better way of analyzing these numbers I would appreciate your thoughts.
By the way,
Regular Skeptoid fans you might be interested in which Wikipedia page is causing the majority of views back to his website.
HINTS - A very cold topic, this page only has an external link to follow and it has been overwhelmingly number one for both November and January with 5% of the total hits. This Wikipedia page receives about 130K views a month Click for Answer.
I always revert blogs used as citations in any of the science articles I edit (I use an anonymous name, I'm one of the top editors at Wikipedia, and I've been blocked and taken up on charges at Arbcom so many times that I'm sure I'll set a record). The reason I do so is the old "what's good for the goose is good for the gander." If in the articles I edit, say anti-vaccine movement, one person may use Respectful Insolence for a "pro-vaccine" POV, i.e., the scientifically supported POV. Then the anti-vaccine lunatics will them leap onto scene using Age of Autism as their "reliable source."
ReplyDeleteI never revert or delete wholesale external links. But in articles that don't get a lot of attention, the external links do include a lot of spam.
In Reliable sources for medical articles which is the go-to rules (err….recommendations) for sourcing medical articles, blogs are out. Even in the fringe articles like homeopathy or remote sensing, most of the skeptical editors will delete blogs. There are a few that are kind of acceptable, like Quackwatch, but even there, it's a battle to use it.
I have written occasionally about the abysmal quality of some of the medical articles on Wikipedia. The fact that it's so high in google rankings for just about anything medical is appalling to me. Anyways, glad I found this blog. Another one I'll be reading every day. :)
Hi Michael! I'm wondering what your thoughts are on the site sciencebasedmedicine.org. It's edited by eight MD's, and most of their articles are reviews of literature. My understanding is that this would count as a "secondary source" and would be at least as valid a source as Ben Goldacre's "Bad Science" column/blog, which is noted as a reliable source in the link you provided. I've linked to sciencebasedmedicine.org in the past on Wikipedia, and the links haven't been taken down by anyone so far. But I'm curious to know if you would consider that to be a blog and not a valid source.
DeleteI apologize for not replying to you sooner. I post at various skeptics' blogs, so sometimes I miss replies to my replies!
DeleteI use Science Based Medicine all the time for my own blog all the time, and I use it whenever I need to debunk something. I am absolutely opposed to using blogs as a citation on Wikipedia, and at any medical article I edit, I remove all blog citations. I will, however, use SBM as a place to find the right citation or to assist me in interpreting a cite. But even though SBM is about the most accurate evidence based medicine site on the internet, it is not peer-reviewed, and, in the end, their opinions are just opinions, no better than what you or I can give.
Now, there is a difference between the External Links section of an article vs. an inline citation. In the latter, I may or may not take down links to blogs like SBM, depending on a judgement call. I actually despise the External link section, because it becomes a catch-all for every single nut job out there. Usually, the only external links that I will add will be to something critical like the website of an organization or something similar. So, I'm a bit of an absolutist on external linking. And I am an absolute absolutist in banning blogs of any sort as an inline citation.
As for Ben Goldacre, I believe the guideline does not allow for his column to be used as an actual citation, but as a source to determine the possible quality of a citation. I assisted in developing parts of that guideline, and I'm almost certain that no one would consider "Bad Science" to be a citation in a medical article.
On the other hand, and this is important, in debunking a claim, there is a bit more leeway. For example, Homeopathy mostly uses medical reliable sources, but there are a couple of exceptions to get just the right quote. Goldacre is always good for a snarky quote that can be used in an article. But if you're going to write that "a meta analysis of clinical results of double blind studies of homeopathy indicate that it is just water", then you need the actual cite from Cochrane Reviews rather than the tertiary or quaternary interpretation of that cite.
I've got to tell you that in several years of editing science and pseudoscience articles on Wikipedia, I have never had to resort to a blog as the citation. Not once. I've always found a perfect primary or secondary source, usually a review article. My problem with Wikipedia is a nasty temper towards the POV pushing twits.
Thanks Michael for your comment and vigilance on WP. Thanks for giving me an open to clarify a bit more.
ReplyDeleteBlogs generally are not good sources for WP. For example if a no-named "reporter" on Huff Post puts up their opinion on Sally Morgan (psychic to Di) [but didn't think to tell her to always wear her seat belt] then that isn't really a good source to quote in WP.
BUT
If a noted "reporter" states their opinion on Sally Morgan (who IMO is a grief vampire) then it might be okay.
AGAIN
If a reporter interviews a notable person who could be considered an expert on psychics then probably that blog would be okay to cite. As long as the WP editor makes it clear who is being quoted. Such as "according to [[Mentalist]][[Banachek]] working for [[JREF]] who says 'it is difficult to tell the difference between a psychic on stage and a mentalist performer at [[The Magic Castle]]'"
I'm making this up BTW
Also want to be clear
Skeptoid is not a blog. I've had this discussion with other WP editors and we came to the formal decision that as long as Dunning is talking about paranormal topics his blog can be cited.
Skeptoid is also on the verge of being a classroom source (free I believe). He is becoming a non-profit and classrooms from elementary to college will have access to the material as well as the exercises. Pretty exciting!
So to be clear. A blog is usually an opinion that is unsourced and not able to be cited on WP. There are exceptions to this rule, and if you add one in to a WP article be prepared to back up your cite. Not with anger or edit wars, but with discussion. You don't know when you might encounter that editor again.
Susan
The issue of whether a blog or a podcast can be a reliable source is an interesting one, there are probably multiple answers to that question.
ReplyDeleteA way to sidestep this issue a bit, but still generate links to worthy podcasts, is via interviews. If you can support a quote or a piece of material in a biography using an interview sourced to a podcast, it gives you a good reason to cite the podcast. I've done this in almost every biographical article I've written. It's also good for readers, because the podcast gives them a further place to go to learn more about that person. (Of course, this strategy doesn't work well with Skeptoid, which does no interviews).
Let me point out another potential issue, one that is more easily worked around. If you repeatedly add links to the same website to Wikipedia over a short period of time, there are anti-spam robots that will pick up on this behavior. There are also manual ways that other users can report spam.
I noticed this recently in regards to my linking to Skepticality on Wikipedia. I found this robot created report about Wikipedia links to Skepticality.com because it linked to my user page. It appears last November someone became interested in whether there was an effort to spam Wikipedia with links to Skepticality. Perhaps they noticed that I spent one afternoon going around and fixing broken Skepticality links to adjust to a site redesign that Derek did. Who knows?
It appears they decided there was no concern, certainly they never took it up with me. I'm pretty sure all the links I added were very organic and within the rules.
But bottom line: go slow with Project Skeptoid, or you might run afoul of anti-spam robots.
Pro Tip Takeaway: Occasionally go to your User Page, and click "What links here" on the left hand side. Sometimes you will find out people are talking about you in "back stage" areas of the WIkipedia that you had no idea existed.
This comment has been removed by the author.
ReplyDelete