I teach a course called "Scientific Controversies and Misconceptions." (You won't be surprised to learn that one of the main topics is evolution vs. creationism.) The most difficult part of the course is teaching students to be skeptical about the scientific literature. The reason why this is difficult is because the main focus is evidence based reasoning and science is an important way to gather reliable evidence. It's uncomfortable to have to mention that errors and fraud are not uncommon in the scientific literature.
I haven't figured out a way to teach students to read the scientific literature with the same skeptical perspective as those of us who have been doing it for decades. It seems as though this is a skill that must be learned through experience and can't be taught.
Almost every issue of Science has a retraction or a caution about a recently published paper. It's easy to get the impression that retractions are on the rise. Is this true? Yes, it is. Here's the data from PubMed Retractions. It shows a 15-fold increase in the number of retractions per publication.
This is quite alarming. Does it mean an increase in the number of papers containing significant errors? Does it mean an increase in scientific fraud and misconduct? Or, does it mean that the scientific community is becoming more active in removing suspect papers from the literature?
Here are articles from two bloggers who discuss these issues:
Orac at Respectful Insolence: Scientific fraud and journal article retractions.
Ed Silverman at Pharmalot: Retractions Of Scientific Studies Are Surging.
In both cases the focus is on the medical literature but the problem is not confined to medical journals. Personally, I think this is just the tip of the iceberg. For every article that's retracted there are a hundred articles that contains serious errors and misconceptions.
3 comments :
I'd be interested in an outline and reading recommendations of the course you teach if it's available.
I am leaning toward the interpretation that it's all good news and that the detection rate is increasing. Although I do believe that there is a slight increase in fraud, the bigger factor IMHO is that there are fewer and fewer ecological niches where there are few people who care to compete with you and reproduce you results directly. As a result, more bullshit eventually gets uncovered.
I agree that the rise in retractions probably represents better detection more than anything else. The problem is that papers are typically only retracted if they are high-profile enough that there's some reason to go after them. If they appear in This One Journal I've Never Heard Of, they'll never attract much attention and will never be retracted. There's just too much junk floating around. The problem is not deliberate fraud, which is very rare, so much as it is the Feynman effect -- "you are the easiest person to fool" -- and the strong incentive to publish positive results. Life science VC Bruce Booth wrote a great post about a related problem a while back (see http://lifescivc.com/2011/03/academic-bias-biotech-failures/); his contention is that 50% of academic studies published in top-tier journals can't be replicated in an industrial lab. Easier to describe the problem, though, than to solve
Post a Comment