The American Society for Biochemistry & Molecular Biology has drawn up guidelines for a new curriculum in undergraduate education. The complete recommendation can be found at Recommended Curriculum for a Program in Biochemistry and Molecular Biology in the Journal Biochemistry and Molecular Biology Education (BAMBED).
Under the list of "Skills that biochemistry and molecular biology students should obtain by the time they have finished their undergraduate program," there are a number of motherhood type statements. One of them is "Ability to assess primary papers critically." We've been discussing these required skills for the past few months. I've questioned the wisdom of teaching undergraduates how to critically evaluate the scientific literature because I think it's a skill that only comes after a lot of experience in the discipline.
There are many confusing papers out there and it's difficult to decide what's right and what's wrong. We can give students our opinion but that's not the same as teaching them how to critically evaluate a paper.
Here's an example of how difficult it is to read the scientific literature. A recent paper by John Stiller (2007) promotes the idea that plants are more closely related to animals than fungi. Here's the abstract.
Evolutionary relationships among complex, multicellular eukaryotes are generally interpreted within the framework of molecular sequence-based phylogenies that suggest green plants and animals are only distantly related on the eukaryotic tree. However, important anomalies have been reported in phylogenomic analyses, including several that relate specifically to green plant evolution. In addition, plants and animals share molecular, biochemical and genome-level features that suggest a relatively close relationship between the two groups. This article explores the impacts of plastid endosymbioses on nuclear genomes, how they can explain incongruent phylogenetic signals in molecular data sets and reconcile conflicts among different sources of comparative data. Specifically, I argue that the large influx of plastid DNA into plant and algal nuclear genomes has resulted in tree-building artifacts that obscure a relatively close evolutionary relationship between green plants and animals.This position is contrary to a whole lot of work that has been published over the past several decades. I don't think very much of this paper and neither do John Logsdon of Sex, Genes & Evolution [Promoting Plants at the Expense of Fungi?] and Ryan Gregory of Genomicron [Discovery wants to "demote" fungi]. Read their blogs to see why we're skeptical about this paper.
How do you explain this to undergraduates? How can you teach them to critically evaluate such a paper when, on the surface, it seems perfectly reasonable and the data seems sound? I submit that most of us work within a model of how we think the history of life has developed over millions of years. That model is based on reading hundreds of papers and getting a "feel" for the data. Some papers are rejected and some are given more credence and this is based on all kinds of intangibles—including the reputation of the authors. Can undergraduates be taught such a thing? I don't think so.