Saturday, January 07, 2012

Should Undergraduates Study the Primary Scientific Literature?

The short answer to the question posed in the title is "yes" but there are many caveats. One of them is that it depends on what level you are teaching. In my opinion, the value of exposing science students to the primary scientific literature (papers) increases as students progress from first year to the year they graduate. Students in their final year of a science program will gain a lot from being exposed correctly to the scientific literature but students in introductory course will hardly benefit at all—and may, in fact, be harmed if it takes time away from learning basic principles and concepts.

It is important to teach critical thinking and it's important to focus education on basic principles and concepts. Most of the basic principles and concepts in a discipline have been developed over several decades. The work that led to those ideas is (usually) in the primary scientific literature but you can't learn the concept by just reading a few key papers. Evolution is a good example but so is our understanding of how cells generate energy from proton gradients, how enzymes work, and how the information in messenger RNA gets translated into proteins.

I find it helpful to remind myself from time to time that the vast majority of the students I teach will never be scientists and many of them aren't really interested in how to do scientific experiments. They will become average citizens in all kinds of careers that have nothing to do with the basic sciences. Our goal is to make them scientifically literate so they will understand why evolution is true, why homeopathy is bunkum, why they should vaccinate their children, and why humans are behind global climate change. I don't think we can achieve that goal by focusing on the primary scientific literature, especially in the early years of undergraduate education.

C.R.E.A.T.E. is a education project funded by the United States National Science Foundation (Grant No. 1021443). It's goal is "transform understanding of science" by using the primary scientific literature as a teaching tool. Here's how they describe their approach ...
The C.R.E.A.T.E. (Consider, Read, Elucidate the hypotheses, Analyze and interpret the data, and Think of the next Experiment) method is a new teaching approach that uses intensive analysis of primary literature to demystify and humanize research science for undergraduates. Our goal is to use the real language of science—the journal article—as an inroad to understanding “who does science, how, and why?” At the same time, we wish to help students (1) experience authentic processes of science, in particular discussion/debate about experimental data and their interpretation (including ‘grey areas’), (2) recognize the creativity and open-ended nature of research, and (3) see the diversity of people who undertake research careers (i.e. not just the genius/geeks of popular culture). As a complement to teaching based on textbooks, which tend to oversimplify the research process, C.R.E.A.T.E. teaching focuses on on authentic published work--peer reviewed journal articles—with students reading either series of papers produced sequentially from individual labs or series of papers from different labs focused on a single line of research.

By reading/analyzing a set of papers published in series from a single lab, students experience the evolution of research projects over a period of years. Using newly-developed C.R.E.A.T.E. pedagogical tools, that encourage multiple approaches to the material (concept mapping, sketching, visualization, transformation of data, creative experimental design) students gain deep understanding of the methods (and biological content/principles) that underlie each individual experiment of the paper. In class, we emphasize scientific thinking--focusing on understanding both why and how each part of the study was done, by examining the hypotheses underlying each aspect of the study, and analyzing/discussing the data represented in each figure and table. Students learn to interpret complex data, draw conclusions, debate interpretations, and re-represent data (e.g. represent tabled data in graphic form) to aid understanding. Content knowledge is reviewed as students consider the principles underlying the techniques used, as well as the overall context of the scientific question being addressed (e.g. a module focused on regeneration would likely include review multiple aspects of cell division, cell differentiation, gene expression and stem cells, drawing on information students learned in other classes and helping them to apply it in a real-world research situation). C.R.E.A.T.E. students thus learn a variety of transferable learning skills that can be applied to complex scientific reading they do in the future. Students design their own proposed followup experiments at several points in the semester, and debate each other’s proposed studies in a classroom exercise modeled on activities of bona fide scientific grant panels. Such discussions reveal the research process to be openended, with multiple branch points or possible “next directions to go;” thus much less linear and predictable than many students expect. Late in the process, students generate a short list of questions for paper authors that are sent as an email survey to each author (not simply the PI). Responses from multiple authors provide unique behind-the-scenes insight into “the people behind the papers,” humanizing the research experience and showing researchers to be complex individuals much like the students themselves.
Here's an example based on Pattern formation during regeneration in planaria.

This is an approach that views experiment as the primary focus of science whereas I tend to see science as a much broader way of knowing. The C.R.E.A.T.E. approach to undergraduate education emphasizes the doing of science rather than the understanding of the results and how they fit into a bigger picture. It probably does a good job of looking at "trees" but not so good a job when it comes to seeing the "forest."

I don't know the correct balance between teaching principles, ideas, and concepts and teaching the experimental approach taken by actual research scientists in their day-to-day activities. There's no question that lab courses are extremely important but I'm quite skeptical about bringing the study of experimental techniques into the lecture courses if it take time away from the conceptual understanding of the discipline.

[Hat Tip: Sandra Porter at Discovering Biology in a Digital World: Learn how to use scientific articles in education at the C.R.E.A.T.E. June workshop]


  1. I disagree with the "should" part of that. If a teacher has a good way of working that in, then fine. But I don't see it as a curricular requirement.

    Undergraduates are mostly acquiring concepts and methods, rather than empirical facts. Research papers that give particularly good accounts of method might have some value. Acquiring concepts should still be the main goal at the undergraduate level

  2. Primary literature can be very valuable in helping students develop a more robust idea of why science is reliable way of knowing. The literature on teaching the Nature of Science is not very flattering to a concept only approach.

    I use some Primary literature with my intro students to help them see where the concepts are coming from. Larry is absolutely correct you can not build an entire course around that approach, but it is a valuable tool, alongside labs and other traditional methods.

  3. The reason why science is a reliable way of knowing is because it develops consensus over time.

    Skepticism is a major part of the scientific way of knowing and that means being skeptical about what appears in the scientific literature. It's almost impossible to pick a scientific paper that was so definitive that nobody questioned the results and they became instantly incorporated into a basic concept or principle.

    The best thing that students can learn from examining the scientific literature is that it is very unreliable. The results in most papers will never be reproduced or they will turn out to be misleading. That's why we have so many genuine scientific controversies.

    We want students to understand that just because a newspaper writes up the latest health crave—eat more tomatoes—doesn't mean that it's true even when it's published in a leading journal.

    That's not how science works.

    I use the primary literature to teach controversies in order to illustrate the real way that science works. This semester, for example, we'll be looking at various papers that present conflicting views on the evolutionary advantage of sex and the amount of junk in your genome.

    We've got to stop teaching that once something is published in a scientific journal it becomes the gospel truth. We've got to abandon the idea that the primary scientific literature is better than textbooks at explaining basic concepts and principles. If you want to understand evolution, for example, read Futuyma's textbook, not the latest papers on evolutionary biology.

  4. Reading primary literature by students should be minimized because the gain to effort ratio is too low (close to zero in fact). In ideal world they do it on their own when they need it.

  5. Yes, they should. What undergraduates don't realize is that science is the great equalizer and everyone doing cutting edge work is equally in the dark. You don't actually need a fancy degree to make a real contribution- just real intellectual engagement.

  6. problem with primary literature, at least in molecular biology, is that a considerable amount of expertise is required to interpret, particularly with respect to techniques used, and with respect to previous work conducted on same topic. Year 3 and 4 undergrads should be increasingly exposed to primary papers esp. while doing thesis research projects in 4th year...but cant see how it could be foundation for teaching basics. Graduate degree is where the real learning begins with respect to research, seems to me.

  7. I have found one scientific paper that is sufficiently nontechnical that it is useful for environmental science classes for undergraduates. This is a hilarious paper on the concept of the tragedy of the commons, applied to teaspoons.

    BMJ volume 331, 24-31 December 2005 (accessible for free at

    "The case of the disappearing teaspoons: longitudinal cohort study of the displacement of teaspoons in an Australian research institute" by Megan S.C. Lim et al.

  8. This Thursday I'll be attempting to teach my students about science using the primary literature. That's a big topic for a single 1.5 hour class, but I'm confident my main points - focussing mostly on how to read a paper to learn what the authors did, why, and how successful they were - will get through.

    It's a 4th-year class (actually labelled a "capstone" class, intended as one of the last classes a student will take in an undergraduate program) in Environmental Science. I'm taking a gradual approach, starting with an in-class discussion of a single paper, followed by a small assignment in which students identify key features of a paper taken from the reference list of the first paper, and concluding with that exercise repeated but with a paper chosen by each individual student.

    The first paper is a recent publication by a former office- and lab-mate of mine; I have some familiarity with the study, which I think makes it easier on myself when it comes to pointing out key phrases in (e.g.) the Methods & Materials section. I am a PhD student, teaching this class as part of the Mentored Teaching program here; my faculty mentor is also a co-author on the paper, along with my graduate advisor. Access to direct experience relating to this paper is another feature of the planned discussion.

    The second paper will be assigned to students at the end of the in-class discussion; they'll have 5 days to write a 1-page description that lays out the primary objective of the paper and the major line(s) of evidence used to acheive that objective.

    The third paper will incorporate the action of searching out a paper, using the library resources here, in addition to the activity of identifying the objective and major evidence / arguments.

    The overall learning objectives of this series of exercises are
    1. The students will learn the major features of peer-reviewed papers that serve as diagnostic features for peer-reviewed papers, and the general structure of a typical paper.
    2. The students will be able to describe the major objectives of a published scientific study, and evaluate whether or not the paper acheived those objectives.
    3. The students will be able to find and retrieve scientific articles using commonly-employed tools (Web of Science, University of Saskatchewan Library website) based on topic searches.

    I think that a student graduating with a degree in science - a B.Sc. - should have as a necessary skill the ability to identify the difference between a peer-reviewed scientific study and other types of publications, and have a fair understanding of the strengths and limitations of those peer-reviewed studies in the current literature. I think my class will provide that skill and that knowledge.

    The first paper:
    Laird, B.D., Peak, D., Siciliano, S.D., 2011. Bioaccessibility of Metal Cations in Soil Is Linearly Related to Its Water Exchange Rate Constant. Environmental Science & Technology 45, 4139-4144.

    I welcome any suggestions or advice anybody here may have for me - and I'll check back in after Thursday's class to let you know how it went.

    comedy bonus: the word verification for this comment is "mania"

    1. Sorry for the delayed reply.

      "Well", to sum it up in a single word. Over the two assignments, all of my students (40) were able to provide me with a short summary of the objective, conclusions, and main evidence of a peer-reviewed paper. Some of them made some rather unusual (in my experience) formatting choices, but all covered the main points well.

      This means the assignments might have been too easy. Nonetheless, I think this demonstrates that, at least among my students, being able to identify what a scientific paper is about is a skill they have, as fourth-year undergrads in a science field.

      I'd like to re-iterate that they were not asked to critique a paper (i.e. evaluate and judge), only to describe some key features of a paper.

  9. These papers are great, I love the idea of looking at the development of a line of scientific thought of time, but they are useless to me. I teach at a community college in the american bible belt. The students I teach are non-science majors, they generally report that their last science class was in the ninth grade, for many my class is either the last or the next to the last science class they will ever take. While many have some level of interest in biology, most are just trying to do the minimum to get by. Go to your local supermarket, grab 30 random people - those are probably my students.

    Primary literature is hard with these students as they are simply not prepared to critically read a paper that assumes they know what a gene is (they only have a vague sense when they come in to my class), how pH works or what an endoplasmic reticulum is. The pre-reqs to get into the class are basically remedial math & english.

    I could use a database of simple, easy to understand research articles. One that doesn't contain words like "dorsoventral axis of the Xenopus retinotectal system". Yes, I can search Pubmed myself and often do, but the reality is that I don't have time to comb through papers like I used to. I have a feeling that many teachers, and especially High School teachers, are in the same boat. These papers need to be (1) clearly written for the average person (actual average, not what we think "should be average") (2) contain figures that a ninth grader could interpret (attempts to explain log scales have derailed entire class periods) and (3) be understandable on their own, without requiring other papers to be read. Oh yes, AND be open access.

    For instance:

    Anyhow - I am collecting these if you have any in mind. Send me the links at jeffreydotmahratgpcdotedu. Someday I hope to have a searchable database of these that primary school teachers might find useful.