Science journal tries to fix problems with transparency and trustworthiness

The editors of Science recognize that they have a problem. They aren't very transparent or trustworthy. This is true. These same editors have been guilty of publishing and promoting lots of poor quality science over the past few years. Three examples come to mind ...

• Arseniclife: Science published a ridiculous claim that arsenic could replace phosphorus in DNA. That paper has been refuted but never retracted.
• Ardipithicus ramidus: Science fell for the authors' hype.
• ENCODE: Science falls for the hype promoted by ENCODE leaders. Editorial and feature writers announce the death of junk DNA

Don't worry. The editors have been working hard to fix the problem. After a year of study they announce their solution in the June 3, 2016 issue in the lead editorial: Taking up TOP. The author is the current Editor-in-Chief, Marcia McNutt.

She begins with ...

Nearly 1 year ago, a group of researchers boldly suggested that the standards for research quality, transparency, and trustworthiness could be improved if journals banded together to adopt eight standards called TOP (Transparency and Openness Promotion).* Since that time, more than 500 journals have been working toward their implementation of TOP. The editors at Science have held additional retreats and workshops to determine how best to adapt TOP to a general science journal and are now ready to announce our new standards, effective 1 January 2017.

So, what is TOP and how is it going to make Science more trustworthy? Does it involve firing some well-known writers and editors? Does it involve better reviewers?

Nope. TOP is just a way of making sure that raw data is available to other researchers.

... we believe the benefits of requiring the availability of data, code, and samples on which the authors' interpretations rest are worth the effort in compliance (and in some cases in adjusting data ownership expectations), while acknowledging that some special circumstances will require exemptions. This practice increases transparency, enables reproducibility, promotes data reuse, and is increasingly in line with funder mandates. We are also requiring the citation of all data, program code, and other methods not contained in the paper, using DOIs (digital object identifiers), journal citations, or other persistent identifiers, for the same reason. Citations reward those who originated the data, samples, or code and deposited them for reuse. Such a policy also allows accurate accounting for exactly which specific data, samples, or code were used in a given study.

That's not going to fix the main problem.

---

Nature journal tries to fix the problem of a scientific literature that's too complex to understand

I recently posted some thoughts on the complexity of the scientific literature noting that many papers are simply too difficult to understand. This includes papers that are well within my areas of interest [How to read the scientific literature? and The scientific literature is becoming more complex].

Nature journal recognizes that there's a problem. A few weeks ago (June 16, 2016) they published a brief comment on Nature distilled.

They begin by describing the problem ...

Any journal that tries to publish the most important results that it is sent, in all fields of science, will run into the same problem. Every bit of our output, we hope, is useful and interesting to somebody somewhere. But even the most optimistic of our editors would concede that the pool of readership for each of these specific advances is only a small subsection of our audience, professional researchers included. To the outside world, science is science. To those who read Nature, science is a multiplicity of specialisms — and specialists.

We know that most of you are specialists, and that you don’t read most of what we present to you. You’re busy people. It is hard enough to follow the literature that you need to read. Even the titles of research papers in an unfamiliar field can look incomprehensible. But if you’re anything like us, one reason you got into science in the first place was curiosity about the world — and not just the tiny piece of it that you now focus on. Wouldn’t it be useful and interesting to keep better track of the rest? Or at least, the rest that is published in Nature, and therefore already judged to be important?

Let's make one thing clear. It's not just the complexity of a paper that's the problem and it's not just that the science isn't explained in easy to understand sentences. There's also the more serious problem of content. Sometimes the papers are hard to understand because the significance of the results is exaggerated and its importance is not placed in proper context.

The ENCODE papers are good examples of this problem. It wasn't easy to understand that they did but, more importantly, it wasn't easy to understand the significance of their results because the authors didn't explain their results very well. They made unsubstantiated claims.

Here's how Nature hopes to fix the problems they identified.

We think so, and this week we begin an experiment to see how many of you agree. We have revisited 15 recently published Nature papers and asked the authors to produce two-page summaries of each. The summaries remain technical — these are not articles suitable for the popular press — but they try to communicate both the research advance and why it matters. The authors of these papers have been enthusiastic — they want the broadest possible readership — and we thank them for their cooperation. Now we want to know what you think. The first three summaries are published online this week (see go.nature.com/1uhcy3x). The rest will be released in the coming weeks. Please take a look. Be brave — pick a topic that you expect to struggle with — and then fill in the online survey to let us know what you think. The rest will be released in the coming weeks. Please take a look. Be brave — pick a topic that you expect to struggle with — and then fill in the online survey to let us know what you think.

I looked at two papers that were about biology and I didn't think the summaries added anything to my understanding. That's partly because the papers weren't that hard to understand in the first place if you were just satisfied with knowing what they did.

Both papers raised lots of questions in my mind about the biological significance of the studies and whether they were accurate and reproducible. The author summaries didn't help much. [Non-coding recurrent mutations in chronic lymphocytic leukaemia and DNA-dependent formation of transcription factor pairs alters their binding specificity].

If the scientific literature is difficult to understand, and it is, then there's a problem with the authors. They aren't able to explain what they did in a reasonable manner and they aren't able to place their work in a proper context so we can evaluate the significance of the result. Asking them to try again (and doubling their citations) is probably not going to help.

The ENCODE authors couldn't do it.

It's a lot like asking the fox to guard the henhouse.

---

TED-Ed misrepresents epigenetics

TED-Ed is the educational arm of TED. Here's what TED says about itself and about TED-Ed ...

TED believes passionately that ideas have the power to change attitudes, lives, and ultimately, the world. This underlying philosophy is the driving force behind all of TED’s endeavors, including the TED Conferences, TEDx, TED Books, the TED Fellows Program, and the TED Open Translation Project. With this philosophy in mind, and with the intention of supporting teachers and sparking the curiosity of learners around the world, TED-Ed was launched in 2012.

TED-Ed is TED’s youth and education initiative. TED-Ed’s mission is to spark and celebrate the ideas of teachers and students around the world. Everything we do supports learning — from producing a growing library of original animated videos , to providing an international platform for teachers to create their own interactive lessons, to helping curious students around the globe bring TED to their schools and gain presentation literacy skills, to celebrating innovative leadership within TED-Ed’s global network of over 250,000 teachers. TED-Ed has grown from an idea worth spreading into an award-winning education platform that serves millions of teachers and students around the world every week.

Can scientists describe what they're doing to a fifth grader?

I'm working on a review of "The Gene" by Siddhartha Mukherjee. It raises a huge number of issues about science writing and the conflict between producing a bestseller and educating the public about science.

As part of the research for that blog post I've been reading all the reviews of his book and I came across an interview with Mukherjee on the Smithsonian website [Siddhartha Mukherjee Follows Up Biography of Cancer With “An Intimate History” of Genetics].

Here's an interesting answer to an important question ...

Research for a book

I'm on sabbatical this term, working on a possible book whose working title is "What's in Your Genome?: 90% of your genome is junk."

Here's some of the most important books I've read (or re-read) in the past few months.

I've also read a lot of papers and scribbled notes on what's important and what's bullshit not. The most difficult part about keeping up with the scientific literature is organizing it in some meaningful way so you can quickly find it again if you need to—something I do just about every day.

Everyone has their own method. What works for me is to keep an electronic reference with key words and links to a file folder on a particular topic. (I use EndNote.) Here are the folders with all the papers I've been reading in the past few months.

I don't know how other authors behave but for me the most difficult thing about writing a book is organizing my thoughts and planning how to present them in the most effective manner. I tend to write too much on too many topics so the initial drafts usually have to be pared down considerably. Keeping that in mind, what are YOUR favorite topics?

---

John Oliver teaches us to be skeptical of scientific publications

We all know that the purpose of education should be to teach students how to think critically. We're not doing a very good job. Take biochemistry, for example. We spend a lot of time transferring information from lecture notes to student notes and then examining students on whether the transfer has worked. We think that teaching students to read the primary literature will make them better scientists when, in fact, teaching them to be skeptical of the primary literature is what's really necessary.

The ENCODE fiasco is just one of many examples where the scientific literature got it wrong. We need to make sure that our students appreciate the important parts of science; namely, the necessity of repeating experiments and the value of scientific consensus. Our students, and many of my colleagues, are prone to hype and promotion just like every one else but that's exactly what critical thinking is supposed to avoid. And it's exactly what proper science—no matter how you define it—is designed to overcome.

If students and scientists are having trouble with these concepts, imagine how difficult it is for the general public. How are they supposed to know that not every "breakthrough" is a real breakthrough and not every new study is correct?

John Oliver did an excellent job of explaining the problem on a recent (May 8, 2016) episode of Last Week Tonight. Watch it. It's worth 20 minutes of your time. The last bit on "Todd Talks" is classic.

---

Teaching about genomes using Nessa Carey's book: Junk DNA

Nessa Carey's book about junk DNA is an embarrassment to the scientific community [Nessa Carey doesn't understand junk DNA] [The "Insulation Theory of Junk DNA"].

Today, while searching for articles on junk DNA, I came across a review of Nessa Carey's book published in The American Biology Teacher: DNA. The review was written by teacher in Colorado and she liked the book very much. Here's the opening paragraph,

The term junk DNA has been used to describe DNA that does not code for proteins or polypeptides. Recent research has made this term obsolete, and Nessa Carey elaborates on a wide spectrum of examples of ways in which DNA contributes to cell function in addition to coding for proteins. As in her earlier book, The Epigenetics Revolution (reviewed by ABT in 2013), Carey uses analogies and diagrams to relate complicated information. Although she unavoidably uses some jargon, she provides the necessary background for the nonbiologist.

The author of the review does not question or challenge the opinions of Nessa Carey and, if you think about it, that's understandable. The average biology teacher will assume that a book written by a scientist must be basically correct or it wouldn't have been published.

That's not true, as most Sandwalk readers know. You would think that biology educators should know this and exercise a little skepticism when reviewing books. Ideally, the book reviews should be written by experts who can evaluate the material in the book.

Now we have a problem. The way to correct false information about genomes and junk DNA is to teach it correctly in high school and university courses. But that means we first have to teach the teachers. Here's a case where professional teachers have been bamboozled by a bad book and that's going of make it even more difficult to correct the problem.

The last paragraph of the review shows us what influence a bad book can have,

As a biology teacher who enjoys sharing with students some details that go beyond the textbook or that challenge dogma, I enthusiastically read multiple chapters at each sitting, making note of what I cannot wait to add to class discussions. “Junk DNA” may be a misnomer, but Junk DNA is an excellent way of finding out why.

Oh dear. It's going to be hard to re-educate those students once their misconceptions have been reinforced by a teacher they respect.

---

Another failure: "The Mysterious World of the Human Genome"

The Mysterious World of the Human Genome
by Frank Ryan
William Collins, an imprint of Harper Collins, London UK (2015)
ISBN 978-0-00-754906-1

This is just another "gosh, gee whiz" book on the amazing and revolutionary (not!) discoveries about the human genome. The title tells you what to expect: The Mysterious World of the Human Genome.

The author is Frank P. Ryan, a physician who was employed as an "Honorary Senior Lecturer" in the Department of Medical Education at the University of Sheffield (UK). He's a member of The Third Way group. You can read more about him at their website: Frank P. Ryan.

The best TED talk ever

I'm not a fan of TED talks. Comedian Will Stephen has figured them out and he gives a perfect example of everything that's wrong with a TED talk. Watch "How to sound smart in your TEDx Talk."

---

The Fermi Paradox: Are we alone in the universe?

All available evidence suggests that we are quite likely the only advanced life form in the Milky Way galaxy. Maybe our planet harbors the only life in the entire universe.

---

A DNA quiz

Jerry Coyne discovered a Quiz on DNA. He calls is a so-so quiz on DNA. He says that one question is really, really, dumb. I disagree, I think there are several dumb questions.

I tried it and got a score of 19/19 in just under four minutes. This is misleading since you have to get every question right before continuing on to the next question. I had to anticipate what the authors wanted in order to proceed.

Try the quiz yourself before reading any further. There are spoilers below!

Richard Dawkins makes a mistake when describing why gene trees are evidence of evolution

Back in 2010, Richard Dawkins was answering questions on Reddit and one of the questions was "Out of all the evidence used to support the theory of evolution, what would you say is the strongest, most irrefutable single piece of evidence in support of the theory."

There are several ways to answer this question. Personally, I would take a minute to explain the difference between the "theory of evolution" and the history of life. I would point out that evolutionary theory includes things like Darwin's natural selection and there is overwhelming evidence proving that natural selection exists and operates today. The entire field of population genetics, which included other mechanisms of evolution such as random genetic drift, is massively supported by thousands of published papers in the scientific literature. There is absolutely no doubt at all that the current basic tenets of evolutionary theory are correct.

An undergraduate biochemistry lecture converts an atheist to Christianity

I'm reading Creation or Evolution: Do we have to choose? by Denis Alexander in preparation for our discussion next Friday at Wycliffe College on the University of Toronto downtown campus [Discussing the conflict between science and religion with Denis Alexander].

Denis Alexander is a biochemist at the University of Cambridge (UK). I thought I'd share one of the stories in his book.

At the church I attend in Cambridge we baptised an undergraduate in the natural sciences who had come to a personal, saving faith in Christ from a completely atheistic background. As is usual in our church, just before being baptised she explained publicly to the whole congregation how she had become a Christian, telling us she had become convinced there must be a God while sitting through a standard biochemistry lecture, hearing the amazing story of how two meters (about six feet) of DNA are packaged into a single cell. Of course the lecturer was not talking in religious terms at all, but she described to us how the beauty of that engineering feat overwhelmed her as she listened, giving her the deep intuition there must be a God, so leading her onward in he personal pilgrimage to put her trust in this creator God through Christ. Truly natural theology at work!

That got me thinking. I've been describing chromatin and packing in my textbooks since the first version in 1987. There must have been several hundred thousand students who have read my descriptions since then.

I wonder how many I've converted?

---

On the (false) significance of a liberal arts education

Joshua Kim posted an article on Inside Higher Education last month (Dec. 8, 2015). The article described his answers to some questions he was being asked in a interview [How Would You Answer These 9 Reimagine Education Questions?].

Here's one of the questions and his answer ...

Question 4: Is there an innovation/idea/movement/methodology that excites you in terms of the future of education?

Yes. A liberal arts education.

Rethinking medical education at the University of Toronto

Watch two medical educators from my Faculty of Medicine at the University of Toronto. They are being interviewed by Steve Paiken of The Agenda. They rightly deplore the traditional lecture style of learning that's common in my university but their solution is more online learning.

The real problem with medical education is that much of the first two years is based on the "memorize and regurgitate" model that we know is ineffective. The best way to change the system is to use evidence-based methods that emphasize student-based learning. The idea is to teach medical students how to access information and how to interpret it rather than have them memorize facts. When teaching biochemistry, for example, it's pointless to ask medical students to take an exam based on structures and pathways that they will forget the day after the exam.

These two physicians are in charge of reforming medical education. They want to please the students by creating a new way of teaching that emphasizes the way "millennials" want to learn. (Short online courses, no lectures.) You'll watch the entire show without hearing any references to the pedagogical literature and what's known to work. Is there any evidence that undergraduate medical students are experts on medical education? (Hint: ... no.)

If this is the wave of the future, I fear that future doctors are not going to be any more informed that the current crop. They will still not be capable of critical thinking.

The way we teach needs to change, but not this way.

---

Joe Hanson tells us about evolution

Joe Hanson Ph.D. (Biology) of It's Okay To Be Smart is posting a series of videos on evolution. They're called The 12 Days of Evolution.

The first one is "What Is Evolution Anyway?" You won't surprised to learn that Joe Hanson conflates "evolution" with "natural selection" and fails to mention the most important features of evolution [see What Is Evolution]. You WILL be surprised to learn that Jerry Coyne has the same objections I do! [Twelve Days of Evolution: #1: What’s evolution?]

We need to do a much better job of educating the general public about the meaning of evolution but first we need to educate the teachers. It's okay to be smart but it's not okay to just pretend to be smart.

---

Evolution as a foundational concept in biochemistry and molecular biology

The American Society for Biochemistry and Molecular Biology (ASBMB) has been promoting a new way of teaching undergraduate courses. The idea is to concentrate on fundamental principles and concepts rather than on trivial details. The various working groups came up with a list of these fundamental concepts under five main headings: Evolution; Matter and Energy Transformation; Homeostasis; Macromolecular Structure & Function; and Biological Information.

I've discussed the concepts before [ASBMB Core Concepts]. There are problems.

Various committees continue to meet in order to build a "concept inventory" to guide the new curriculum. There have been a series of workshops organized around the main themes. The participants in the workshops are, for the most part, teachers at small universities and colleges. They have lots of experience teaching undergraduate courses but they aren't necessarily experts in the subject material.

I saw this clearly when I attended a session at the last Experimental Biology meeting in Boston last April. The purpose of the meeting was to review the major concepts in Evolution and Homeostasis. I met a great deal of resistance from the workshop leaders when I tried to explain the concepts of neutral alleles and random genetic drift and show them why they were so important when comparing sequences and constructing phylogenetic trees.

INTEGRATING EVOLUTION AND HOMEOSTASIS WITH THE CORE CONCEPTS OF BIOCHEMISTRY AND MOLECULAR BIOLOGY
Symposium Tues. 9:45 am Boston Convention & Exhibition Center, room 256

Chaired: E. Bell

9:45 RCN-UBE: Integrating Evolution and Homeostasis with the Core Concepts of Biochemistry and Molecular Biology J.E. Bell, A. Aguanno, P. Mertz, M. Johnson and K.M. Fox. Univ. of Richmond, Union Col., NY, Univ. of Alabama, St Mary’s Col. of Maryland and Marymount Manhattan. (559.2)

Presenters:
Small Group Work: Integrating Evolution and Homeostasis into the Core Concepts E. Bell, Univ. of Richmond A. Aguanno, Marymount Manhattan Col.

Group Discussion on Core Concept integration with Homeostasis A. Aguanno, Marymount Manhattan Col.

Small Group Work: Question Development Involving Evolution and Homeostasis M. Johnson, Univ. of Alabama

Group Presentations and Discussion on Question Implementation K. Fox. Union Col.

This same group has published some of their findings in the July/August issue of the education journal, Biochemistry and Molecular Biology Education (BAMBED)¹ (Aguanno et al. 2015).

Here are the learning objectives they have developed under the "Evolution" concept.

• central importance of the theory of evolution
• Darwin's theory of evolution
• process of natural selection
• evidence for the theory of evolution
• molecular basis of natural selection

I really think this misses the boat in a biochemistry context where molecular evolution plays such an important role. It will be hard to discuss genome organization and junk DNA, for example, if students don't know about population genetics and random genetic drift. It will be hard to explain (correctly) why different proteins in different species have different amino acid sequences if students don't know about neutral alleles.

I pointed this out to the authors at the meeting and stimulated a discussion about these concepts. The authors, and the other teachers in the room, were pretty certain that the differences in amino acid sequences were all due to natural selection. Most of them had never heard of random genetic drift.

The problem here is that the learning objectives and the "capstone experiences" are being developed by teachers who don't really understand evolution. It is assumed that the best people to work on the new curriculum are experienced teachers but that's demonstrably false. (It applies to the other concepts as well.)

It turns out that biochemistry professors are not as knowledgeable about core concepts as you might imagine.

The authors surveyed 161 teachers in 143 institutions across the USA to find out what are the most important concepts in a biochemistry and//or molecular biology course.

The results, right, indicate that less than 8% of the respondents thought that evolution was an important concept.

This could be due, in part, to the fact that biochemistry courses are often taught by professors who are members of a chemistry department but no matter what the explanation it looks like we have a lot of work ahead of us if we are going to convince our colleagues to make evolution a core concept.

I'm pretty sure that many of the people who teach our introductory biochemistry courses at the University of Toronto don't see evolution as a core concept and don't understand modern evolutionary theory.

---

1. Disclaimer: I am on the editorial board of that journal.

Aguanno, A., Mertz, P., Martin, D., and Bell, E. (2015) A National Comparison of Biochemistry and Molecular Biology Capstone Experiences. BAMBED 43:223-232. [doi: 10.1002/bmb.20869]

Nature publishes a misleading history of the discovery of DNA repair

The history of DNA repair is well-known. Here's a quote from "Early days of DNA repair: discovery of nucleotide excision repair and homology-dependent recombinational repair" by W.D. Rupp in 2013 (Rupp, 2013).

This article describes events related to the first papers published in the 1960s describing nucleotide excision repair (NER) and homology-dependent recombinational repair.

Here's are the relevant papers.

Setlow, R.B., and Carrier, W.L. (1964) The disappearance of thymine dimers from DNA: An error-correcting mechanism. Proc. Natl. Acad. Sci. (USA) 51:226–231. [Full Text]

Boyce, R.P., Howard-Flanders, P. (1964) Release of ultraviolet light-induced thymine dimers from DNA in E. coli K-12. Proc. Natl. Acad. Sci. (USA) 51:293–300. [Full Text]

Pettijohn, D, and Hanawalt, P. (1964) Evidence for repair-replication of ultraviolet damaged DNA in bacteria. J. Mol. Biol. 9:395–410. [PubMed]

Lessons not learned from 50 years ago

A few months ago Nature published an article on how to create a science-literate population. There's a letter in the Sept. 17th (2015) issue that addresses this point by reminding readers of another article published 50 years ago (1965).

The title of that older article was "New thinking in undergraduate education." Here's what it said ....

Students are in danger of "spending too much of their time memorizing facts, and [have] insufficient time at [their] disposal to master the principles underlying [their] subject and to develop [their] powers of thought." .... the most important purpose of a university education is to teach [students] to think for [themselves] ... it may on occasion demand a re-examination of the whole approach to a subject in undergraduate courses."

I remember that the biology department where I was an undergraduate (Carleton University in Ottawa) organized a weekend conference to discuss revising undergraduate education in 1967. I spoke about the need to focus on ideas and concepts and get away from boring lectures about facts.

plus ça change, plus c'est la même chose
UPDATE: A reader asked for the references. The letter in the Sept. 17th issue is from Barry S. Winkler [doi: 10.1038/525321f]. The original article from 1965 is in the issue of Feb. 27, 1965 [doi: 10.1038/205835a0]. The 2015 article referred to in the recent letter to Nature is Bradforth et al. (2015).

---

Bradforth, S.F., Miller, E.R., Dichtel, W.R., Leibovich, A.K., Feig, A.L., Martin, J.D., Bjorkman, K.S., Schultz, Z.D., and Smith, T.L. (2015) University learning: Improve undergraduate science education. Nature 523:282-284 [PDF]

Best blog post in the past year

3 Quarks Daily is running their annual contest to pick the best blog posts in the past year. The finalists will be picked by popular vote and the winner will be selected from the finalists by Nick Lane. You can review the rules at: Nick Lane to Judge 6th Annual 3QD Science Prize.

The formal description of the prize is "6th annual prize for the best blog and online-only writing in the category of science." This is important because although the rules refer to "blog posts" and "blog entries" it's clear that most of the nominees are more like online poplar science articles than typical blog posts.

Here's a list of the current nominees ...