Friday, March 21, 2014

Science still doesn't get it

The latest issue of Science contains an article by Yudhijit Bhattacharjee about Dan Graur and his critique of the ENCODE publicity disaster of September 2012. The focus of the article is on whether Dan's tone is appropriate when discussing science.

Let me remind you what Science published back on September 7, 2012. Elizabeth Pennisi announced that ENCODE had written the eulogy for junk DNA. She quoted one of the leading researchers ...
This week, 30 research papers, including six in Nature and additional papers published by Science, sound the death knell for the idea that our DNA is mostly littered with useless bases. A decade long project, the Encyclopedia of DNA Elements (ENCODE), has found that 80% of the human genome serves some purpose, biochemically speaking. “"I don’t think anyone would have anticipated even close to the amount of sequence that ENCODE has uncovered that looks like it has functional importance,"” says John A. Stamatoyannopoulos, an ENCODE re searcher at the University of Washington, Seattle.

Beyond defining proteins, the DNA bases highlighted by ENCODE specify landing spots for proteins that influence gene activity, strands of RNA with myriad roles, or simply places where chemical modifi cations serve to silence stretches of our chromosomes. These result are going “to change the way a lot of [genomics] concepts are written about and presented in textbooks,” Stamatoyannopoulos predicts.
That's pretty cocky stuff. Re-writing the textbooks, indeed.

This is just a small example of the over-the-top rhetoric announced by the members of the ENCODE Concortium and eagerly picked up by gullible "science journalists" like Elizabeth Pennisi.

Many people, including me, criticized the ENCODE Consortium for misprepresenting their data in order to attract publicity. What they showed is that a significant percentage of our genome is transcribed if you add up all the tissues they examined. They also showed that our genome is littered with sites that bind transcription factors.

Many scientists think that this represents spurious transcription that produces junk RNA with no biological function. We think that the huge number of transcription factor binding sites is just what you'd expect of DNA binding proteins that recognize small stretches of DNA sequence. Most of these sites will have no biological function.

All of these issues were thrashed out back in 2007 when ENCODE published their pilot study but there was no mention of the controversy when ENCODE launched their promotion in September 2012.

Eventually some of the strong criticism of ENCODE leaders made it into the scientific literature. The paper by Graur et al. (2013) was just one of them. They didn't pull any punches ...
Thus, according to the ENCODE Consortium, a biological function can be maintained indefinitely without selection, which implies that at least 80 − 10 = 70% of the genome is perfectly invulnerable to deleterious mutations, either because no mutation can ever occur in these “functional” regions or because no mutation in these regions can ever be deleterious. This absurd conclusion was reached through various means, chiefly by employing the seldom used “causal role” definition of biological function and then applying it inconsistently to different biochemical properties, by committing a logical fallacy known as “affirming the consequent,” by failing to appreciate the crucial difference between “junk DNA” and “garbage DNA,” by using analytical methods that yield biased errors and inflate estimates of functionality, by favoring statistical sensitivity over specificity, and by emphasizing statistical significance rather than the magnitude of the effect. Here, we detail the many logical and methodological transgressions involved in assigning functionality to almost every nucleotide in the human genome. The ENCODE results were predicted by one of its authors to necessitate the rewriting of textbooks. We agree, many textbooks dealing with marketing, mass-media hype, and public relations may well have to be rewritten.
How are Elizabeth Pennisi and Science dealing with this controversy? Do they acknowledge that they may have been taken in by the hype of the ENCODE leaders? Do they acknowledge that they may have been wrong to pronounce the death of junk DNA?

Not a chance. There's been barely a peep out of them in eighteen months. Instead, we get this wishy-washy article by Yudhijit Bhattacharjee that wonders whether Dan Graur and his colleagues may have been impolite in exposing bad science and shoddy journalism. Here's what Bhattacharjee says,
The heart of his critique was that ENCODE researchers had made an unwarranted leap in the interpretation of their data. The project involved thousands of experiments. In some, researchers exposed cells to a multitude of transcription factors: molecules that bind to genomic DNA to initiate transcription into RNA, the first step in making a vast array of proteins required for metabolism. In other experiments, researchers identified and inventoried the different RNA molecules produced in various types of human cells. The results showed that more than 70% of DNA in the genome is transcribed into RNA; 8% latched on to transcription factors. Altogether more than 80% of the genome showed some kind of biochemical activity—the basis for ENCODE's claim that 80% of the genome is functional.

That inference, Graur inveighed, was utterly wrong because the mere transcription of a stretch of DNA or the binding to a transcription factor is not a function unto itself. He didn't say it simply; he said it with merciless mocking that, to some, undermined his message. "Graur wrote such a negative paper that it was hard to read," says Bradley Bernstein, an ENCODE researcher at Harvard University. Graur's criticism is so over-the-top that it's not worthy of a response, Bernstein adds. In his Chicago talk, Graur showed a photograph of chewing gum stuck to a shoe as an example of "a function that fits the ENCODE definition." "The fallacy of ENCODE's logic," he said, is this: "We know that some functional regions are transcribed. Ergo, all transcribed regions are functional." Toward the end of the presentation, he showed a photograph of dollar bills taped together in the shape of a toilet paper roll—his view of what ENCODE had achieved with the $288 million spent on the project so far.
I wonder if Bradley Bernstein has commented on the ENCODE publicity campaign and whether he agrees with the outrageous claims made by the ENCODE leaders. Does anyone know?

Graur's criticism, by the way, is perfectly reasonable. Not only that, it's criticism that's been in the scientific literature for half a decade and widely discussed in the media and the blogosphere since 2007. I don't blame Graur for being angry at the ENCODE leaders who ignored the critiques of their earlie claims from 2007—they deserve harsh criticism.

Most of Yudhijit Bhattacharjee's article is about Dan Graur ("The Vigilante"). The emphasis is on Graur's atheism and his unpopular stance on many issues. The article expresses concern about the way Dan Graur expresses his criticism and not about whether he is right.
At the same time, Graur's combative approach has earned disapproval from some quarters. "Would a dispassionate and polite reply have been less visible?" Nature Methods asked in an editorial last fall that slammed Graur for engaging in what the journal saw as uncivil discourse. "Is provocation necessary to get attention from a 'big science' consortium such as ENCODE? We do not think so."
Nature is responsible for most of the false information spread by the ENCODE Consortium in their publicity campaign. I've yet to see a "mea culpa" from Nature admitting that they blew it by publishing the original papers announcing that 80% of the genome is functional. Instead, we see editorials attacking the messenger who brings the unwelcome message. Shameful.

So, how are the ENCODE leaders responding to the attacks? Are they listening? Do they admit that they were wrong to ignore the alternative explanations of their data that arose in 2007? Do they admit that they may have been guilty of a slight bit of exaggeration? That they bamboozled many science journalists?

Yudhijit Bhattacharjee asked John Stamatoyannopoulos. Remember that he's the one who said the textbooks will be rewritten.
Graur and other critics place undue emphasis on the 80% figure, says John Stamatoyannopoulos, an ENCODE principal investigator at the University of Washington, Seattle. The real take-home lesson, he says, is that "there is a tremendous amount of activity encoded in the genome"—much more than researchers had suspected.

Given the current state of knowledge, Stamatoyannopoulos says, scientists need to remain "fairly agnostic" about the potential function of various genomic elements. In other words, while the likes of Graur are asking, How do you know it's functional? Stamatoyannopoulos and others are asking the opposite: How do you know it's not?

That logic infuriates Graur. "If you don't know a function, assume as a null hypothesis that it doesn't have function, and if you find a function, you'll refute the null hypothesis," he says.

I asked Graur if his detractors were right in calling him rude. He didn't think so; moreover, he felt rudeness was irrelevant to the discourse. "Science is not about abiding by a code of behavior put forward by Miss Manners," he told me. "In science, a strong voice is sometimes needed to fight self-promotion and self-delusion."
Stamatoyannopoulos infuriates me as well. He was wrong about the claim that 80% of our genome is functional and now he's backtracking by saying that junk DNA hasn't been killed off but just challenged—we should be "agnostic." Like most of his colleagues, he is completely ignoring all the evidence supporting the concept of junk DNA in order to promote the importance of his own work [see Five Things You Should Know if You Want to Participate in the Junk DNA Debate].

That is not how good scientists should behave. Good scientists behave like Dan Graur.

My talk was right after Dan Graur's at the evolution meeting in Chicago last summer. Nobody was able to mount a credible defense of the ENCODE results and nobody was able to defend the idea that most of our genome is functional. I challenge any ENCODE leader to a debate about the existence of junk DNA. If they are so confident that they told the truth then they shouldn't be afraid to defend it in a public debate.

Do you think Science, Nature and the ENCODE Consortium will ever admit that they were wrong?


Bhattacharjee, Y. (2014) The Vigilante. Science 343:1306-1309. [Abstract] [doi: 10.1126/science.343.6177.1306 ]

Graur, D., Zheng, Y., Price, N., Azevedo, R. B., Zufall, R. A. and Elhaik, E. (2013) On the immortality of television sets: "function" in the human genome according to the evolution-free gospel of ENCODE. Genome biology and evolution 5, 578-590. [doi: 10.1093/gbe/evt028]


17 comments :

  1. A nearly perfect blog post.

    You serious about a debate?

    ReplyDelete
    Replies
    1. Nearly? :-)

      Yes. I'm serious. I'd love to take on all of the ENCODE leaders. If they want to defend themselves in a group then I reserve the right to call in help on my side. I have a few people in mind.

      Delete
    2. Yes, "nearly". You misspelled "Nature" in your last sentence. : )

      I wonder what the ENCODE people would say in their defence? They've already conceded most of your main points. Hopefully this debate will come off. Do you plan to contact some of them directly? They may not all be regular Sandwalkers.

      Delete
    3. I have a list of quotes from ENCODE scientists where they admit they never proved most of the genome was functional. Put those quotes up on the projector screen, they're dead in the water. Dead dead dead.

      Delete
    4. Professor Moran,

      Please keep amusing us and continue to make this blog the best there is...
      I would love to see you take on ENCODE in a public debate....

      Delete
    5. Professor Moran,

      Anybody in particular from ENCODE you would like to debate...?

      Delete
  2. Is there a Latin name for this fallacy? I mean attacking your opponent's manners if you can't refute his critique. If not, I suggest "argumentum ad decorum".

    ReplyDelete
    Replies
    1. I've heard it in context of racism/sexism as “the tone argument”

      Delete
  3. I love how they accuse Growl of being "provocative" (like it's bad when he does it) but the ENCODE leaders came up with their "80% functional" figure in order to provoke the audience of non-scientists (it's good when they do it.)

    ReplyDelete
  4. Dan can take it as well as dish it out. He was widely ridiculed at the time for his claim that guinea pigs weren't rodents. I don't recall that he whined about it, and I think it may have made him more cautious about sensational claims in general. Thus his debunkings of the live Permian bacterium, the facile molecular clock, and ENCODE. Did I miss any?

    ReplyDelete
  5. If Graur had written the paper the way his critics would have prefered even fewer would have realized that something is wrong with ENCODE. Although it is insufficient the Science paper may still become a turning point because until now there weren't many reactions to Graur or Doolittle outside the blogosphere and twitter.

    ReplyDelete
  6. Nice post, but perhaps you should call your headline writer in for a chat.

    On reading Bhattacharjee's article I found it quite balanced. For instance, he does make the point that there are multiple lines of evidence in favour of junk DNA and that all of these were ignored by ENCODE.

    I guess you _could_ say the focus of the article was on whether Graur's tone was appropriate. But what struck me instead was that it highlights how a bunch of people are trying to deflect his criticism by focussing on tone - a point Graur has made himself.

    Overall, it seemed to me that the article made Graur look good; the ENCODE folk - not so much.

    ReplyDelete
  7. A new paper in Biol Philos tries to defend ENCODE:

    Germain et al. (2014): Junk or functional DNA? ENCODE and the function controversy
    DOI 10.1007/s10539-014-9441-3


    From the paper:
    In light of this epistemological strategy, we believe that the best interpretation of ENCODE’s claim of function is methodological: it is the claim that 80 % of the genome is engaging in relevant biochemical activities and is very likely to have a causal role in phenomena deemed relevant to biomedical research. In other words,
    while this 80 % cannot strictly speaking be called functional (according to the CR
    account) as ENCODE claimed, it is very likely to be. This means that, according to
    ENCODE, most of what has been called junk DNA cannot be ignored in biomedical
    research, and that 80 % of the genome is potentially relevant.

    ReplyDelete
    Replies
    1. The last manuscript has been added to institutional repositories and should be indexed by google shortly. In the meantime, the link is: http://air.unimi.it/handle/233039
      Since people appear to react harshly (even before reading it), we'd like to take this opportunity for a little disclaimer. There's a broad consensus that ENCODE inflated its results, and our aim wasn't to give a general defense of the whole project. However, we tried to show that there's more to the debate than this, that there's a deeper disagreement about which it's important to discuss. So we kindly ask the readers to go beyond a "with me or against me" reaction and give serious consideration to what we consider important and problematic conceptual issues.
      Finally, please feel free to contact us to bring to our attention anything you think is relevant to the discussion.
      Best regards,
      Pierre-Luc, Emanuele and Federico

      Delete
  8. A philosophy journal, these guys did no experiments, and the article costs $40 US

    ReplyDelete
  9. Is freaking Nature calling him out for being sensational and proactive?! Yeesh.

    ReplyDelete