Nature opposes misinformation (pot, kettle, black)

The lead editorial in last week's issue of Nature (Dec. 8, 2016) urges us to Take the time and effort to correct misinformation. The author (Phil Williamson) is a scientist whose major research interest is climate change and the issue he's addressing is climate change denial. That's a clear example of misinformation but there are other, more subtle, examples that also need attention. I like what he says in the opening paragraphs,

Most researchers who have tried to engage online with ill-informed journalists or pseudoscientists will be familiar with Brandolini’s law (also known as the Bullshit Asymmetry Principle): the amount of energy needed to refute bullshit is an order of magnitude bigger than that needed to produce it. Is it really worth taking the time and effort to challenge, correct and clarify articles that claim to be about science but in most cases seem to represent a political ideology?

I think it is. Challenging falsehoods and misrepresentation may not seem to have any immediate effect, but someone, somewhere, will hear or read our response. The target is not the peddler of nonsense, but those readers who have an open mind on scientific problems. A lie may be able to travel around the world before the truth has its shoes on, but an unchallenged untruth will never stop.

I've had a bit of experience trying to engage journalists who appear to be ill-informed. I've had little success in convincing them that their reporting leaves a lot to be desired.

I agree with Phil Williamson that challenging falsehoods and misrepresentation is absolutely necessary even if it has no immediate effect. Recently I posted a piece on the misrepresentations of the ENCODE results in 2007 and pointed a finger at Nature and their editors [The ENCODE publicity campaign of 2007]. They are responsible because they did not ensure that the main paper (Birney et al., 2007) was subjected to appropriate peer review. They are responsible because they promoted misrepresentations in their News article and they are responsible because they published a rather silly News & Views article that did little to correct the misrepresentations.

That was nine years ago. Nature never admitted they were partly to blame for misrepresenting the function of the human genome.

Pot, kettle, black

Is there a more effective way of challenging misrepresentation and getting corrections—or at least acknowledgement of error? Perhaps there is. Two Nature editorials from the week before addresses this very issue. The first one says Academia must resist political confirmation bias. I wish they had changed the online title to just read "Resist confirmation bias." That would have been more appropriate. Not all biases are political.

Here's an example of confirmation bias. Imagine you are convinced that humans are much more complex than all other species. You were disappointed to learn that we have about the same number of genes are many other species and even fewer than some plants. Imagine you are ignorant of modern evolutionary theory—you think that natural selection is all-powerful. You are very skeptical of junk DNA because it conflicts with your view of a complex genome that has been honed by selection.

Imagine you do some experiments provide evidence of pervasive transcription. Most of the transcripts are confined to one type of cell and they are present at very low levels. Maybe it's just junk RNA that has no function. You do a test to see whether the DNA that's complementary to those transcripts is conserved or not, knowing that sequence conservation is an important proxy for function. The test shows that the DNA is not conserved. What do you conclude?

The normal response is to conclude that you're probably dealing with junk RNA that has no biological function. You don't have proof but it seems very likely that this is the proper explanation of your result. But there's a problem. You are already convinced, for non-scientific reasons, that most of our genome should be functional. You are already convinced, for non-scientific reasons, that there's some sort of "missing complexity" in the human genome that needs to be explained.

This leads you to re-interpret the result and view it as some sort of confirmation of your biases. You construct an elaborate hypothetical scenario that gets around the lack of conservation problem and allows for massive amounts of biologically functional transcripts that aren't conserved but may be selected in the future. There must be something wrong with using conservation to identify function and rule out junk. You publish a paper saying,

Our data support these hypotheses, and we have generalized this idea over many different functional elements. The presence of conserved function encoded by conserved orthologous bases is a commonplace assumption in comparative genomics; our findings indicate that there could be a sizable set of functionally conserved but non-orthologous elements in the human genome, and these seem unconstrained across mammals.

This is what Birney et al. said in their 2007 paper. That's confirmation bias at work. This misresentation of their results and misrepresentation of their conclusions was pointed out many times in 2007 and afterward.

The second editorial in the Dec. 1 issue is: Post-publication criticism is crucial, but should be constructive. The subtitle carries the main message: "In an era of online discussion, debate must remain nuanced and courteous." Here's how Nature editors think criticism should be handled,

Anyone who finds flaws should seek corrections with diplomacy and humility. A gloating sense of ‘gotcha’ does not help to provide constructive criticism; some ill-considered phrases have caused lasting damage. But many scientists use their blogs for credible, restrained, nuanced criticism, often engaging the authors whose works are criticized.

Sharing and discussion of scientific work has changed drastically in a world of blogs, online repositories and Twitter. The fact remains, however, that self-correction is at the heart of science. Critics — curated or not — should be courteous, but criticism itself must be embraced.

That's a good description of how the criticism of Nature and the ENCODE authors was handled in 2007. Here's how I responded in 2007 [What is a gene, post-ENCODE?].

This is not news. We've known about this kind of data for 15 years and it's one of the reasons why many scientists over-estimated the number of humans genes in the decade leading up to the publication of the human genome sequence. The importance of the ENCODE project is that a significant fraction of the human genome has been analyzed in detail (1%) and that the group made some serious attempts to find out whether the transcripts really represent functional RNAs.

My initial impression is that they have failed to demonstrate that the rare transcripts of junk DNA are anything other than artifacts or accidents. It's still an open question as far as I'm concerned.

And here's how Mike White politely responded back in 2007 [Our Genomes, ENCODE, and Intelligent Design ],

Despite what you may read, there is still a lot of junk DNA. The ENOCDE project does not "sound the death-knell for junk DNA." Our genomes are filled with fossils of genetic parasites, inactive genes, and other low-complexity, very repetitive sequence, and it's extremely clear that most of this stuff no functional role. Much of this sequence may be transcribed, but remember that the ENCODE evidence for most of this transcription is indirect - their direct measurements only detected transcripts for ~14% of the regions they studied. Even if much of it is transcribed, this mainly suggests that it is not worth expending energy to actively repress this transcription, since there are so many other controls in place to deal with unwanted transcripts in the cell.

Here's third polite response from John Timmer [ENCODE finds the human genome to be an active place],

There seems to be three possible interpretations for all these extra transcripts. One is that, even though we haven't detected a biological function, and evolution doesn't conserve them, they are actually specifically functional. This would be the "there is no junk DNA" take on matters. The opposite extreme would be an "it's all junk" view of it. From this perspective, the starting and stopping of transcription is just an inherently noisy process and doesn't do humans enough harm to create a selective pressure to improve it.

Somewhere between the two would be the view that few of these extra transcripts are useful in themselves, but it's useful having them present on the collective level. Reasons could include anything ranging from excess RNA performing some sort of structural function through to the random transcripts being a rich source of new genes.

Personally, I fall into the "it's all junk" end of the spectrum. If almost all of these sequences are not conserved by evolution, and we haven't found a function for any of them yet, it's hard to see how the "none of it's junk" view can be maintained. There's also an absence of support for the intervening view, again because of a lack of evidence for actual utility. The genomes of closely related species have revealed very few genes added from non-coding DNA, and all of the structural RNA we've found has very specific sequence requirements. The all-junk view, in contrast, is consistent with current data. We've wondered for decades how transcription factors can act specifically and at long distances despite their relatively weak specificity for DNA. This data answers that question simply: they don't.

Did any of these polite considerate criticisms have an effect? Nope. Nada. Nothing.

We know this because five years later, in 2012, ENCODE published their complete analysis of functional regions of the human genome and they doubled down on the definition of "function." In the second stage of the publicity campaign, Nature and their editors went out of their way to ignore all previous criticism and actively promote the idea that most of our genome is functional and junk DNA is dead.

So I call "bullshit" when I read Nature editorials calling for constructive and polite criticism. By 2012, most bloggers realized that the only way to get attention when you are criticizing a big powerful group of scientists and their journalist friends is to be a little less polite. Here's how John Timmer responded to the second round of misrepresentation in 2012 [A bad precedent, repeated],

This brings us to the ENCODE project, which was set up to provide a comprehensive look at how the human genome behaves inside cells. Back in 2007, the consortium published its first results after having surveyed one percent of the human genome, and the results foreshadowed this past week's events. The first work largely looked at what parts of the genome were made into RNA, a key carrier of genetic information. But the ENCODE press materials performed a sleight-of-hand, indicating that anything made into RNA must have a noticeable impact on the organism: "the genome contains very little unused sequences; genes are just one of many types of DNA sequences that have a functional impact."

There was a small problem with this: we already knew it probably wasn't true. Transposons and dead viruses both produce RNAs that have no known function, and may be harmful in some contexts. So do copies of genes that are mutated into uselessness. If that weren't enough, just a few weeks later, researchers reported that genes that are otherwise shut down often produce short pieces of RNA that are then immediately digested.

So even as the paper was released, we already knew the ENCODE definition of "functional impact" was, at best, broad to the point of being meaningless. At worst, it was actively misleading.

But because these releases are such an important part of framing the discussion that follows in the popular press, the resulting coverage reflected ENCODE's spin on its results. If it was functional, it couldn't be junk. The concept of junk DNA was declared dead far and wide, all based on a set of findings that were perfectly consistent with it.

Four years later, ENCODE apparently decided to kill it again.

... ENCODE remains a phenomenally successful effort, one that will continue to pay dividends by accelerating basic science research for decades to come. And the issue of what constitutes junk DNA is likely to remain controversial—I expect we'll continue to find more individual pieces of it that perform useful functions, but the majority will remain evolutionary baggage that doesn't do enough harm for us to eliminate it. Since neither issue is likely to go away, it would probably be worth our time to consider how we might prevent a mess like this from happening again.

The ENCODE team itself bears a particular responsibility here. The scientists themselves should have known what the most critical part of the story was—the definition of "functional" and all the nuance and caveats involved in that—and made sure the press officers understood it. Those press officers knew they would play a key role in shaping the resulting coverage, and should have made sure they got this right. The team has now failed to do this twice.

But even that kind of response didn't get much attention. Here's how Dan Graur succeeded in Graur et al. (2013).

Here, we detail the many logical and methodological transgressions involved in assigning functionality to almost every nucleotide in the human genome. The ENCODE results were predicted by one of its authors to necessitate the rewriting of textbooks. We agree, many textbooks dealing with marketing, mass-media hype, and public relations may well have to be rewritten.

... So, what have we learned from the efforts of 442 researchers consuming 288 million dollars? According to Eric Lander, a Human Genome Project luminary, ENCODE is the “Google Maps of the human genome” (Kolata 2012). We beg to differ, ENCODE is considerably worse than even Apple Maps. Evolutionary conservation may be frustratingly silent on the nature of the functions it highlights, but progress in understanding the functional significance of DNA sequences can only be achieved by not ignoring evolutionary principles.

High-throughput genomics and the centralization of science funding have enabled Big Science to generate “high-impact false positives” by the truckload (The PLoS Medicine Editors 2005; Platt et al. 2010; Anonymous 2012a, 2012b; MacArthur 2012; Moyer 2012). Those involved in Big Science will do well to remember the depressingly true popular maxim: “If it is too good to be true, it is too good to be true.”

We conclude that the ENCODE Consortium has, so far, failed to provide a compelling reason to abandon the prevailing understanding among evolutionary biologists according to which most of the human genome is devoid of function. The ENCODE results were predicted by one of its lead authors to necessitate the rewriting of textbooks (Pennisi 2012a, 2012b). We agree, many textbooks dealing with marketing, mass-media hype, and public relations may well have to be rewritten.

That's what gets attention. It's not polite and it's not constructive.¹ But it's proper criticism and it's fair. ENCODE and Nature didn't listen in 2007 so they needed to be hit over the head in a way that would force them to respond.

They did respond. In 2014 ENCODE basically retracted many of their claims, albeit, with lots of qualifiers (Kellis et al., 2014). Nature and their editors, on the other hand, have never admitted to their mistakes in 2007 and in 2012. The journal did, however, mention the ENCODE retraction in an editorial published on May 7, 2014 [ENCODE debate revived online]. Some editor wrote,

Kellis says that ENCODE isn't backing away from anything. The 80% claim, he says, was misunderstood and misreported. Roughly that proportion of the genome might be biochemically active, he explains, but some of that activity is undoubtedly meaningless, leaving unanswered the question of how much of it is really 'functional'. Kellis also argues that focusing on the portion of the genome that is shaped by natural selection can be misleading. For example, he says, genes that cause Alzheimer's disease or other late-in-life disorders may be largely immune to evolutionary pressure, but they are still definitely functional.

Do you see the irony? The ENCODE Consortium is now claiming their results were "misunderstood and misreported" in the media. This isn't true, the papers themselves are responsible for the misrepresentation [The truth about ENCODE]. But even if this accusation were true, the main culprit is Nature which blithely reports this criticism without realizing it's directed at them!

Nature editors ignore criticism and promote misrepresentation but they write editorials condemning others who do the same thing.

Hypocrites.

The Tone Wars

There's been a lot of push-back concerning frank and open criticism that's less than polite. It's led to the "Tone Wars." What it boils down to is this. Polite criticism is okay because we can ignore it but frank and harsh criticism in not the way scientists are supposed to behave because it's harder to ignore and it makes us feel guilty. Another part of the "Tone Wars" is to advocate ignoring blogs and only pay attention to criticism that's published in the scientific literature where it can be peer-reviewed. This complaint often comes from authors who have greatly benefited from PR campaigns in the popular press outside of the scientific literature.

Let's quote again from the Nature editorial on May 7, 2014—the one responding to the ENCODE retraction of their original claims.

In the social-media age, scientific disagreements can quickly become public — and vitriolic. A report from the ENCODE (Encyclopedia of DNA Elements) Project consortium proposes a framework for quantifying the functional parts of the human genome. It follows a controversial 2012 Nature paper by the same group that concluded that 80% of the genome is biochemically functional (Nature 489, 57–74; 2012). Dan Graur, who studies molecular evolutionary bioinformatics at the University of Houston in Texas and is a vocal ENCODE critic, weighed in on this latest report. ENCODE's “stupid claims” from 2012 have finally come to back to “bite them in the proverbial junk”, Graur wrote on his blog. The targets noticed. “Some people seek attention through hyperbole and mockery,” says the report's first author Manolis Kellis, a computer scientist at the Massachusetts Institute of Technology in Cambridge. “We should stay focused on the issues.”

This is just the latest skirmish in an ongoing battle. In a scathing 2013 article, Graur and co-authors argued that the ENCODE researchers had essentially ignored evolutionary evidence that suggests that only 2–15% of the genome was under pressure from natural selection (D. Graur et al. Genome Biol. Evol. 5, 578–590; 2013).

Of all the criticisms over the previous seven years (2007 - 2014), which one gets attention? Is it the polite, constructive criticisms published on so many blogs? Nope.

Hypocrites.

---

Image Credit: The image of the pot and kettle mocking each other is from a list of English Idioms at: The Pot Calling The Kettle Black.

1. An example of "constructive" criticism would be to call for a halt in funding the major ENCODE labs and for an apology from Nature.

Birney, E., Stamatoyannopoulos, J. A., Dutta, A., Guigó, R., Gingeras, T. R., Margulies, E. H., Weng, Z., Snyder, M., Dermitzakis, E. T., et al. (2007) Identification and analysis of functional elements in 1% of the human genome by the ENCODE pilot project. Nature, 447:799-816. [doi: 10.1038/nature05874]

Graur, D., Zheng, Y., Price, N., Azevedo, R.B., Zufall, R.A., and Elhaik, E. (2013) On the immortality of television sets: “function” in the human genome according to the evolution-free gospel of ENCODE. Genome Biology and Evolution, 5:578-590. [doi: 10.1093/gbe/evt028]

Kellis, M., Wold, B., Snyder, M.P., Bernstein, B.E., Kundaje, A., Marinov, G.K., Ward, L.D., Birney, E., Crawford, G.E., and Dekker, J. (2014) Defining functional DNA elements in the human genome. Proceedings of the National Academy of Sciences, 111:6131-6138. [doi: 10.1073/pnas.1318948111]