More Recent Comments

Friday, September 18, 2009

How Bad Papers Get Published in Good Journals

 
Donald Williamson used to be a marine biologist. He has some strange ideas about evolution. He thinks, for example, that the reason why butterflies have distinct larval and adult stages is because they arose from the fusion of two separate species—a larva-like species and a butterfly-like species.

Lots of us have crazy ideas but it's a real challenge to get them published in the peer-reviewed literature, and that's how it should be. The reason why science is so successful as a way of knowing is, in part, because it's dominated by skepticism and a requirement for evidence-based rational thought. The system sometimes impedes the acceptance of real innovative ideas but not for long. What is does do successfully, however, is weed out the kooks. But even that doesn't work all the time.

The Proceedings of the National Academy of Sciences (PNAS) is a prestigious journal that's run by the National Academy of Sciences. Once elected to the academy, members have some special privileges when it comes to publishing in the journal. They can "contribute" one of their own papers, in which case they can have a great deal of influence on choosing reviewers, or they can "communicate" the paper of a friend or colleague, in which case they choose the reviewers and send the reviews to the editor of the journal.

It's easy to see the potential for abuse but the remarkable thing is that the process actually works quite well. The quality of papers "contributed" or "communicated" is, in general, no worse than that of papers published in other front-line journals. Two of my own papers were "contributed by" my former Ph.D. supervisor and the process was a rigorous as any other.

But when the system fails, it fails spectacularly.

Lynn Margulis is a member of the National Academy. She "communicated" a paper by Donald Williamson on his strange idea about butterfly evolution (Williamson 2009). That's when the excrement hit the fan.

The editor of PNAS, Randy Schekman, has announced that the "communicated by" option for members will end in July 2010 [PNAS Nixes Special Privileges for (Most) Papers] The Science article reporting on this change in policy leaves little doubt about what prompted it.
An example of alleged gamesmanship popped up online 28 August in PNAS. Lynn Margulis, the noted biologist at the University of Massachusetts, Amherst, communicated a paper by Donald Williamson, a retired marine biologist in the United Kingdom. In it, Williamson promoted his longheld, intriguing—and, say most other biologists, almost certainly misguided—theory about the origins of caterpillars and butterflies. Current biological theory argues that they were always a single species and that each stage evolved via natural selection. Williamson argues instead that two distinct species (one caterpillar-like, one butterfly-like) somehow fused into a hybrid way back when. One species' sperm must have fertilized the other's eggs, transferring genes laterally across species in a non-Mendelian fashion.

Margulis was unavailable for comment, but Williamson says, "Lynn Margulis is prepared to put her name and reputation on the line" to prove that "genome mergers" occur in evolution, a position his paper supports. He also says he knows that Margulis sent his paper to a half-dozen academy reviewers. Williamson says that he thinks they were all positive reviews, but Margulis told Scientific American last week that she canvassed six or seven reviewers to find the two positive reviews necessary to push the paper through.
Shame on you, Lynn Margulis, You've made some outstanding contributions to biology over the years—endosymbioisis being the best example—but it's time to hang up your hat and retire gracefully. Your latest ideas are totally wacky and your inability to distinguish between science and fantasy—as evidenced in your promotion of the Williamson paper—is an embarrassment to those of us who, for several decades, have been holding you up as an example of a successful and creative scientist.


Williamson, D.I. (2009) Caterpillars evolved from onychophorans by hybridogenesis. Proc. Natl. Acad. Sci. (USA) Aug 28, 2009 [Epub ahead of print] [PubMed] [doi: 10.1073/pnas.0908357106]

41 comments :

Buddha Buck said...

Isn't there something disingenuous, improper, or at least reputation-killing to shop a paper around 6 or 7 reviewers to get 2 good reviews for publication?

Is PNAS going to retract the paper?

A. Vargas said...

Of course when ultradarwinian hogwash is published in PNAS (which is often), no one says a peep... Larry just screams when everyone else does.
I still have to read the williamson paper...then I'll have an opinion.

Eamon Knight said...

Margulis is famous as a Maverick Who Turned Out To Be Right. Unfortunately, she seems to be trying to continue playing the same schtick (on her own account, or by proxy), apparently unaware that, in fact, most mavericks are wrong. Reality can be such a bitch.

charlie wagner said...
This comment has been removed by the author.
charlie wagner said...

"Your latest ideas are totally wacky and your inability to distinguish between science and fantasy..."

As I recall, that's what they said about endosymbiosis when she first proposed it decades ago!

And they said the same thing about Barbara McClintock's "jumping genes".

You have no credibility in these matters, Larry.

Bayesian Bouffant, FCD said...

—and, say most other biologists, almost certainly misguided—


I guess a high-brow magazine like Science can't bring itself to use the phrase "batshit insane."

Bayesian Bouffant, FCD said...

Blaise Pascal: Isn't there something disingenuous, improper, or at least reputation-killing to shop a paper around 6 or 7 reviewers to get 2 good reviews for publication?

Hmmm, that reminds me of something.

Michael Behe's tetimony under cross-examination at the Kitzmiller v. Dover trial

Q. Okay. Now you stated on Monday that Darwin's Black Box was also peer reviewed, right?
A. That's correct.
Q. You would agree that peer review for a book published in the Trade Press is not as rigorous as the peer review process for the leading scientific journals, would you?
A. No, I would not agree with that. The review process that the book went through is analogous to peer review in the literature, because the manuscript was sent out to scientists for their careful reading.
Furthermore, the book was sent out to more scientists than typically review a manuscript. In the typical case, a manuscript that's going to -- that is submitted for a publication in a scientific journal is reviewed just by two reviewers. My book was sent out to five reviewers.

charlie wagner said...

"but it's time to hang up your hat and retire gracefully."

Lynn Margulis has made important contributions to science and is a respected member of the scientific community.

How dare you counsel her so disrespectfully?

If anyone should "hang up their hat and retire gracefully" it is you.

(if you're wondering why I'm making such a fuss about this, I must tell you; I'm still angry about how they treated Barbara McClintock at Cold Spring Harbor)

charlie wagner said...

Williamson wrote in his paper:

"I reject the Darwinian assumption..."

The kiss of death...

Why do you think she had to canvas six or seven reviewers to find two positive reviews?

Buzz said...

but it's time to hang up your hat and retire gracefully...

Sorry; your disappointment with Margulis in this particular instance is justified, but who are you to tell her she should stop doing research for good because she supports one wacky idea?

Carlo said...

but who are you to tell her she should stop doing research for good because she supports one wacky idea?

I don't know if I'd go as far as telling her to retire, but it all boils down to this: either you perform evidence-based, ultimately hypothesis driven science, or you don't. There's no room for ambiguity here. You can't disregard evidence when it's no convenient - that's scientific fraud.

The Williamson paper is awful, it ignores everything we know about speciation and developmental genetics (I blogged about it here). If a graduate student came up with this idea, and their supervisor knew anything about the subject matter, they'd end up getting a lecture about what science is. It's perfectly reasonable to call such a view being promoted by senior scientists utterly shameful.

Carlo said...

er... that's 'when it's not convenient".

A. Vargas said...

The article is fun, but I am not convinced. I think it is scientific enough, in the sense that clear predictions are made: 1) you shuld be able to hybridize onychophorans and insects 2) You should be able to track down an onychophoran genome besides the insect genome.

The lack of any new data to support 1 or 2 is what is disappointing (I wondered about those 2 just from the title of the paper) , and the kind of thing a normal person like me would need to publish in PNAS. But I'm all for fun. You usually read fun things like this in the smaller journals only, though.

It is well-documented that hybrids crosses between different species can produce new species with hybrid genomes (as is the case with allopolyploidy). This frequent phenomenon certainly does not fit speciation through typically "darwinian' gradualism, but is extensively documented to have occurred naturally (and experimentally) in many known species of plants and animals.

The question is to the limits of phylogenetic distance for species hybridization, and experimentation is the way to prove it. I think the most impressive I've heard are crosses at genus family and even among snakes, separated by tens of millions of years, but never anything like among phyla.

Williamson himself argued he had gotten an urchin larva from fertilizing a tunicate egg with urchine sperm, but many controls were missing.

http://8e.devbio.com/article.php?ch=23&id=228

Since he takes time to talk about many thing except this experiment, I guess there was some artifact involved? contaminated seawater?

Christopher Taylor said...

I don't know if anyone has ever conclusively explained what happened in Williamson's tunicate X echinoid experiment (after all, to do that you'd probably have to have been there at the time). I believe the most popular explanation is that he was simply incorrect about the sea urchins being solely male, which Williamson of course denies vehemently.

If I may reproduce Williamson's (2001) own summary of the results:

Eggs of the ascidian Ascidia mentula have been fertilised with sperm of the sea-urchin Echinus esculentus to give plutei, i.e. wholly paternal larvae. The majority of the fully developed plutei did not develop Echinus rudiments, but retracted their larval arms to produce spheroids, each with an adhesive disk. There is no corresponding stage in the development of Echinus x Echinus. Some spheroids lived up to 90 days from hatching, but none developed further. A minority of the plutei developed Echinus rudiments and metamorphosed into urchins in 37-50 days. Four years later, the three surviving urchins all produced eggs, which were fertilised with wild Echinus sperm to give apparently normal pluteus larvae. Nucleotide sequences were investigated for a ribosomal and a mitochondrial gene extracted from tube-feet of the urchins. These sequences were wholly paternal, i.e. resembling Echinus.

In other words, Williamson claims that the resulting sea urchins must have been hybrids, because otherwise where did the eggs come from? Whereas other researchers have simply replied that if it quacks like a duck...

InfuriatedSciTeacher said...

Sorely tempted to feed the troll... resisting.

Margulis should know better than using her name to further something that's that far out of tested scientific thinking... endosymbiosis was out there at the time, but plausible. This would require strong documentation of another selective process, and lacking that is an assertion floating in thin air.

Bayesian> Behe got that part right, but what he ignored was the fundamental difference between reviewers for his book and reviewers for peer reviewed journals: you don't get to choose your reviewers based on how likely they are to agree with your paper.

DK said...

you don't get to choose your reviewers based on how likely they are to agree with your paper

Of course you do: you send the MS to an editor friend of yours or to a buddy (or a former supervisor) to "contribute". And, just in case, you disqualify several people who you suspect are not likely to agree. All of this is practically a default and everyone is doing it precisely because it decreases the likelihood of negative review.

Marc said...

She was right about endosymbiosis (which was not her original idea), and from then on she thinks that she's right about anything. Just look at that Gaia crap.

Extraordinary claims require extraordinary evidence. And the Williamson paper doesn't come even close. When I first saw it, I seriously thought it was a joke.

I'm not sure why these fellows are so angry about Larry's comments. She obviously perverted the peer review process.

A. Vargas said...

I like Lynn Margulis, this world is too full of unoriginal, narrow-minded neodarwinians, she doesn't buy it and she pokes their eye all the fuckin time... I love it!
So, she decided to use her PNAS powers to introduce a rather speculative paper? That happens on PNAS all the time...all this fuss is just because its Lynn Margulis, who excells at pissing off the (still abundant) neodarwinians.

Of course, I'll take Lynn margulis over 100 Larry Morans, any day. I think by and large she is absolutely right: Symbiosis is a huge, underestimated factor of evolution. The cases demosntrating this are spectacular and ever increasing

Carlo said...

So, she decided to use her PNAS powers to introduce a rather speculative paper?

There's no point on 'speculating' about things that we know are incorrect. As Jerry Coyne pointed out in his blog post about this paper, this hypothesis makes easily testable predictions that we already know are false. The genomes of insects should have large complements of their genes with obviously different evolutionary signatures (i.e., the trees generated from 'larval' genes should not agree with those generated from the other genes - assuming we attempt to draw a deep-enough time phylogeny). Furthermore, we know quite a bit about speciation genetics and this ignores all of it (there was a successful mating between 2 phyla?).

You can be as underdog-ish and contrarian as you like, but supporting crap science just because it's different isn't helping anyone.

A. Vargas said...

"Furthermore, we know quite a bit about speciation genetics and this ignores all of it"

Aha...so, what is your opinion about the origin of new species by species hybridization? Species with combined genomes exist. Allopolyploidy. Do you think species hybridization is a very darwinian way of speciating?

I agree that the phylogenetic distance is large and seems very improbable. But some still pretty large phylogenetic distances can be breached in hybrid crosses (I understand snakes that diverged in the cretaceous can hybridize, as well as cetaceans of different genus)

Since you know so much about the genetics of speciation, please enlighten us on how is it that you establish just how much phylogenetic distance is "too much".

Carlo said...

Aha...so, what is your opinion about the origin of new species by species hybridization? Species with combined genomes exist.

I will refer you back to the excellent treatise on speciation genetics by Jerry Coyne and Allen Orr called Speciation (Sunderland; 2004).

Yes, speciation by hybridogenesis is quite common in plants, but not so much in animals. I believe that certain closely related species of toads are thought to have arisen by hybridization, but I'm not up on the literature.

In terms of reasonable divergence times, I think that we can make educated guesses based on the numerous studies that have been conducted in insects. Reproductive isolation is already firmly established in species that have been diverged by about a million years in Drosophila (e.g., crossing D. simulans with D. sechellia). Males produced by the crosses are sterile, females are fertile, but can show abnormalities. By 2.5 million years, (e.g., D. melanogaster and D. simulans) all offspring are sterile and show massive developmental defects.

In these examples, we're talking about closely related species of the same genus. In the Williamson paper, we're supposed to be considering the mating between an Onychophoran, which modern phylogenetics clusters near Crustaceans at the base of the arthropod lineage, and an insect. These two species had to be at least diverged enough such that they had radically different morphologies and developmental body plans. So yes, I think that it's reasonable to ask for a mechanism of how this could have occurred.

Furthermore, why Butterflies specifically? Wouldn't it be more parsimonious to assume that all holometabolous insects acquired their larval stage in one hybridization (with subsequent losses in particular lineages)? It's difficult enough to imagine how the hybrid genome suddenly 'knew' to express some genes at one time and others at a different time and somehow had all of the mechanisms in place required to make that transition. Are we supposed to accept that this has happened repeatedly?

A. Vargas said...

I don't buy Williamson's hypothesis, but I'd make you a bet that hybrids are possible over much larger timescales than you think. To begin with, "divergence time" does not equal "cannot hybridize by allopoplyploidy". Allopoliploidy occurs between clearly distinct species, with different sets of chromosomes and all.

I'm already on bibliographic hunt...so watch out...

Oh an you may want to check reptiles and butterflies for more animal examples of speciation by hybridization.

Marc said...

Hey Vargas,

I think that, if you really look into Margulis's opinions, you'd be really disappointed. Turns out she's for the most part pretty "neodarwinian". I've actually met her and and that's pretty clear.

So I guess you're going to have to find a new hero... That Ray Confort fella seems to fit your expectations...

Bayesian Bouffant, FCD said...

Buzz: ...but who are you to tell her she should stop doing research for good because she supports one wacky idea?

He already explained why he disapproves. It is not the "one wacky idea," it is the subversion of the peer-review process. And Margulis has thrown in with more than "one wacky idea." Marc has already mentioned "the Gaia crap." It is also worth mentioning her statements on HIV and AIDS.

Anonymous said...

Even a stopped clock gives the correct time twice per day.

Bayesian Bouffant, FCD said...

Even a stopped clock gives the correct time twice per day.

You're dating yourself. My LED clock stopped, and it just sits there and gives nothing.

Anonymous said...

Vargas:
"I still have to read the williamson paper...then I'll have an opinion."

And doubtless the same number of people will care about your opinions after you do so.

SLC said...

I an not competent to make any judgments on the Williamson paper but it has been clear for some time that Prof. Margulis has turned into a whackjob in her advancing years, much like Linus Pauling, William Shockley, and J. Allen Hynek did. Among other things, she hob nobs with Holocaust deniers, rejects the theory that HIV causes AIDS, and is a 9/11 truther. But of course, that's good company for clowns like Vargas and Wagner.

A. Vargas said...

What is this, incompetent commenter's day? hehehe
Anybody got something to say about some actual Biology? You guys are boring me to death, seriously.

heleen said...

A. Vargas said: You guys are boring me to death, seriously..
Would it be possible that Vargas is bored to silence? The You Guys might be bored enough to doubt that Vargas is a scientist.

Sigmund said...

There are more to fault than Lynn Margulis on this one. The major problem that should have prevented the paper being published was not its hypothesis but the fact that this particular hypothesis is easily tested using genomic sequencing of the sort that is routine nowadays. For a journal of the impact of PNAS one would expect the editors to at least ask the basic in sequence verification before they would allow such a wild claim to be made in their journal.
Ideas that are against the grain should always have a chance of publication but they should be more than speculative, especially when a simple obvious experiment is all that is needed to eithere dismiss them or to give them at least some basis of plausibility.

Sigmund said...

Blaise Pascal said "Isn't there something disingenuous, improper, or at least reputation-killing to shop a paper around 6 or 7 reviewers to get 2 good reviews for publication?"
The opposite case sometimes occurs as well. Our group had a paper reviewd by 'Science' a couple of years back when all the reviews were pretty favorable but asked for fairly minor revisions before acceptance. We did all these, sent it back and one of the original reviewers had an illness and so opted out of the rechecking of the reviews. The editors, rather than accepting the other reviewers verdict (both positive) decided to send the paper out to a new reviewer who was negative and thus the paper was eventually not accepted.
Unfortunately our group is not such a big name and cannot kick up a fuss over this as big name groups can do and we were left with a rather bitter experience of big journal editorial practices.
As in the case of PNAS a lot of the blame is not to be placed on the individual reviewers but on the editor who gives the go-ahead (or not) for publication.

SLC said...

Re A. Vargas

If Mr. Vargas finds himself bored, I suggest he register his discontent by voting with his feet and taking himself to a blog where he would not be bored.

A. Vargas said...

Other bloigs are ven more boring...here guys like Taylor and Carlo gave informative answers...the rest...well, they must be importations from Pharyngula haha

A. Vargas said...

What sigmund says is interesting too...sound sliek a very unfair and frustrating experience.
What I can say is that, long run, where you publish is not relevant. Bad papers published in great journals are forgotten. Good papers are well-cited, no matter how small the journal that published them. It's all about content.
The unfair thing is that those with the contacts to publish in the "great" journals can get funding more easily, and of course, not always deservedly so.

heleen said...

A. Vargas said: well, they must be importations from Pharyngula haha
Pharyngula, where Alexander Vargas has been put into the dungeon ...

A. Vargas said...

precisely! haha. That site is all about those dumbass creationists...megaboring

A. Vargas said...

By the way, I already found a reference for a hybridization of species that diverged 45 million years ago...so...Nyahaha!

Unknown said...

A paper out in PNAS this week shows that this hypothesis can be disproved with available data. More at http://blogs.ucdavis.edu/egghead/2009/10/26/caterpillars-are-not-descended-from-a-worminsect-hybrid/.

Anonymous said...

PNAS continues to publish "junk science", or "poorly substantiated" claims if one wants to be charitable. Sometimes reading articles feels like reading a marketing promo where the authors are using carefully crafted language to present their poor results in an exciting way. A recent article (past couple of years) claimed to have wonderful de novo results. Not only it did not stand up to scrutiny once you carefully read their supplementary data, in light of the supplementary data the main article was presenting a method that was not useful at all! (I am intentionally being oblique in order to avoid pointing fingers at any one person or group).

Communicated by was certainly part of the problem. I am glad they are nixing the "communicated by". In my opinion, this option led to a lot of "gaming the system" and "inside baseball promotion" - not the kind of thing you want to see in science.

I'd really like someone to do a survey of the articles published in the PNAS in the last 10 years to see how many of the "breakthrough" exciting technologies have held up to the test of time.

Unknown said...

Lynn Margulis was a nut-job. She believed AIDS was really syphilis. For a microbiologist to make this claim is absolutely ridiculous. She should know better. Syphilis can be easily detected, and its symptoms do not match those of AIDS. As has been shown before with AIDS denial (Duesberg, Mullis), this position stems from homophobia. So no, this isn't Margulis being a "maverick." To be a maverick, you have to be *right*.

Neo-Darwinism has more evidence behind it than most scientific theories - even those in physics. (Still waiting on that theory of gravity.) Endosymbiosis may be a force of evolution, but hybridogenesis in animals is a fictitious exaggeration. These claims are easily testable - Williamson's claims have been debunked within the very parameters he set himself. Margulis could never show, genetically, that flagella originated from spirochete-to-archaea fusion, yet she insisted this was the case until the end.

Was she a great scientist? Early on, yes. But she took that ball and ran with it, and went into some very strange (and wrong) places.

And please stop comparing the sexism Margulis experienced to that of McClintock. McClintock supported her hypotheses with heavy, solid evidence. Margulis stopped short after her success with mitochondria/chloroplasts.