Carl Zimmer, one of the top science writers in the world, has written an article for the New York Times with the following provocative title: A Sharp Rise in Retractions Prompts Calls for Reform. It's partly about the rise in the rate of retractions1 in scientific journals. This is a serious problem and it's hard to figure out the underlying cause, in spite of the fact that many of the people who comment think they know the answer.But there's much more to this story as Carl explains on his blog [Dysfunctional science: My story in tomorrow’s New York Times].
In tomorrow’s New York Times, I’ve got a long story about a growing sense among scientists that science itself is getting dysfunctional. For them, the clearest sign of this dysfunction is the growing rate of retractions of scientific papers, either due to errors or due to misconduct. But retractions represent just the most obvious symptom of deep institutional problems with how science is done these days–how projects get funded, how scientists find jobs, and how they keep labs up and running.As usual, Carl's got it right. There's something wrong with science, or perhaps I should say there's something wrong with the biological sciences since Sean Carroll doesn't see the same problem in physics [Is Physics Among the Dysfunctional Sciences?].
1. The rate is about 0.04%. Compare this to the rate of fraudulent creationist publications, which is close to 100%.
"The rate is about 0.04%. Compare this the the rate of fraudulent creationist publications, which is close to 100%"
ReplyDeleteCan you point to one that isn't fraudulent? I'd be amazed to see one.
Robert Gentry's work with polonium halos is an example of legitimate science by a creationist, which Gentry (incorrectly) cited as evidence for the young Earth that fits his particular creationist cosmology. Whether that counts as a "creationist publication" is debatable, since it appeared in a legitimate scientific journal. But the experimental results reported were sound. On a slightly different vein, the American Scientific Affiliation is an organization of Christians with scientific degrees. It publishes serious papers by real scientists who investigate different claims with respect to origins. Some of the scientists are creationists, some are not, but most of them appear to be sincere, rather than frauds. I admit that these are but the tip of a largely rotten iceberg.
DeleteFor additional examples of ‘dysfunctional science’ see these articles: “Why Most Published Research Findings Are False” (http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124) and “Lies, Damned Lies, and Medical Science” (http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/).
ReplyDeleteHowever, the number of ‘retractions’ and the significance of ‘lies’ are not endangering the progress of science. What might do that is: lack of basic knowledge and interest in fundamental principles and concepts; persistence of misleading dogmas; not to mention, the culture of using science primarily as a career vehicle (just look at the CVs of our distinguished scientists, which are so full of ‘titles’, ‘positions’ and ‘prizes’ that they make the infamous former Soviet generals, with their chest full of medals, look modest).
Anyway, why would scientists be different than the rest of the people?
Anyway, why would scientists be different than the rest of the people?
DeleteBecause science is based on evidence, rational thought, and healthy skepticism. We expect scientists to be better than the general public.
We also expect that Christians should be more moral than the average citizen so we hold them to a higher standard. :-)
Amen to that.
DeleteI second that sentiment as well, Larry. What a pity most of my fellow Christians who sound off on this subject seem willing to cut ethical corners in order to advance their agenda.
DeleteDo you see this as a problem right across the biological sciences or do some disciplines see a greater number of retractions than others? If the problem is largely confined to one or two disciplines then this is arguably less worrying (relatively speaking) and potentially easier to deal with than one which encapsulates the whole of the biological sciences.
ReplyDeleteI think that retractions and fraud are the symptom of a larger problem. As I mentioned on the New York Times site, some disciplines are in worse shape than others when it comes to dysfunctional science. Evolutionary psychology is a good example of one of the worst cases.
DeleteHowever, all biological sciences seem to be suffering. I haven't yet encountered a biological scientist who thinks their discipline is in good shape.
This is a serious problem and it's hard to figure out the underlying cause, in spite of the fact that many of the people who comment think they know the answer.
DeleteDespite what you are saying, it's not hard at all. It's not a single underlying cause but several of them.
Thanks for the response Larry. I'm a student of biology but I have to admit to not having heard a great deal from supervisors and colleagues in this regard. It's something I'll certainly look into when I move institutions later this year.
DeleteThe peer review process is the problem here- granted I am not the first person to say this, nor do I intend to propose any solution. With that stated, I do not think the problem with peer review is solely because of the increase in submissions/publications - it has more to do with the lack of attention given to the philosophy of science in the research process of biological scientists.
ReplyDeleteI actually think systematics/phylogenetics is in good shape. Is that too narrow a discipline for you?
ReplyDeleteIt depends what you mean. If you include the papers on the Three Domain Hypothesis then there's a lot of bad science. A lot of the stuff about the demise of the tree of life is also looking very much like dysfunctional science.
DeleteAnd I still don't think the debates over various types of cladism are always examples of good science.
There was a time not long ago when the debates over various methods of tree building got totally out of control and some very silly claims were made.
What do you think about the fights between lumpers and splitters?
This almost needed two discussion. 1 for fraud (which most of us comments have focused on so far) and the second for bad science. I think all fields have their share of bad science, but that is harder to quantify than the retractions / fraud part. Many in various fields are guilty of, or are currently promoting the bad science part, so is much harder to objectively discuss.
DeleteBut my little anecdote - I work in mitochondrial biology (now focusing on mammlian mito and mito disease, but used to do metazoan phylogenetics using whole-genome datasets). Since mitos keep popping up in ageing, and more and more common diseases, more people are moving into the field (a good thing!) but few of them seem to have read any papers published before 2000. Many labs seem to be continuously re-inventing the wheel, but in the process are making a lot of mistakes that were already dealt with in the 1980's.
I wonder how much of the current bad science can be attributed to the lack on knowledge of the past work. I know that Sandwalk has touched on that problem before...
-The Other Jim
I wouldn't say there's a lot of bad science in systematics. There is of course some. There always will be. But I see no sign that the field as a whole, or on average, is pathological. A few bad papers here and there don't cause a significant problem, nor do I see any increase in the proportion of bad papers from any time in the past. In fact, we are finally getting a good handle on phylogeny. What's to complain about?
DeleteFights between lumpers and splitters are an odd thing to bring up, since the common position these days is that it's arbitrary. If you want a good fight on that subject, go back a hundred years.
My inner cynic is not at all surprised by this. It's no reflection on biology as a science, it's just a reflection on the motivations of a fraud.
ReplyDeleteThe big motivation, saith my inner cynic, would be fraud for gain. Money from selling a snake oil medication, patents on processes which produce things which don't actually work, fame and authority (even if temporary) based on publications. Biology has a vulnerability in that a study establishing a conclusion may be too difficult to reproduce quickly to stop the fraud. If I publish a study showing that some random chemical, given to 5-year-olds, will make them smarter at the age of 20, that's 15 years before the study can be definitively discredited, and at least 5 before it can be challenged. That's a lot of time to manufacture and sell snake oil. Whereas with physics, most experiments can be duplicated in the same facility by different people fairly quickly.
And, adds the inner cynic, ideologues don't get hot and bothered about physics. There's nobody thumping their holy book and insisting that the gravitational constant is higher than that currently used, but there are all kinds of people who want to challenge biological science for religious or political reasons.
This may well be wrong (although my inner cynic compels me to add "but if it's wrong, the reason is probably even nastier") but I must confess that if I were going to commit scientific fraud, I would at least consider biology first.
There is nothing new about fraud in science. Anybody remember Piltdown Man or Sir Cyril Burt?
DeleteYes, it's complicated.
ReplyDeleteBut this paragraph really rings true right now. "But other forces are more pernicious. To survive professionally, scientists feel the need to publish as many papers as possible, and to get them into high-profile journals. And sometimes they cut corners or even commit misconduct to get there."
The "goal" of your own academic position is becoming harder to obtain. With the volume of papers published each day, the feeling that you will get caught may be smaller and smaller. If the return to risk ratio gets more tempting, more cheaters are inevitable.
-The Other Jim
While I think the issue of scientific fraud and the reasons it occurs needs to be addressed. I can't help but think 2 things when I first read this. One, how much of this increase is due to people actually getting caught (meaning more people have access to more publications, and are able to "fact check")? Two, The fact that these things are getting caught is a good thing. Science is self correcting (to some extent). There is too much competition out there to not be right, and there are plenty of scientists ready and willing to challenge your claims.
ReplyDeleteThe last thing I must say is that there is so much new information being generated on a daily basis, I feel like it is easy for reviewers (of papers and grants) to give someone a free pass because they have published in high impact journals previously (thinking that If they put out solid work before, then this must also be good science). Is the answer to this a totally anonymous peer review? I don't know, but I think it would help.
In addition, I get the impression that the drive to support your own salary as an investigator and the salary of your post-docs and graduate students leads to the development of large labs and many projects. This does not allow the primary investigator to focus on good work (unless they are super-human). The focus becomes publish, publish, publish (get more money, money, money). I personally dislike the idea of very large labs. I get the feeling that this causes a separation of the PI from the actual work and in some cases a feeling of apathy about the quality as long as something is published (so your grant can be renewed).
-Science newb, optimistic graduate student (4th year in), feeling the pressure to publish (even though I have a few publications to my name, I still worry) BW
In response to "One, how much of this increase is due to people actually getting caught (meaning more people have access to more publications, and are able to "fact check")? "
DeleteIf we are just catching more fraud, that is a good thing. But at the same time I really doubt it.
I am in that "senior postdoc looking to transition to independent" phase. Many of my peers seem very unhealthily obsessed with getting a Science/Nature/Cell paper as a minimum requirement to get the next stage. If that is your perceived "minimum level" on your CV, doesn't it seem a little desperate? Don't you think that they may be over-optimistically interpreting that gel that is the key observation that turns the story from a common one to a S/N/C "quality" one?
The incentives are too great. Look at the "Arsenic Life" flop. She even named the strains with her career goals in mind.
Second - with the ease of access has come the flood of papers that you need to read. How may papers a month do you read? I use pubmed to send me a keyword search of abstracts each month. Last month I got 188 abstracts, and I flagged 91 of them to read. In reality, I will read maybe 10-20 carefully enough to have any chance of noticing something. Then there are the papers I find by the other methods that I need to read...
-The Other Jim
“The peer review process is the problem here…” (Thummim)
ReplyDelete“Many in various fields are guilty of, or are currently promoting the bad science part, so is much harder to objectively discuss.” (Anonymous/The other Jim)
“I think that retractions and fraud are the symptom of a larger problem.” (Laurence A. Moran)
“I must confess that if I were going to commit scientific fraud, I would at least consider biology first” (The Vicar)
Many of these problems might have to do with the fact that many scientists, like other competitive professionals, are going to adapt to whatever the nebulous entity called “system,” (i.e. our professional environment) requires. And, apparently, in highly active fields of science, such as Biology, we are integrated in a “science pyramid scheme” described (i.e. disguised) as a “Ponzi Sci-fi” by Surelyyourjoking in a comment to article “Lies, Damned Lies, and Medical Science” (http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/).
One more thing hit me during a chat with some friends. Look at the "faster than light neutrons" story. The CERN people approached the story with caution and skepticism, and and were very open to the chance that they were wrong. A well established principle in physics was being challenged, if they were correct.
ReplyDeleteNow imagine someone in biosciences who has some preliminary data that "the central dogma of molecular biology" is wrong? Will they behave like CERN, or more like this?
http://sandwalk.blogspot.de/2012/01/mind-of-james-shapiro.html?showComment=1328742052040#c9077880841917658438
-The Other Jim
It depends on who they are, doesn't it? Anyway, there's a major difference. We had good reason to believe that faster than light neutrinos (not neutrons) were impossible. But it's perfectly conceivable that there might be some mechanism, somewhere, that might contradict the central dogma, even if it were correctly stated. I can't offhand think how such a mechanism would work, but it isn't physically impossible.
Delete“Now imagine someone in biosciences who has some preliminary data that "the central dogma of molecular biology" is wrong? Will they behave like CERN, or more like this?”
DeleteDo you mean like this: http://www.ncbi.nlm.nih.gov/pubmed/9811807
@ Claudia Bandea,
DeleteI'm not sure how proteins misfolding other proteins violates the central dogma...
@ John Harshman,
Tachyons apparently could have explained faster-than-light travel. So there was a conceivable mechanism. See ~5:00 at http://www.youtube.com/watch?v=qJ0m13iJw0k
-The Other Jim
Do you mean like this: http://www.ncbi.nlm.nih.gov/pubmed/9811807
DeleteNo. Larry will explain the central dogma to you if you ask him. But prions don't violate it in the slightest.
I just read Larry’s article The Central Dogma of Molecular Biology, which is one of the best pieces I have seen on the subject.
DeleteI think, Crick's quote seems to be very clear on what he meant by Central Dogma: “transfer of information from nucleic acid to nucleic acid, or from nucleic acid to protein may be possible, but transfer from protein to protein, or from protein to nucleic acid is impossible”(emphasis mine).
As shown also in Fig 1, Crick had no conceptual problem envisioning transfer of information among nucleic acids and from nucleic acids to protein, and that includes transfer from DNA to protein (which, yes, could be accomplished with some ingenious engineering of ribosomes; anyone wants to jump on that?, surely it will make gazillions of front pages!).
Crick’s problem was with transfer of information from protein to protein, or from protein to nucleic acid, which he considered taboo. And here is where the prion hypothesis comes into play. See, for example, these articles:
“A scientific revolution? The prion anomaly may challenge the central dogma of molecular biology” (http://www.ncbi.nlm.nih.gov/pubmed/16065057) and “Prion diseases and the central dogma of molecular biology” (http://www.ncbi.nlm.nih.gov/pubmed/10390630)
It is important to mention here that in regard to the term ‘information’ used in the context of Central Dogma, it makes sense that Crick referred to ‘coding information,’ as in ‘hereditary information’ not as in ‘biological information’ which all proteins and nucleic acids use during their endless interactions.
Tachyons "explain" faster than light travel...of tachyons. If they explain anything. Neutrinos aren't tachyons. Sorry, but it's true that neutrinos traveling faster than light is theoretically impossible. That's what all the furor was about: a supposedly impossible result that, if confirmed, would have overturned physics.
DeleteI just read Larry’s article “The Central Dogma of Molecular Biology” (see ‘Essays and Articles’), which is one of the best pieces I read on the subject.
DeleteI think, Crick’s quote seems to be crystal clear on what he meant by Central Dogma: “transfer of information from nucleic acid to nucleic acid, or from nucleic acid to protein may be possible, but transfer from protein to protein, or from protein to nucleic acid is impossible”.
As shown also in Fig 1 of Larry’s article, Crick had no conceptual problem envisioning the transfer of information between DNA and RNA, and from nucleic acids to protein.
Crick’s conceptual problem was with transfer of information from “protein to nucleic acid” and from “protein to protein”; and this is where the prion hypothesis comes into play. See, for example, these articles:
http://www.ncbi.nlm.nih.gov/pubmed/16065057
http://www.ncbi.nlm.nih.gov/pubmed/10390630
@ Claudiu Bandea
Delete(Sigh)
Please read, for instance,
http://sandwalk.blogspot.de/2011/05/central-dogma-strawman.html
-The Other Jim
@ John Harshman
DeleteI assume you are missing a "not" in the second sentence.
They did look into Tachyon neutrinos as an explanation...
http://arxiv.org/abs/1201.3284
... but excluded them based on the reported data.
-The Other Jim
@ Anonymous (The Other Jim)
DeleteThanks for pointing to me the article “The Central Dogma Strawman.” I’m still navigating though Larry’s blog, and it will take me a while to catch up with all his entries and comments.
Indeed, this is another excellent post by Larry on Central Dogma, and there are many interesting associated comments. But, did you try to make another point?