Sunday, July 03, 2016

The scientific literature is becoming more complex

A recent paper by Cordero et al. (2016) looked at the biological scientific literature in 1993, 2003, and 2013. They found that the average publishable unit (APU) almost doubled in twenty years. There were substantial increases in the number of tables & figures and the number of panels per figure. The number of pages increased as did the number of references and the number of authors.

I agree that papers are becoming more complex and more difficult to understand for the average scientist; especially those outside of the specific field of study. The authors of this study point out a number of problems with this increase. I'd like to highlight one of them.

With respect to the number of authors, they say,
Concomitantly, with the increase in information density we note a significant increase in the number of authors per article that also correlated with the average IF of the journal. Since the famous de Solla Price predictions [38], trends toward an increasing number of authors per publication have been widely documented [23,39–44]. Such a trend of increasing collaboration could be explained by the causes suggested above for the growth of information density. The costs associated with the generation of cutting-edge scientific information, the funding restrictions, and the associated risks in scientific publishing in a “winner-take-all” reward system [45] may motivate scientists to team-up, pool resources and fractionate the risks through co-authoring. Also, the increasing complexity of scientific research has resulted in greater specialization of scientists [46], which in turn suggests that the inclusion of additional techniques requires the recruitment of additional investigators to provide that data and thus serve as co-authors. This trend could have both positive and negative consequences. Increased interaction between scientists in diverse fields could translate into greater communication and the possibility for advances at the interfaces of different disciplines. On the other hand, an increase in the number of authors, some of whom bring highly specialized knowledge, could result in reduced supervision of larger groups, and less responsibility per author for the final product and reduced integration of data.
I think the major consequence is the lack of responsibility of individual authors in a multi-author study. With increased specialization, there are fewer and fewer authors who see the big picture and who are capable of integrating the results from several subspecialties. The fact that the studies include work from several highly specialized techniques that only a few people understand also makes it harder for the average reader to evaluate the paper.

It's likely, in my opinion, that many of the authors on the paper don't fully understand the techniques being used by their colleagues. This is a big change from the science I grew up with.

Cordero et al. are worried about the possibility of fraud.
The growth in authors brings with it the concerns about the possibility that as more authors are added, there is an increased likelihood of some individuals with reduced integrity and capable of misconduct joining the group. In this regard, we note that the inclusion of one individual who has been accused of misconduct in numerous studies has led to dozens of retractions of scientific publications.
This is a very real danger but I think that outright fraud is not a significant worry. What concerns me more is the tendency to gloss over the limitations and possible misinterpretations of complex data analyses. The specialist who performs these analyses probably doesn't intend to misrepresent or exaggerate the significance of the result; it's just that they have become so used to using a particular technique (i.e. a software package) that they have forgotten those limitations. They don't communicate them to their colleagues who, because they don't understand the technique, don't realize there's a problem.

Cordero et al. summarize their results ....
In summary, our study documents a change in the literature of the biological sciences toward publications with more data over time. The causes for these trends are complex and probably include increasing experimental options and changes to the culture of science. At first glance, this data could be interpreted as a cultural change opposite to data fragmentation practices. However, it is also possible that an increase in publication density can still occur over a ‘salami slicing’ culture if the publication unit to be segregated is larger to begin with, as the result of technological improvements and increasing numbers of scientific authors. The benefits and debits of this trend for the scientific process are uncertain at this time but it is clear that there have been major changes to the nature of scientific publications in the past two decades that are likely to have major repercussions in all aspects of the scientific enterprise.
I think they're on to something.


Cordero, R. J., de León-Rodriguez, C. M., Alvarado-Torres, J. K., Rodriguez, A. R., and Casadevall, A. (2016). Life Science’s Average Publishable Unit (APU) Has Increased over the Past Two Decades. PloS one, 11(6), e0156983. [doi: 10.1371/journal.pone.0156983]

7 comments :

  1. I'd recommend reading Andrew Robinson's "The Last Man Who Knew Everything" about the 18th century polymath Thomas Young, who was a productive scientist in biology, medicine, physics, and even linguistics. The trend towards specialization has been going on for centuries. It's inevitable that the more we know, the smaller percentage of it any one person can know.

    ReplyDelete
    Replies
    1. That's not what I mean about specialization. The problem isn't specialized knowledge, it's specialized technology.

      Delete
    2. Can you make that distinction more precise? As far as I can tell, the issue is specialized knowledge, a part of which is knowledge about specific methods.

      Delete
    3. Indeed. It is meaningless to attempt to distinguish the two. Microscopy, and hence the basis of our knowledge of cells, was once a "specialized technology" -- arguably electron microscopy still is. Likewise knowledge of crystallography and NMR which we all rely on to get molecular structures, although most of us couldn't say exactly how. Computational methods, which Larry seems to be uncomfortable with, are simply another branch of knowledge which some people have more familiarity with than others but which we all rely on in the modern era.

      Delete
    4. @Jonathan Badger, despite its broad use, microscopy remains a specialized technique/technology. As a microscopist I spend a lot of my time pulling out my hair when I read the literature, as people are not aware of the limitations of microscopy, and therefore draw inappropriate/impossible conclusions from their images. Improvements in the tech has made it easy to use; significant expertise is still required for interpretation - which I think was one of Larry's main points.

      The one point that wasn't made in the article is the very negative effect this increase in APU has had on early and early-middle career investigators. Given the limited budgets these groups face, it can be very difficult to achieve the degree of productivity expected for tenure/promotion/grant renewal/first grant/etc - especially given that many of those standards are based on historical norms. My first paper as a PI had nearly 3X the data of my first paper as a PhD student, and took nearly 3X the time to produce - despite appearing in the exact same journal.

      Delete
  2. What is your opinion based on Larry? The last paper you have published or the one before that?

    ReplyDelete
  3. "What concerns me more is the tendency to gloss over the limitations and possible misinterpretations of complex data analyses."

    Oh boy, god knows I can relate to this one... :(

    ReplyDelete