Monday, May 09, 2016

John Oliver teaches us to be skeptical of scientific publications

We all know that the purpose of education should be to teach students how to think critically. We're not doing a very good job. Take biochemistry, for example. We spend a lot of time transferring information from lecture notes to student notes and then examining students on whether the transfer has worked. We think that teaching students to read the primary literature will make them better scientists when, in fact, teaching them to be skeptical of the primary literature is what's really necessary.

The ENCODE fiasco is just one of many examples where the scientific literature got it wrong. We need to make sure that our students appreciate the important parts of science; namely, the necessity of repeating experiments and the value of scientific consensus. Our students, and many of my colleagues, are prone to hype and promotion just like every one else but that's exactly what critical thinking is supposed to avoid. And it's exactly what proper science—no matter how you define it—is designed to overcome.

If students and scientists are having trouble with these concepts, imagine how difficult it is for the general public. How are they supposed to know that not every "breakthrough" is a real breakthrough and not every new study is correct?

John Oliver did an excellent job of explaining the problem on a recent (May 8, 2016) episode of Last Week Tonight. Watch it. It's worth 20 minutes of your time. The last bit on "Todd Talks" is classic.




30 comments :

  1. You have to admit the moment in the Todd talk where wearing a lab coat gives one more credibility is spot on!

    ReplyDelete
  2. Actually Larry, Oliver did a shitty job of defending science. On the one hand, he declares that science is done so poorly because the sharpest tool science has in the bag,re plication...you guessed it...is rarely done... for an slew of (excuses) reasons.

    Then, he goes right into how people should not be misusing the results of science in some TODD kind of way.

    But wait...isn't it because sooooo little science is actually replicated that people can so easily abuse results and conclusions?

    Hmmm.

    So Larry, maybe you oughta rething Oliver as your go to poster boy to bolster your claim that ENCODE is engaging in TODDish behavior.

    If anything, it is the shitty quality of the vast majority of what marginally passes for science that should the target of your wrath.

    But we know the real reason you diss ENCODE. They've got money, lots of money. And that means new technology not far off...which of course could embarrass the shit out of Larry Moran.

    So sure Larry, go right on ahead with that jDNA book....fast...before ENCODE does come up with some more uncontrovertible evidence that lends clear support to the claim the majority of ncDNA does in fact have higher order command and control functionality.

    Then you can claim that if only you had had that same technology ( at the time of the writing of your jDNA book), you would have certainly come to the same conclusion as ENCODE.

    Most certainly. Without a doubt.


    ReplyDelete
    Replies
    1. But we know the real reason you diss creationism and IDiots. They've got money, lots of money. And that means new technology not far off...which of course could embarrass the shit out of Larry Moran.

      Delete
    2. Wow, Steve, the collapse of Darwinism is at hand (again and/or still...)! Perhaps you'd like to hold your breath waiting?

      Delete
    3. Steve,

      Such a mean-spirited tirade. Will you even read the book when it is published? Maybe you can write a review of it from your internet-connected computer, sitting amid your modern amenities, vaccinated, well fed, not dead from any number of diseases, not scratching at fleas infesting your home, with running clean water. And you can tell us all about how science sucks.

      Unless that science can be misinterpreted to lend support to your ID/creationist fantasies. Maybe ENCODE will prove the whole genome is functional, which would not say a damn thing one way or the other whether evolutionary theory is true, but filtered through your ideological lenses will prove god did it. ENCODE will come to your rescue and save you from reality, Steve. Just have faith!

      Delete
    4. Mean spirited? When in Rome..........

      Oh, and you are absolutely right. If/when ENCODE does confirm the genome is fully functional, it won't make evolution totally wrong (but mostly).

      It will make Larry Moran wrong (absolutely).

      Delete
    5. judmarc, darwinism couldn't possibly collapse since it hasn't risen higher that Matzo.

      Do let us know when you detect any movement.

      Delete
    6. Steve,

      "Oh, and you are absolutely right. If/when ENCODE does confirm the genome is fully functional, it won't make evolution totally wrong (but mostly)."

      Explain how if ENCODE confirms the genome is fully functional, it will make evolution mostly wrong.

      "It will make Larry Moran wrong (absolutely)."

      As well as all current scientific knowledge on the subject. I hope you are not holding your breath waiting for this one. Maybe you could just pray on it instead.

      Delete
    7. ENCODE's great. Epigenetics is great. Alfred Russel Wallace, James Shapiro - all great. Why? Because they all, in their way, undermine darwinism. Probably.

      Delete
    8. easy Chris, jDNA is promoted as evidence that the genome is a kluge, doing endless, mindless trial and error, and the vast amount of jDNA is a product of that messy non-teleological mechanism of change.

      On the other hand, if/when ENCODE demonstrates that all these ALU and transposons and whatnot are in fact elaborate management tools, then it will put a stake in the heart of the notion of evolution as a non-teleological process.

      It will confirm that nature is teleological through and through; precisely what atheists like Larry Moran hope to avoid.

      Taking the teleology out of evolution has always been an irrational approach and still is, even after 150 years of Darwin.

      Delete
    9. Steve,

      Have you been paying attention to anything about ENCODE and functionality of the genome these past many months?

      "On the other hand, if/when ENCODE demonstrates that all these ALU and transposons and whatnot are in fact elaborate management tools, then it will put a stake in the heart of the notion of evolution as a non-teleological process.

      It will confirm that nature is teleological through and through; precisely what atheists like Larry Moran hope to avoid."

      How would a bunch more regulatory elements say anything about teleology? Teleology is just one of those vague terms ID/creationists use that they can't define, define, or quantify in real life. It amounts to nothing more than wishful thinking on your part.

      "Taking the teleology out of evolution has always been an irrational approach and still is, even after 150 years of Darwin."

      There was no teleology in evolution to take out. There is zero evidence of "teleology", planning or forethought in evolutionary processes, and therefore no reason to posit such a thing. If ID/creationists really wanted to make a case for their beliefs, they would be trying to empirically prove something unique to their "theory". But they never do. They have been predicting the demise of evolutionary theory for over a hundred years. Any day, now!

      Delete
  3. To continue in the same vein:

    http://www.vox.com/2014/12/20/7422377/science-retraction/in/7948804

    http://www.vox.com/2015/3/14/8203595/pubpeer

    ReplyDelete
    Replies
    1. just goes to show you Tom that organized science is susceptible to the same human weaknesses that affects organized religion.

      No one, i mean no one is immune to the desire for wealth and fame.

      At the end of the day, reproducibility and falsifiablity have not saved science from human desire, have not created a huge enough distinction from organized religion to take the high road.

      Science' marketing problem is as severe as ever.

      The day science solves its retraction and irreproducibility problems is the day the catholic church solves is pedophile problem.

      Fact is the church has more incentive and pressure to reform than science does. There is too much money and fame to be gotten from winning grants. For the church its downhill extremely fast if they dont reform.

      So my money is on the church reforming before science ever decides to even attempt to seriously address the problem.

      Delete
    2. Hey Steve,
      you know, I don't get it. You've been hanging around this blog for quite some time. You clearly spend lots of time coming up with all these 'contributions'. But the main theme of your 'contributions' consists of pure hatred toward science. And if it's not hatred, you're projecting the many inabilities of your religion to change on to science. Like this post above.

      This line is one of many:
      The day science solves its retraction and irreproducibility problems is the day the catholic church solves is pedophile problem.

      It's no surprise you're projecting the smoke screens all religious institutions are putting up to prevent the truth regarding the pedophiles in these organizations of coming out. Even in so-called advanced western countries committees investigating pedophile activity run into religious institutes stalling fact finding until the bastards who committed the crimes are dead and buried. Leaving the little boys they raped maimed for life, these little boys feeling guilty because no one will believe the priest/ imam etc. raped him.

      Now lets take a look at the few hundred retractions in science. Yes, people start believing in their pet theories so much, they start to photoshop experimental results. Or fake their experiments. But because the results of the experiments are public, everybody can try to reproduce the outcome in their own lab.
      It's all open to public. Nobody can hide behind a smoke screen, in the end it's clear who committed the crime of faking data and who will be fired --> The original author.
      It's the scientific method which discovers the crime of faking data, and it's all open to public.

      The person who committed the crime of faking scientific data will most likely be fired and perhaps he'll be sued to repay the money he/ she wasted.

      While the pedophile will remain sheltered within the religious institution, because it's an internal affair and god will judge in the end.

      So yeah, it's clear the severity of the crimes, pedophile vs retraction, are comparable, right Steve?

      Delete
    3. Ed, you miss the point.

      Atheists promote science as a more reliable alternative to religion to acquiring knowledge. They promote replicability and falsifiability as key characteristics that distinguish it and make it superior to religion.

      Yet the frequent violation of these 'pillars' of science render it a weak challenger to religion as 'a way of knowing'.

      Science has always been and will always be an argument from utility precisely because it has an inherent flaw it shares with religion; human weakness.

      Desire a better mousetrap? Go to science and technology.

      Desire understanding of whe we exist and have consciousness? Go to metaphysics and theology.

      So the point is not a contrast between pedophilia and retraction> Rather, it is: Does science avoid the traps humans set for themselves in other areas of human endeavor like theology/religion?

      The answer is science does not.

      So science should just stick to what it knows best, how to transform knowledge of the material aspects of life into useful objects.

      Delete
    4. "Atheists promote science as a more reliable alternative to religion to acquiring knowledge. They promote replicability and falsifiability as key characteristics that distinguish it and make it superior to religion."

      No, scientists do these things (not all scientists are atheists, and not all atheists are scientists). Atheism is simply the provisional conclusion that gods do not exist, there being no credible evidence to the contrary.

      "Yet the frequent violation of these 'pillars' of science render it a weak challenger to religion as 'a way of knowing'.

      Science has always been and will always be an argument from utility precisely because it has an inherent flaw it shares with religion; human weakness."

      Of course science is subject to human weakness, like every other human endeavor - because it is done by humans. Science, however, has a self-correcting capacity: scientists change their ideas about how things work based on empirical evidence. Religion does no such thing. Religion assumes it already knows the truth based on faith. That makes science a much more reliable way of knowing.

      "Desire understanding of whe we exist and have consciousness? Go to metaphysics and theology."

      Why do we exist, Steve?

      Delete
    5. So science should just stick to what it knows best, how to transform knowledge of the material aspects of life into useful objects.

      That would be engineering (or sub-specialties of it), so it appears you ought to stick to whatever you know best, which is not science.

      Delete
    6. At the end of the day, reproducibility and falsifiablity have not saved science from human desire, have not created a huge enough distinction from organized religion to take the high road.

      And that, kids, is how the Church discovered quantum physics, relativity, genomics, evolution, and the rest of modern knowledge!

      Delete
    7. and that kids is why science will always stay an argument from utility.

      Are you satified kids that quantum physics, relativity, genomics, and evolution have answered the most pressing questions in your life?

      I thought so.

      oh, but do please tip your hat to science' brainchild, technology for at least making life a bit more comfortable and convenient.

      It does ease the pain (somewhat).

      Delete
    8. I'm not sure what these "most pressing questions" are to you, but to many they are things like "Why do I exist?" "Which animals am I allowed to eat?" and "Who should be allowed to use the women's bathroom?" Such people have science to thank for the fact they do not need to worry about questions like "How do I avoid freezing to death, or dying before age 25 from things like dental infections?"

      Delete
    9. It does ease the pain (somewhat).

      Hmm, well let's consider the "argument from utility." You can take advantage of science and treat an illness and the patient won't die. Or you can fail to take advantage of science, the patient dies, and you can pray about it and reach "the peace that passeth all understanding."

      I guess it depends on whether you think "What causes that disease, and what can cure it?" are "big questions," or whether you should think of them as merely utilitarian and instead contemplate big questions like "Why do the people I love have to die?"

      Delete
    10. "Yet the frequent violation of these 'pillars' of science render it a weak challenger to religion as 'a way of knowing'."

      What, exactly, IS religion's "way of knowing"? Praying on it? What does that even mean? 'It is in scripture, therefore, we know it'?
      Tell us how religion is a way of knowing anything other than its own mantras and doctrines, if you can.

      Delete
    11. Manning,

      Rational thought and logical analysis.

      Science limits itself to the understanding of matter and its manipulation.

      'nuf said.

      Delete
    12. Steve,

      "Rational thought and logical analysis."

      That's not religion, that's philosophy. Religion is the assumption that some myths are real stuff. Religion is not a "way of knowing," but a way of making a fool out of yourself.

      Delete
    13. Well, technically this is any deductive discipline, which would include philosophy, but also mathematics and theology. I.e. anything where you start out with a couple of prepositions and then try to infer what they entail. It's not clear what is meant by "Religion as a way of knowing", but theology certainly qualifies. Catholic theology certainly is the only valid way of knowing the answer to the question "What do catholics have to believe?" (which is somewhat different from "What do catholics actually believe?" which is a socialogical question. I'm pretty sure that a rather solid number of them are Arianists, which of course makes them technically heretics).

      Delete
    14. start out with a couple of prepositions and then try to infer what they entail

      Could you help me with that a bit? I'm trying "off" and "through", and I can't get anywhere.

      Delete
    15. Typo is a typo. Somehow presupposition and proposition decided to run into one another there.

      Delete
  4. Even gadflies are a product of evolution.

    ReplyDelete
  5. Unless things have changed drastically since I did a sociological study of a chemistry lab in the 1980's, the idea that nobody does replications is absolutely bogus.
    The thing is, the labs weren't publishing single experiments; they were publishing "projects" or coordinated groups of related experiments. And, typically, the first experiments in the project would be replications of someone else's study, for which they were interested in pursuing an extension. They would replicate mainly to check their equipment and make sure they were on the same page. They would then run original experiments, and the conclusions of those experiments were the topic of their paper(s). The scientists whose findings they replicated would, ideally, get credit for being cited.
    Occasionally, of course, the results couldn't be replicated because they were wrong. In the one case I studied where that happened, the scientist who discovered the error did not rush into print with a "failure to replicate" paper. He worked and worked until he was sure his own, divergent results could be replicated. Then he published his own results, mentioning in the paper they failed to replicate the other lab's results. (Incidentally, the scientists with the bad results were professors, and the scientists who reported failing to replicate their results was a mere grad student. According to the ID mythology, the grad student should have had his career destroyed for taking on the establishment. In fact, as a result of this work, he was offered a post-doc at Harvard.)
    The moral to the story is, don't trust sociology of science studies (or, worse, philosophy of science "studies") that rely on analyzing publication data. If the social scientists haven't spent time in the lab seeing what really goes on, they aren't getting the true picture.

    ReplyDelete