Saturday, August 28, 2010

The Scientific Method

 
Over at Respectful Insolence, Orac picked up on a comment by Dr. R.W. and said [Some excellent questions for medical reporters] ....
An excellent idea, and Dr. R.W. has a list of some things that every medical journalist should know. My favorites? These:
  • Outline the scientific method. (I'm betting there are a lot of journalists out there reporting on medicine who can't outline the scientific method.)
  • Explain why consideration of biologic plausibility is important in the evaluation of health claims and why evidence based medicine often fails when biologic plausibility is not taken into account. (This one is hard, but knowing the answer would eliminate a lot of truly ignorant articles like the one Parker-Pope wrote yesterday.)
I was surprised that Orac picked teaching the scientific method as one of his favorites. I don't think there's any such thing as a "scientifc method" that garners majority support among scientists and/or philosophers. This led to a discussion in the comments section of his post.

What do you think? Is there a "scientific method"? If so, what is it?


22 comments :

  1. I think that there are ways of doing science that all have some features in common. I don't think that it is necessary to follow a particular sequence of steps as "the scientific method." Unfortunately, the latter is common in education.

    ReplyDelete
  2. Regardless of whether there is some unique feature that is common to all methods actually used by scientists and whose absence indicates non-scientific-ness, most people who talk about "the scientific method" usually talk about Popper's falsificationism, i.e. inventing hypotheses, making deductions (i.e. coming up with predictions), and trying to falsify those experimentally. Since this picture excludes too much of what we regard as good science, claims about "the scientific method" are usually wrong regardless of whether there is a unique "core" scientific method.

    ReplyDelete
  3. Popper's falsification is the only idea that, apparently, can never be falsified...

    1) The proof is in the pudding. If there was a consensus "Scientific Method," and it was useful, wouldn't be part of every science department's graduate program? In other words, doesn't the fact that we don't teach "Scientific Method" in grad school say something?
    2) An excellent book about the demarcation problem is "What is this thing called Science?" by Alan Chalmers.

    ReplyDelete
  4. I think there's more of a "scientific attitude" rather than a "sceintific method". And the great features of that attitude are critical thinking and favouring empirical data whenever possible. So at the end "bad science" is basically "bad thinking" and/or making stuff up gratuitously.

    ReplyDelete
  5. A good start to approach this problem is to ask if mathematics is a science: http://en.wikipedia.org/wiki/Mathematics#Mathematics_as_science

    I think mathematics is something that is discovered and therefore subject to empirical criticism.

    http://lesswrong.com/lw/1zt/the_mathematical_universe_the_map_that_is_the/

    ReplyDelete
  6. How about "checking stuff out to see if it's true?"

    ReplyDelete
  7. This is very much on my mind lately as I start my first semester of an assistant professor-ship at a community college. This semester I am teaching 2 different introductory biology courses for non-majors and an important question is "what exactly do they NEED to know?". Of course, covering your college's curricula is a good start, but what do you do with The Scientific Method? Is it really so important that non-scientists understand each distilled "step" (and will they even remember them after the class?)? I show them the standard minimalist list of steps (Obsservations, form a question, etc etc) and then tell them that this is a gross oversimplification and that, while it DOES happen at times it is embedded in a much much much larger and more complicated process. I then show one of the really complicated "scientific method" figures you can find on the internet that includes collaboration, manuscript rejections and blind stabs in the dark. I then emphasize that the important part is that, when science is being done "at its best" (don't ask me what I mean by this) the scientists are willing to give up what they know as "facts" if new observations pop up that contradict it. I said willing, not that they always do (or should). The other thing I emphasize is the difference between hypothesis and theory, and how "theory" definitions outside of science differ from the formal scientific word. Wow, I need to post this in my blog. Thanks for listening to this... any feedback/suggestions for how I should "teach about science"? I was a researcher for almost 15 years, but its interesting how little researchers think about the process they are using while doing it.

    ReplyDelete
  8. No, there isn't such a thing as "the scientific method". The closest we can come is "follow your curiosity wherever it leads you.

    ReplyDelete
  9. I made a t-shirt that I think addresses this very question.

    Picture 1

    Picture 2

    ReplyDelete
  10. Yeah, I think there are methods/procedures for trying to gain knowledge that can collectively be called "scientific."

    What do you mean by "science," Larry, when you say that science and religion are incompatible ways of knowing?

    ReplyDelete
  11. Jud asks,

    What do you mean by "science," Larry, when you say that science and religion are incompatible ways of knowing?

    Science is a way of knowing characterized by rationality, evidence, and skepticism.

    With the possible exception of strict Deism, there don't seem to be any religious beliefs that are compatible with the scientific way of knowing.

    In addition, I reject the idea that religion (or theism) is a valid way of knowing. Nobody has provided examples of a religious way of knowing that produces anything close to what we would call "knowledge." (Think of knowledge as a short-hand for "universal truth"—something that most people would accept as a correct explanation.)

    ReplyDelete
  12. Dr. Moran writes: Science is a way of knowing characterized by rationality, evidence, and skepticism.

    * * *

    Think of knowledge as a short-hand for "universal truth"—something that most people would accept as a correct explanation.

    I think this gives us the beginnings of a journey toward a definition of "scientific method."

    ReplyDelete
  13. I just finished having my say about this in our department's first year college course called "introduction to biological thinking."
    What I do is avoid talking about a scientific method, and instead talk about scientific reasoning that has these features: argue from evidence, materialism, and hypothesis testing. I also state and give examples of science that don't quite fit this model (like the human genome project). I also point out that most scientists don't think about this stuff. They just try to figure out how things work, try to be honest about it, and in Feynman's phrase, "try not to fool themselves."

    ReplyDelete
  14. Maybe I'm just stuck in the eleventh grade, but I always have the following chambered as what the "scientific method" is: observation, hypothesis, experiment, results, conclusion, peer review.

    If most lay-persons just understood what this process was, perhaps we'd clear up a lot of the hogwash. Perhaps.

    Or maybe I'm stuck in the eleventh grade....

    ReplyDelete
  15. Heathen Mike says,

    Maybe I'm just stuck in the eleventh grade, but I always have the following chambered as what the "scientific method" is: observation, hypothesis, experiment, results, conclusion, peer review.

    If most lay-persons just understood what this process was, perhaps we'd clear up a lot of the hogwash. Perhaps.


    There are many things wrong with teaching this simplistic version of "the scientific method."

    1. It's wrong. That's not how most science is done. Lots of excellent science does not require experiments. (The "method" was based on physics and chemistry.)

    2. It detracts from the most important aspects of science as a way of knowing; namely, rationalism, skepticism, and an emphasis on how things fit into the big picture (consistency).

    3. I plays into the hands of those who view the world in very simplistic, black-and-white terms.

    4. It lends support to a lot of bad science. There are many scientific papers that follow this "method" but still reach incorrect conclusions (Darwin was wrong! There's no such thing as junk DNA. Evolutionary psychology.).

    5. It promotes the idea that scientific thinking can only be done in the natural sciences, not in history, politcs, or education.

    6. It makes science sound like a robotic process, ignoring the creativity that characterizes the best of scientific thinking.

    ReplyDelete
  16. Ok, I get your criticisms and I think they're valid from a professor's viewpoint. What Orac was referring to and what I was talking about was what lay persons should be aware of and understand.

    I read the comment thread over at Orac's blog and it seems like you are arguing about a higher level discussion about what science is and what methodology(ies) should be used to find out about the world around us in every branch of inquiry.

    All we are talking about is what average Joe-on-the-street should know. People should know that the Earth is a sphere - no lay person needs to get into oblate spheroids.

    ReplyDelete
  17. Heathen Mikes says,

    Ok, I get your criticisms and I think they're valid from a professor's viewpoint. What Orac was referring to and what I was talking about was what lay persons should be aware of and understand.

    What I'm talking about is what the average lay person, science journalist, and student, should know about science.

    Form that perspective, I think you are Orac are wrong. I want these groups to know the truth about what science is and what it isn't. I don't think it's productive to teach them things that we know to be incorrect. That's not a very good example of scientific thinking.

    Are you a fan of "framing"? :-)

    ReplyDelete
  18. No, not a fan of framing, I just think that asking everyone from university professors to lay journalists and non-scientists to have a deep and nuanced understanding of what science is is asking too much at this point. I mean, it's obviously asking too much.

    If people just understood that homeopathy and reiki (among many others) have conclusions that don't follow from their basic ridiculous premises, we wouldn't have homeopathy or reiki.

    Understanding that peer review just means that the consensus of experts is what matters, not one lone "renegade" - even though that one outsider may one day be proven to be right, but it's not likely.

    There are levels of understanding and for most people, the "grade school" one is the only one they'll need, and it would serve them well.

    ReplyDelete
  19. Heathen Mike said:
    ...the "grade school" one is the only one they'll need, and it would serve them well.

    I disagree. If most lay people thought that science consisted of "the scientific method" alone, then much of good science would be viewed not with healthy skepticism but with abject contempt. Almost nothing from the 19th century would be accepted under this view of science. Moreover, as Larry pointed out a lot of crap still would be accepted because it went through the process and came out OK despite the failures of peer-review.

    I think that the only reason to support the stepwise model in educating laypersons is because we've given up on educating them on the formal meanings of words like skepticism and rational. Too many lay people use the word "skeptical" to mean "I think you're wrong because your explanation doesn't fit my preconception." They use the word rational to mean "common sense." In other words, there is a plague of horrifically sloppy thinking in our supposedly well-educated society. I would rather fix this by teaching people, including grade-schoolers reality in as accurate a way as I can than to give them a quick and all too dirty set of Cliff's Notes.

    ReplyDelete
  20. I always taught my students that there is a scientific method, and that it's pretty general:

    (1) Get an idea about what's going on. (2) Before assuming that it's true, test it in some way. Often, you make a prediction of something that will be true if your idea is true, and then go see if that prediction is true.

    The "scientific method" isn't exclusive to science. Little kids do it when learning about how the world works. Good auto mechanics do it before ordering the $400 part. (Not all auto mechanics are good.)

    The scientific method can be simple. It's almost common sense. Effectively testing some ideas is difficult, of course. And even if an idea passes the first tests and seems true, later observations may over throw it.

    ReplyDelete
  21. Larry, it should be relatively easy to incorporate most or all your 6 points into a slightly expanded definition/presentation of what scientific method is and how it works, shouldn’t it? Why not submit a rewrite/addition to Wikipedia? Or even better, a very short paper to a journal of your choice? Bottom line is that there does need to some sort of generally agreed upon definition of scientific method. Many educators seem to use the basic definition and incorporate at least some of the additional points you make.

    The biggest problem with the blanket statement that “There is no such thing as scientific method” is that it plays into the hands of two groups that are enemies of science: (1) ivory-tower academics (relativists, post-modernists, extreme leftists, etc.) who suffer from science envy, do not understand science, and/or fear/ distrust science for any number of irrational reasons. These types love to denigrate science in order to vainly make it seem as though their way(s) of doing “research” is every bit as good as science; and (2) scientifically illiterate purveyors of the hundreds of garbage pseudoscience ideas out there (all forms of creationism, homeopathy, magnetic bracelets, sure-fire miracle cancer cures, cold fusion, etc. etc. etc.)

    Witness the Sokal hoax in 1996 where the ivory-tower academic pseudoscience types were made to look like the ridiculous fools that they are. What was their response? Merely to break out into apoplectic pompous academic bluster attacking Sokal. They had no useful response, and they were never going to admit how incredibly stupid Sokal made them look.

    How about the SSHRC debacle in 2006 where funding was refused to Dr. Brian Alters for a proposed study on "the detrimental effects of popularizing anti-evolution's intelligent design theory on Canadian students, teachers, parents, administrators and policymakers."

    One of the reasons SSHRC provided for the denial of funding was:

    “Nor did the committee consider that there was adequate justification for the assumption in the proposal that the theory of Evolution, and not Intelligent Design Theory, was correct. . . .” ?!?!?!?!

    This certainly seems to indicate they have no understanding of what science is or how it works if they are somehow trying to put idiot design on par with evolution. And they have the temerity to include Science in the name of their organization?? Isn’t this a wonderful example of scientifically illiterate lefties playing into the hands of scientifically illiterate right wing fundagelical morons??

    WKM

    ReplyDelete