The Edge has asked people to describe whether they have changed their mind about anything and if so, why? It's the Annual Question for 2008.
Some of the replies are worth discussing. For example, Irene Pepperbreg has changed her mind about the meaning of the scientific method [The Fallacy of Hypothesis Testing]. I think she makes some good points, notably ....
Third, I've learned that the scientific community's emphasis on hypothesis-based research leads too many scientists to devise experiments to prove, rather than test, their hypotheses. Many journal submissions lack any discussion of alternative competing hypotheses: Researchers don't seem to realize that collecting data that are consistent with their original hypothesis doesn't mean that it is unconditionally true. Alternatively, they buy into the fallacy that absence of evidence for something is always evidence of its absence.I think this is a serious problem in science today. There are too many papers being published without any serious discussion of competing explanations. There are too many papers that fail to critically examine their own basic assumptions or the possible flaws in their experiments.
I'm all for rigor in scientific research — but let's emphasize the gathering of knowledge rather than the proving of a point.
There may be a reason for this behavior—scientists don't want to draw attention to possible flaws in their work for fear that the granting agency will find out—but that doesn't excuse it. Scientific rigor demands that you present both sides of a scientific debate in a fair and unbiased manner. The failure to address the arguments of your opponents is nothing less than failing to be a good scientist.
Similarly, the failure to recognize the possible flaws in one's own explanation is the mark of a bad scientist.
While Irene Pepperbreg may be right about the flaws in today's method of doing science, I'm not prepared to throw out the baby with the bathwater. Hypothesis-based science is still important. You just have to form the right hypotheses and put your work in context. The problem, in my opinion, isn't that hypothesis testing is a fallacy: the problem is that it's not being done properly.
2 comments :
Do most scientists really think in terms of proving/disproving a hypothesis, or is most science done as a means of answering a scientific question. If you were a physics researcher, would you try to prove/disprove the hypothesis stating that "the sky is blue" or would you instead devise experiments to answer the question "what color is the sky?"
NIH grants require you to state a hypothesis at the beginning of the application and this always seems like a weird, fudged kind of statement for most that I've read.
A plug here--if this subject interests you, we (CSHL Press) have recently published a really interesting book on Experimental Design that delves into these philosophical issues, and better yet, gives a great explanation of what controls one should really include in one's experiments (something sorely lacking these days). More info here:
http://tinyurl.com/2z488l
The problem, in my opinion, isn't that hypothesis testing is a fallacy: the problem is that it's not being done properly.
Yes!
In my own field, geology, we have had the problem of not paying sufficient attention to hypothesis testing. People have often simply gone out into the field, looked at a section, gone home and written that up, collecting evidence that ultimately supports their own particular theory. Someone else, working on the same problem but with a different theory, does the same and you end up with two groups both finding evidence for their side and ignoring the other explanation. A great example of this can be found in the relativey obscure arguments surrounding the nature of glaciation in the irish sea during the last glacial maximum. Testing multiple working hypotheses needs to be done much more often in geology (and is increasingly done so).
Post a Comment