Wednesday, November 06, 2013

The Adaptation Assessment Probe

I'm taking a MOOC on evolution that's designed for educators [Evolution: A Course for Educators]. One of the things that was covered in the first lecture was a test on "adpatation" taken from a book called "Uncovering Student Ideas in Science, Volume 4: 25 New Formative Assessment Probes. The book is published by the National Science Teachers Association (NSTA).

Let's take the test ...
Adaptation

Three friends were arguing about what would happen if a population of rabbits from a warm, southern climate were moved to a cold, northern climate.1 This is what they said:

Bernie: "I think all of the rabbits will try to adapt to the change."

Leo: "I think most of the rabbits will try to adapt to the change."

Phoebe: "I think few or none of the rabbits will try to adapt to the change."

Which person do you most agree with and why? Explain your ideas about adaptation.
I agree with Bernie. I think all the rabbits will try to adapt to the colder weather by finding warmer, more cozy, burrows and by cutting down on their activity during the cold nights. I think they will adapt by eating more. If hair growth is related to temperature, as it is in some mammals, then the rabbits will adapt by growing thicker coats.

Let's see how I did.

Oops! That's not the "right" answer. The correct answer is what Phoebe said. Here's what they say on the website ...
The best answer is Phoebe's: "I think few or none of the rabbits will try to adapt to the change." The key word here is try. Biological adaptation involves genetic variation that allows some individuals to survive a particular change, such as a change in the environment, better than others.
I didn't read the question carefully. I didn't notice that what they were asking about was not just "adaptation" but "adaptation by natural selection." Silly me.

We turned our clocks back one hour last weekend and I'm still adapting adjusting to the change.

If this is the kind of nonsense that the National Science Teachers Association thinks is important then it's no wonder that evolution education is in trouble.


1. Not a good test for Australian students! :-)

36 comments :

  1. Since this is a course aimed at educators, it is perhaps a course developed by educators. I am very accustomed to their habit of using unstated assumptions and opaque jargon. I run into it regularly when trying to help my kids with homework. The directions they write would send a bus driver over a cliff if they were driving directions.

    ReplyDelete
  2. 'Adapt' isn't opaque jargon. It's a word that has slightly different meanings in different contexts. In the context of a course on teaching evolution, it's not unreasonable to assume that its meaning is 'heritable change that increases fitness'. But maybe you can suggest better wording that still assesses the same issue.

    One or the things that MOOCs are very good at (for the people who teach them) is showing the unexpected ways that quiz questions can be interpreted by students. In Useful Genetics we've had to go back and rewrite many of our quiz questions because the wording we initially used wasn't sufficiently unambiguous.

    ReplyDelete
    Replies
    1. Yes, rewriting questions is essential. But in this case, italicizing Phoebe's "try" would be all that's needed?

      Delete
    2. But maybe you can suggest better wording that still assesses the same issue.

      "Three friends were arguing about how natural selection would happen if a population ..."

      Delete
    3. At face value, the question sounds like it's asking about the particular, individual, rabbits that were moved (and Larry's answer would also be mine). If it's asking about populational change over generations, then it needs to say so.

      Delete
    4. Regardless of whether "adapt" includes mechanisms other than natural selection, I still don't see how "try" is anything other than a misconception that needs to be corrected (which was the point of the exercise). The only evolutionary mechanism that involves animals "trying" to adapt is Lamarckian (the cartoon version of it anyway; what Lamarck actually wrote isn't quite as silly).

      Delete
    5. The problem with the wording in those questions, from my point of view, is that you simply don't know if the author's are using the word "trying" as a metaphor (in which case Larry would be correct) or if they are using it in a non-metaphorical way (in which case the second option would be correct). You can't know a priori in what sense the wording was chosen, specially because I've seen this kind of careless wording many times before.

      Personaly, when I read that, I chose the second option, because organims don't try to adap (in the NS sense) to anything, but I thought that was not the author's intention.

      These questions only make sense if it is clear that they are not using "try" in a loose sense, and you can't possibly know it a priori.

      Delete
    6. Actually, none of the answers would be correct. Phoebe says "few OR none". Clearly, NONE is the only correct answer, since no rabit will "try" to adapt in the NS sense.

      Delete
    7. Jonathan Badger says,

      Regardless of whether "adapt" includes mechanisms other than natural selection, I still don't see how "try" is anything other than a misconception that needs to be corrected ...

      You don't really mean that, do you? Read this article on altitude training to see how athletes TRY to adapt to high altitudes in order to gain a competitive advantage. It shows that the word "try" is perfectly suitable when you are using some definitions of "adapt."

      Delete
    8. """Regardless of whether "adapt" includes mechanisms other than natural selection..."""

      What other mechanisms for *adaptation* are there besides NS?

      Delete
    9. That meaning of "adapt" is not part of evolution (unless you believe in the inheritance of acquired characteristics), so I don't see how that's relevant.

      Delete
    10. Pedro -- lots of other methods *in theory*-- that's kind of Arlin Stoltzfus' whole point of what he calls "constructive neutral evolution". In practice, I'm not sure if any actual adaptations can convincingly arise without the help of NS.

      Delete
    11. I asked the question because it seemed that by unintentional mistake while writing you could have conflated adaptation with evolution. I don't doubt you know the difference, but I could have missed something. English is not my native language.

      Delete
    12. That meaning of "adapt" is not part of evolution ...

      Perhaps not part of "evolution" but part of biology. If teachers wanted this question to only be about natural selection then they should have said so. Otherwise, Bernie and Leo are tight.

      Delete
  3. Since populations adapt in the evolutionary sense, Bernie's phrasing isn't very good either. All the individuals would adapt in the sense of getting used to or coping with the change. Given that Social Darwinism in some form or another is pretty much in the air, unlike modern evolutionary theory, I think that trying to clarify that evolutionary success is not a triumph of the will has something to be said for it. Ticking off the student with a trick question versus an unexpected answer that might serve as a signpost for a conceptual difficulty? Sounds like an excellent question for a self-graded quiz if the class has students who take anything that doesn't pay off in points seriously.

    ReplyDelete
    Replies
    1. I disagree that it sounds like an excellent question. An "excellent question" needs to be clear so that it gets a valid answer (correct or not). In these questions, I would not know if the word "trying" is being used carelessly in a loose, metaphorical sense (bad wording/phrasing) or if the authors actually mean "try" literally. Unless the reader knows clearly what is being asked, any answer is useless.

      Delete
    2. The thing is, neither a loose nor a literal meaning of the word "try," has anything at all to do with adaptation in the evolutionary sense. And, the context of a lesson or unit on evolution means the natural presumption is to read the question as about evolutionary adaptation.

      The unconscious equation of personal effort with success regardless of objective circumstances (both good or bad,) indeed the very notion that individuals instead of populations are evolving are the context in which "try" fits. The seeming ambiguity of the question comes almost entirely from the preconceptions.

      A question that confronts a preconception is an excellent question. I think it is obvious that this particular example could use some built-in hints to try to force the student to a correct answer. But I can't really agree that the purpose is nonsense, however irritating it is to be "tricked" by an unwholesome subtlety in questions.

      But in the long run science is about learning how to find the answers, not about the smart students being able to score points. An indispensable part of this is learning to ask the right questions. In evolutionary biology, there are very few questions that involve "try." In one way this is a very simple point.

      As to whether it's an excellent question when used in a certain way depends entirely upon whether the student understands the point that "trying" to adapt is nonsense in evolutionary explanations, As I said, I think this example can use more hints, because students fixated on points resent being "tricked." And this is true even when there isn't really a trick, but preconceptions instead.

      Delete
    3. """But I can't really agree that the purpose is nonsense, however irritating it is to be "tricked" by an unwholesome subtlety in questions."""

      I wasn't referring to the "purpose" of the question, which seems perfectly valid. The point here is that it is poorly phrased, from the unclear meaning of the term "try" to the concious or unconscious suggestion that individuals and not populations adapt.

      So maybe we do agree, but we were talking past each other.

      Delete
    4. S Johnson says,

      A question that confronts a preconception is an excellent question.

      I couldn't agree more. If the misconception being addressed is whether individuals can evolve by natural selection then that's worth discussing. If the misconception being addressed is whether an organism can "try" to evolve then that's also worth addressing. If the misconception is about distinguishing between multiple meanings of the word "adapt" then that's important as well.

      The problem here is that the question addresses one misconception (individuals trying to evolve) but gets hung up on another misconception (that the word "adapt" always means evolution by natural selection).

      Delete
    5. Moran said:

      """The problem here is that the question addresses one misconception (individuals trying to evolve) but gets hung up on another misconception (that the word "adapt" always means evolution by natural selection)."""

      I didn't think that was a problem because since the point of the whole exercise was to confront misconceptions in Evolution I took for granted that the use of the word "adapt" was related to evolutionary adaptation. I found that the use of the word "try" and the suggestion of individuals evolving instead of populations were far more ambiguous, because it is so common to use bad phrasing.

      Delete
  4. I'm reminded of an exercise that John Freshwater used in his class that said animals have to recognize that an improvement is needed before evolution can occur.

    ReplyDelete
  5. I think all the rabbits will die during the winter, whatever they may 'try". Is the claim being made here that natural selection is a choice made by individuals? Or is it intended to drive home Yoda's point "Do or not do; there is no 'try'"? This question is so bad that I can't tell what they're trying to ask, even after reading the explanation.

    The problem you raise, about the multiple meanings of "adapt", isn't even the biggest flaw.

    ReplyDelete
  6. It's a terrible question. It seems from the answer that the point they were trying to make was that animals cannot intentionally 'try' to adapt to an environmental change; it depends on their existing genetic inheritance plus any mutations they might have had, So none of the three friends' answers was correct.

    ReplyDelete
  7. As a science teacher of HS biology, I also find the question poorly worded. First students would interpret try as a decision the organism is attempting to do. I think it is because of this wording that Larry answered the way he did. The rabbits chose to behave in response to the changes. From an evolutionary perspective I would not work it this way simply because we are constantly trying to STOP students from thinking about evolution as a choice individuals make. The context IS key but the wording needs to reflect it. IMHO - poorly worded and disappoints me that NSTA material would be so poorly designed.

    ReplyDelete
    Replies
    1. I found the question in the 'how to teach evolution' video section which even scares me more. A person working at the MoNH should know better. Or am I the one who is wrong here?

      Delete
    2. I agree. I was following along in the video and thinking they were making a good point. The word 'adaptation' is used by creationists in a way that I think may be unclear, so it seemed an especially good word to focus on to clarify its use in the context of biological evolution. And then their sample question invalidated their intention.

      Delete
  8. There's no problem with their answer. For instance, I've been trying really hard to grow wings as part of my commitment to becoming a more angelic person. It hasn't been successful so far but like they said it just doesn't always happen for you.

    Your answer, OTOH, would suggest that I try to become more angelic by cutting down my sarcasm (coincidentally also my girlfriend's suggestion) but that just isn't in the cards.

    ReplyDelete
  9. I got it right away. however the clue was it would be the girl name who got it right. There is a complaint and agenda to bring more girls into sciency stuff. I see it everywhere. They are counting identities and deciding what is the right answers.

    they are making the point its unguided selection because people think evolution is about guided evolution from some innate ability.
    creationists believe if everyone knew evolution is about happanchance on mutations then they would be more skeptical then they are right now.
    Oddly enough the present support is based on seeing evolution as a real mechanism working with gears and pulleys in the biological entities.
    Turning bugs into buffalos by selectionism would seem impossible to many more people.
    This YEC creationist welcomes a more educated population on how evolution is claimed to work.

    i do think there are innate triggers for rabbits as to explain their changing colour and growing hair etc just as needed. Just like people.

    ReplyDelete
  10. Binky is trying to learn GROW LONGER FUR! But Binky already knows four moves! Forget a move to learn GROW LONGER FUR?

    ReplyDelete
  11. Students have a strong tendency to assume that organisms strive to change evolutionarily. Students have a lot of trouble wrapping their minds around the concepts that evolution happens to populations, not individuals, and that organisms don't try to do it. I have to spend far too much time working on this idea with intro bio students. I agree that this test question is awkward, but it's getting at something important that students need to know. The question should be improved (NONE of those bunnies were trying to adapt in the evolutionary sense of the word) but it should be asked.

    ReplyDelete
    Replies
    1. I think everyone here agrees that the question was important. The problem is the phrasing, and the fact that none of the answers is actually correct.

      Delete
    2. The irony is that it IS an important issue. You would think that the teachers would go out of their way to avoid any ambiguity and focus like a laser on exactly what the problem is. They could have asked specifically about natural selection and they could have had one of the students give a correct answer about populations, mutations, and many generations.

      There is way too much sloppy thinking in the field of evolution education.

      Delete
  12. Although I agree the question could have been worded better, formative probes are not "test" questions. Probes are designed to uncover student misconceptions BEFORE teaching a lesson so that student misconceptions can be identified and addressed directly during the proceeding instruction.

    ReplyDelete
    Replies
    1. Excellent! This question will certainly uncover a "misconception" about the multiple meanings of "adapt."

      Delete
    2. """Probes are designed to uncover student misconceptions BEFORE teaching a lesson so that student misconceptions can be identified and addressed directly during the proceeding instruction"""

      Then this is a pretty badly designed "probe". If your question is ambiguous (as is the case), all answers will be worthless by default. Just look at the different answers here at this very blog. Do you think Moran and all the others who posted comments here have many misconceptions in evolutionary theory? Or isn't a much better explanation that the "probe" is just bad?

      Delete