Monday, January 29, 2007

Student Evaluations Don't Mean Much

Inside Higher Ed has just commented on a new study of student evaluations [New Questions on Student Evaluations]. The results are not surprising. They confirm all previous studies showing that student evaluations aren't what everyone thinks they are.

Previous studies suggested that students are rating generosity and personality and not quality of teaching. For example, a study of ratings on RateMyProfessor [‘Hotness’ and Quality] showed that,
... the hotter and easier professors are, the more likely they’ll get rated as a good teacher. As far as students — or whoever is rating professors on the open Rate My Professor site — are concerned, nothing predicts a quality instructor like hotness.
The new study from Ohio State University finds "... a strong correlation between grades in a course and reviews of professors, such that it is clear that students are rewarding those who reward them." Duh!

Now a cynic might say that this simply means that good teachers are doing such a good job that their students get higher grades. Thus, the evaluations truly represent the quality of the teacher and not how easy they mark. Well, that's not what the study suggests,
The Ohio State study, however, provides evidence for the more cynical/realistic interpretation — namely that professors who are easy (and aren’t necessarily the best teachers) earn good ratings. The way the Ohio State team did this was to look at grades in subsequent classes that would have relied on the learning in the class in which the students’ evaluations were studied. Their finding: no correlation between professor evaluations and the learning that is actually taking place.
The authors of the report show that student evaluations are practically worthless but in the interest of appeasing students they close with a mealy-mouthed sop as reported on the Inside Higher Ed site,
The authors stress that there are many ways — such as adjusting for student bias for easy graders or bias against certain groups of instructors — to continue to use student evaluations as one tool for measuring professors’ performance. But they write that, used alone and unadjusted, they appear highly questionable.
Let's see if I understand this logic .... student evaluations are biased and useless but instead of abolishing them we continue to use them to measure Professor's performance as long as we use other criteria as well.

Why? Why not get rid of student evaluations? We've known for decades that they don't work. Let's try and find another way for students and Professors to work together to improve university education. Student evaluations are ignored by all responsible Professors and they give students the false impression that their opinion is valued.

There has to be a better way. I believe that university students can provide useful and constructive criticism but only if they give up on the popularity contest and stop pretending that it has anything to do with quality of learning.

(As I write this, I'm supposed to be making up exam questions. I think I'll make some of them a bit easier .... )

[Hat Tip: Uncertain Principles]

9 comments:

  1. Sadly I find that student grades and teachers don't have in a general scope, any correlation at all.

    In fact, I have been at university for quite some time (I'm working student) and have had the chance to try certain classes with different teachers from year to year. It's odd to see that given the same program for the class, that a change in teacher makes passing rates go from 10% to 95%! And obviously, everyone thinks the latter is a better teacher!

    So basically, what I take from this is that given the same class program, each teacher is a different class in his/her self.

    ReplyDelete
  2. For example, a study of ratings on RateMyProfessor

    Oh come on. RMP is a resource for students alone to let them know what they're getting with a professor. You can't determine anything concrete from them.

    I never rate anyone on RMP hot or not because I don't care.

    ReplyDelete
  3. Obviously ratings are much about the teacher as a person. That isn't unimportant in a teaching situation. I can remember a certain math teacher who shouldn't have been forced to be one because she could not communicate if so her life depended on it. And she seemed as unhappy about the situation as us students. (Well, even that she didn't communicate well. Totally unreadable in every way.) Which is why students skipped most of her classes and made special study groups to compensate.

    So yes, student evaluations should tell us something. I'm not sure about exactly what it is, though. :-)

    ReplyDelete
  4. Yet for lack of a better forum, student evaluations are the only thing out there. I doubt there can be a better medium to communicate student opinions, mainly because of apathy from all sides. Like you said (partisan paraphrasing!), many professors (especially tenured ones) don't care.

    On a lighter note, if teaching ability were quantifiable, there would most likely be a standard distribution... wouldn't most professors then, by their own rigorous marking standards, be only mediocre? :)

    Anyway, back to studying for your exam. :(

    ReplyDelete
  5. Sometimes it's even hard to tell how good a professor is at the end of the class. I took a class from one professor who seemed to talk down to his students, who never seemed to answer questions in quite the way I expected, and in general just seemed to keep me guessing. It was also a hard class, though I managed a good grade. But I had nothing good to say about the man.

    Six months later, I finally appreciated how much I had learned in that class, and how much had actually stayed with me. A year later I took another class from the professor, with the same result. I can't say that he taught me a lot, or that he just facilitated a great deal of learning. Either way, he's a very good teacher.

    Karen

    ReplyDelete
  6. With benefit of ... well, of a considerable number of years of hindsight, my best university teachers were those who were involved themselves and who found ways to challenge me to learn. I'm not sure that my opinion of them at the time would have been of much benefit to anyone.

    I recall a math professor who was able to communicate his own interest in a topic and his satisfaction that you had followed him and learned something.

    In other words, it was less "here's what I must teach you in this course" and more "here's what we'll have the opportunity to explore."

    ReplyDelete
  7. You should see how much weight these things are given- often by totally innumerate humanists who think rating averages taken to two decimal places mean something- in most small liberal arts colleges in the US. You'd have a stroke.

    ReplyDelete
  8. Hey !... we HATED the test,
    but we LOVE you ... !
    lol... just disproves that article I guess..... should increase the sample size ;)

    ReplyDelete
  9. This comment has been removed by the author.

    ReplyDelete