Student Evaluations Don't Mean Much

Inside Higher Ed has just commented on a new study of student evaluations [New Questions on Student Evaluations]. The results are not surprising. They confirm all previous studies showing that student evaluations aren't what everyone thinks they are.

Previous studies suggested that students are rating generosity and personality and not quality of teaching. For example, a study of ratings on RateMyProfessor [‘Hotness’ and Quality] showed that,

... the hotter and easier professors are, the more likely they’ll get rated as a good teacher. As far as students — or whoever is rating professors on the open Rate My Professor site — are concerned, nothing predicts a quality instructor like hotness.

The new study from Ohio State University finds "... a strong correlation between grades in a course and reviews of professors, such that it is clear that students are rewarding those who reward them." Duh!

Now a cynic might say that this simply means that good teachers are doing such a good job that their students get higher grades. Thus, the evaluations truly represent the quality of the teacher and not how easy they mark. Well, that's not what the study suggests,

The Ohio State study, however, provides evidence for the more cynical/realistic interpretation — namely that professors who are easy (and aren’t necessarily the best teachers) earn good ratings. The way the Ohio State team did this was to look at grades in subsequent classes that would have relied on the learning in the class in which the students’ evaluations were studied. Their finding: no correlation between professor evaluations and the learning that is actually taking place.

The authors of the report show that student evaluations are practically worthless but in the interest of appeasing students they close with a mealy-mouthed sop as reported on the Inside Higher Ed site,

The authors stress that there are many ways — such as adjusting for student bias for easy graders or bias against certain groups of instructors — to continue to use student evaluations as one tool for measuring professors’ performance. But they write that, used alone and unadjusted, they appear highly questionable.

Let's see if I understand this logic .... student evaluations are biased and useless but instead of abolishing them we continue to use them to measure Professor's performance as long as we use other criteria as well.

Why? Why not get rid of student evaluations? We've known for decades that they don't work. Let's try and find another way for students and Professors to work together to improve university education. Student evaluations are ignored by all responsible Professors and they give students the false impression that their opinion is valued.

There has to be a better way. I believe that university students can provide useful and constructive criticism but only if they give up on the popularity contest and stop pretending that it has anything to do with quality of learning.

(As I write this, I'm supposed to be making up exam questions. I think I'll make some of them a bit easier .... )

[Hat Tip: Uncertain Principles]