Sunday, April 21, 2013

Judging the Quality of MOOCs

Another journalist has written about Massive Open Online Courses (MOOCs). This one is in The New York Times: Two Cheers for Web U!.

Most of these articles about MOOCs are not very good but this one is different. The author expresses some skepticism and hit the nail on the head when he says ...
But the first thing I learned? When it comes to Massive Open Online Courses, like those offered by Coursera, Udacity and edX, you can forget about the Socratic method.

The professor is, in most cases, out of students’ reach, only slightly more accessible than the pope or Thomas Pynchon.
But that's not what I want to talk about today. I want to discuss the quality of these courses and how you might go about judging whether they are truly teaching the subject correctly.

Here's the problem. Too many people, like A. J. Jacobs, the author of today's article, assume that because the lecturer is famous or from a "top" school, the material must be accurate and up-to-date. As A. J. Jacobs puts it ...
On the other hand, how can I really complain? I’m getting Ivy League (or Ivy League equivalent) wisdom free....

With the exception of a couple of clunkers — my plodding nutrition professor might want to drink more organic coffee before class — most of my MOOC teachers were impressive: knowledgeable, organized and well respected in their field.
Students are not in a position to judge whether a professor is "knowledgeable" about the material being covered in a course. In the case of MOOCs, many students just assume that because the professor is from an Ivy League school then he/she knows how to teach an introductory course properly.

That's a very bad assumption as I've shown when I examined the biochemistry material being taught in the MIT courses Where Are the Best University Teachers?. Same is true for the courses at the Khan Academy.

I'm currently in Boston at EB2013 where I'm hanging out with biochemistry and molecular biology teachers and textbook authors. Many of these experts are not household names but they are the experts in their field, which is teaching. If you really want accurate information about the fundamental principles and concepts in biochemistry and molecular biology then you should take the courses they teach. You'll get a far better education than if you listen to professors from big-name research intensive-universities.

The recent ENCODE publicity disaster is just one example of the fact that top-notch researchers don't necessarily understand the fundamentals of subjects that are just outside of their own area of expertise.

Let's try and put a stop to this myth that the best teachers are professors from Ivy League schools. There's very little evidence to support that myth, especially in fields that I'm familiar with: evolution, biochemistry, and molecular biology,


  1. Seems to me that to evaluate the relative merit of MOOCs vs Lectures it is necessary to do longitudinal studies of many students in different settings. Will students perform their jobs better down the road if they take one course vs the other? I assume that some of those are being done? At this point all I see are assertions.
    You mention, for example, that the online biochemistry course at MIT is poor. After following your link I saw the video and it seemed fine to me. As I am not a biochemist it would be useful to know why you think that lecture is poor.

    1. You're talking about the lectures on Glycolysis, Respiration, and Fermentation by Graham Walker.

      His very first sentence is, "Let's look at what was was probably the first energy-producing system that evolved ... glycolysis."

      This is wrong on so many levels that it's hard to know where to begin. We don't know what was the first energy-producing system but it was probably some simple phosphotransfer reaction. It's certain that the membrane-associated electron transport system coupled to redox reactions on inorganic compounds were around long before glycolysis.

      Glycolysis didn't become an energy-producing system until after cells evolved the gluconeogenesis pathway. You had to make glucose before you could degrade it. Even now, there are many species of bacteria that still don't have the standard glycolysis pathway but they can all make glucose using most of the same enzymes that are found in the glycolysis pathway.

      The free energy graph makes no sense. He has the first six reactions of glycolyiss showing a next INCREASE in free energy change. These reactions would never happen if this were true.

      The he has the NADH reaction showing an enormous negative free energy change when, in fact, this is a near-equilibrium reaction where ΔG is zero. The reaction proceeds readily in the opposite direction during gluconeogenesis. I wonder how he explains that!

      Not a very good way to start a lecture. In the first 12 minutes he tries to cover two basic concepts, evolution and theromodynamics and screws up both of them.

  2. You seem to be opposed to MOOCs in principle. But wouldn't all your concerns be answered if there was a mechanism for selecting the best teachers for online classes? Isnt it just a matter of finding one of those excellent small-college profs and having them teach with some feedback from their peers?
    I still think theres a place for MOOC, even though that might mean I'm out of a job


    1. We have excellent textbooks. In what way is a MOOC better than reading a textbook?

    2. Half of the appeal of MOOCs has nothing to do with MOOC but relates simply to the fact that they hold a promise of the competency-based education. In that respect, a good textbook is just as good.

      That said, the answer to your question is "in the same way the real life lecture is better than reading a textbook" - more diverse sensory input.

  3. The author of that NYT article wasn't very impressed with the pedagogy, so I just posted the following comment :

    "I'm preparing my own MOOC right now (Useful Genetics, starts May 1 on Coursera).

    Students typically underestimate how much work and careful thought instructors put into preparing the material they present, in both online and face-to-face courses. Each of my 15-minute videos relies on about 15 hours of preparation, mainly deciding what to teach and how to teach it.

    Students also typically come to science courses expecting that acquiring and regurgitating information will earn them a good grade. But a good instructor will develop assessments that push students beyond this, There's no reason multiple-choice questions can't be intellectually challenging, especially in the 'open-book' context of an online course. And peer-graded assessments are not just a way to shift the work of grading from the instructor to the student - the peer-grading process can also be a powerful pedagogical tool to get students thinking and communicating about what they're learning."

  4. I am currently taking Eric Lander's '7.00x Introduction to Biology - The Secret of Life' over at EdX. For anybody with average to little to no background to biochemistry/molecular biology/genetics, such a course is a godsend.

    He has begun from the very basics (chemical bonds, etc.) and slowly and logically progressed to (currently) transcription and translation and their variations in different organisms.

    Already he has mentioned many great experiments not commonly known to anybody outside the field (from the few last weeks I remember Gobind Khorana and Niremberg, Crick et al's 'On the Triplet Nature of the Genetic Code', Arthur Kornberg's experiments, Meselson and Stahl...) and the way he teaches is great, always asking his students how they would solve such-and-such problem that at some point posed a challenge to scientists.

    Larry should stop being so grumpy.

    Realistically, not everybody can afford to study molecular biology or something similar and have great professors at the same time. Many of us have different jobs and are just interested in the thing.

    At Coursera, Mohammed Noor's course was great. Now, 'Genes and the Human Condition (From Behavior to Biotechnology)' is beginning, and soon, Rosie Redfield's course as well. Can't wait to see them both.

    1. There are two problems that concern me.

      First, I'm not convinced that some of these courses are taught properly. I know that students seem pleased but they would be just as pleased if what they were taught was completely wrong. More importantly, they have no idea what they might be missing if it's never covered in the course. Let me know what he taught on photosynthesis and fundamental concepts in metabolism.

      Second, I'm a fan of student-centered learning and you just can't do that in an online course. In an ideal classroom there should be lots of discussion and debate and students should take an active role in their own education.

    2. Students can never know that what they are being taught is correct, comprehensive and not total BS. *That is, however, true for both MOOC and standard university courses.* Nobody who is not an expert in the field can distinguish that.

      We can only trust the credentials of the professors and universities that offer these courses.

      And no, you can't have much student-teacher interaction during a MOOC [although TAs and sometimes the professor himselft are active on the forums]. That's true. For most people, though, the choice is not MOOC vs classroom course, but MOOC vs no course at all.

      The EdX course for instance is supplemented by online-readable books from NCBI; for the current topics that is the 'Molecular Biology of the Cell. 4th edition. Alberts B, Johnson A, Lewis J, et al.'

      And since you are so sceptical on the subject of MOOC, Larry, what is your alternative for providing biology-oriented education to those thousands who can never enroll in a good university but thirst to know more?


  5. Larry is just trying to protect the ecosystem that feeds him. Ultimately, in vain. MOOCs will win. Yes, good in person teaching is infinitely better than any of the MOOC can ever be. The key word is "good". Almost no in person teaching these days is any good because teachers are not selected per their ability to teach, teachers are not taught to teach and teachers have few incentives to teach well (and many just plainly don't give a damn). Besides, the real purpose of education is credentialing/signaling, not acquiring the knowledge (proof: the widespread cheating). The smart ones succeed, and the dumb ones will fail, all pretty much regardless of the teaching quality - it is only a minor factor in the big scheme of things. In this environment then, giving the capable students a chance to achieve their goals faster and less expensively is a net benefit to the society. So MOOCs will prevail. Slowly but surely, despite all the cries from ivory towers.

    1. I'm currently in Boston hanging out with good teachers. If these people had their way then university education would be far, far better than it is now and MOOCs would be seen as no different than reading a good textbook.

      Your prediction may come true but I'm not yet willing to give up the fight to make university education better than it is.

      BTW, your cynicism about my motives is quite laughable. Most of my colleagues would be quite happy to turn over their teaching to a computer. It gives them more time in the lab. University administrators will figure out a way to get rid of classrooms and give credit for MOOCs AND still charge outrageous amounts of money for tuition in order to get your degree from Harvard. With any luck, they will be able to enroll twice as many students as they do now.

      Only a few people like me will be upset when this happens. Some might wonder why the average citizen is still scientifically illiterate but they'll never realize that destroying undergraduate education was the cause.

    2. Most of my colleagues would be quite happy to turn over their teaching to a computer. It gives them more time in the lab.

      I know. And that is precisely why any MOOC is better than them.

      But once taxpayers figure out that all the enormous money they spend thinking that they are spending on valuable education is actually spent on professors doing what they love doing, the whole system will crumble. Even if some professors don't understand or acknowledge it, it means that their jobs (particularly for non-experimentalists) will go more or less the way the jobs for typewriter repairmen went.

      BTW, do you actually have a plan for making university education much better? What is it that is "their way" that can make education "far, far better"? Let's hear it. I personally think that the only way it can be done is by radically changing the whole system. Just giving much better lectures certainly won't do it.

    3. The above discussion does not apply to smaller liberal arts colleges and universities that focus on teaching and at which research is a "luxury." In my experience, it is generally true that the better the researcher (in terms of grants received, publications, citations, etc.) the the worse they are at teaching. The professors at my university are there because they wanted to teach and so are generally much better instructors than those at larger research-oriented univesities, at least in my experience.

  6. I recently did a MOOC on genetics and evolution. I found the lectures were of a much better standard than the 'live' ones I attended 40 years ago. My only objective was to become more knowledgeable about a subject I was interested in. The course met my requirements brilliantly.

    1. Let's be clear about one thing. I am NOT saying that typical university lectures are better than most MOOCs. Quite the contrary. I'm saying that the reason MOOCs are becoming popular is because they are as bad as the typical university classroom.

      I'm glad that you became more knowledgeable after taking an online course. How do you know that what you learned was accurate?

      Why is taking MOOC better than buying a good textbook on the subject?

  7. Online lectures are more efficient than text books in getting new ideas across. Text books are better as references for when the ideas are no longer new to the student. If I was still teaching I would use online lectures (good ones selected by me) - students would be required to watch them before coming to class, so that all of the contact time could be spent on discussion rather than presentation. I expect this would be much more successful than expecting students to read a chapter of the text book before coming to class.

    This is not the same as moocs. We can get the best of both worlds.

  8. I still think theres a place for MOOC, even though that might mean I'm out of a job

    So MOOCs will prevail. Slowly but surely, despite all the cries from ivory towers.

    There seems to be some thought that MOOCS could supplant the unusual classroom-based university experience. I doubt it. One great benefit of the standard school system is that it provides a structured environment where students are fairly demanded to become exposed to information in a way they are unlikely to on their own. Its not about the information here: schools have never had a monopoly on information and they certainly do not now.

    Though some students might have an enduring "thirst for knowledge" and therefore benefit from MOOCS, most do not. Without the structured course schedule typical of academic systems they will find themselves doing other things when they should be listening and studying and learning. Aside from class structure, there is also something to be said for learning alongside their peers in a shared, collective experience, as opposed to sitting in their bedroom alone in front of a computer.

    The other factor of course is that there is a lot more to the university experience than classes. For many students, it their first opportunity to escape the provincialism of their small homogenous home towns.

    Otherwise I agree that for those who it is not practical or possible to enroll in university, online courses are a nice opportunity.

  9. I'm not sure that there's less student-student interaction in a well-designed xMOOC than in a big face-to-face course. ('Connectivist' cMOOCs are almost pure interaction.) Like face-to-face instructors, MOOC nstructors who want to promote interaction can do so, and those who don't, dont.

    Even in MOOC courses with no required interaction, students who want to interact do so in the discussion forums, which are much more lively than those of any face-to-face course I've taught.

  10. I agree with Prof.Moran. The question is not about MOOCS but the way a subject is approached and developed while teaching. A good textbook does it a lot better.