Quizzes with No Wrong Answers: #FutureEd Week Two

I’m behind in the “History and Future of (Mostly) Higher Education” MOOC led by Duke University’s Cathy Davidson, but after this evening’s efforts, I’m only one week behind. My Week 1 reflections focused mostly on the structure and processes of the MOOC, not its content. Week 2 for me was more about the content. In this post and the two following posts, I share a few thoughts from Week 2, mostly about assessment of student learning.

As I mentioned in my last blog post the quizzes in the #FutureEd MOOC have no wrong answers. Most questions are of the “mark all that apply” variety, and all the answers are correct, by Cathy Davidson’s design. She says that, according to research, students remember the wrong answers they choose on multiple-choice questions, so she had designed her weekly quizzes to feature no wrong answers.

This quiz format struck me as… unusual. If students know that all the all the answers on a quiz are correct, they have little incentive to think deeply about the quiz questions. Indeed, I scored a 100% on the Week 2 and Week 3 quizzes without watching a single lecture video (upon which the quizzes are based) or even reading quiz questions.

Other MOOC participants have argued that the quizzes function as a helpful review of the week’s material, assuming one reads the questions with some care. I see some value there, certainly, but the quizzes still bugged me, so I did a little digging into the literature this week. As it turns out, working through a quiz with wrong answers and receiving feedback on that work is far more useful to learning than simple review of the material.

See Annie Murphy Paul’s article in Time last year, which provides a very readable summary of a much longer paper exploring the research on learning activities. Rereading is one of the least helpful study activities, according to the research, and that’s essentially what these quizzes provide: an opportunity to “reread” the video content. Far more helpful is self-testing. Here’s a quote from a frequently cited literature review on the testing effect:

“The modal outcome is that practice testing outperforms restudying, although this effect depends somewhat on the extent to which practice tests are accompanied by feedback involving presentation of the correct answer. Although many studies have shown that testing alone outperforms restudy, some studies have failed to find this advantage (in most of these cases, accuracy on the practice test has been relatively low). In contrast, the advantage of practice testing with feedback over restudy is extremely robust. Practice testing with feedback also consistently outperforms practice testing alone.”

Note the mention of “presentation of the correct answer.” That implies that incorrect answers are possible. To get the benefit of the testing effect, students must have the possibility of answering questions incorrectly.  The quizzes in the #FutureEd MOOC don’t have this property, and so they aren’t that useful as learning activities.

If Cathy Davidson wasn’t intentionally using the structures of the MOOC to get us thinking about the future of higher education, I wouldn’t worry about all this. But she is, and so I do.

Image: “mini mnms,” inajeep, Flickr (CC)