# Mastery Quizzes for Mastery Learning

This past spring, I made a number of changes to my first-year writing seminar on the history and mathematics of cryptography. Some of those changes, like my use of Perusall for asynchronous active learning, were motivated by the changes in teaching formats required by the COVID-19 pandemic. However, my experiment with mastery quizzes this spring was something I had been planning since before the pandemic. I’ve been a part of a number of committees and working groups over the last several years looking at student persistence in STEM courses, particularly first-generation students and students of color, and some form of mastery assessment came up repeatedly as a way to make STEM courses more equitable learning environments.

What is mastery assessment? For a definition, I’ll crib from the introduction to special issue of *Problems, Resources, and Issues in Mathematics Undergraduate Studies* (aka PRIMUS) on mastery grading, authored by Robert Campbell, David Clark, and Jessica OShaughnessy. Mastery assessment involves three things:

- Explicitly identifying learning objectives for students,
- Assessing for mastery and not partial understanding, and
- Providing students with multiple opportunities to demonstrate mastery.

Let’s consider an assessment technique I’ve been using for years: test corrections. Typically, after each test in one of my math courses, I’ll give students the chance to rework any problems they missed to earn partial points back. I give them a deadline (maybe one week) and I tell them they can’t seek help from any humans other than me. They can earn up to one-third of their points back if they submit accurate corrections, so a student who scored a 70 out of 100 can pull that grade up to an 80. I learned this technique from one of my graduate school professor, Mike Mihalik, and I’ve used it for years to motivate students to learn whatever they missed on one exam to help prepare them for the next exam.

One can consider test corrections a form of mastery assessment. Students have a much better sense of the learning objectives of a test when they return to that test to rework missed problems. Students who only received partial credit on a problem have a chance to rework the full problem correctly. And I’m giving students multiple opportunities (well, two) to demonstrate they understand the material. Test corrections are a lightweight version of mastery assessment in that they don’t fully implement those three principles and thus miss the full benefits of mastery assessment, but they’re also not much more work for the instructor. Grading the tests the first time is quick because you don’t have to provide much feedback, and grading the corrected tests is quick because most of the problems are correct. The extra work comes during office hours, when the students who most need assistance in the course come in for help with their test corrections. I consider that time very well spent.

What are the full benefits of mastery assessment? I see three main benefits, each tied to one of those principles of mastery assessment:

- Students can have a clearer sense of what’s expected of them on assessments, which helps guide their study and reduces test anxiety.
- Instructors can have a greater confidence in knowing students understand concepts and can apply techniques in future courses.
- Assessment can become a low-stakes learning opportunity instead of a high-stakes stressor for students.

These benefits are particularly valuable for underrepresented and first-generation students. We know that these students often experience greater anxiety about tests and grades, which inhibits performance. They can also take a little longer to develop the study skills they need for success in college. And they can often be resistant to the kind of help-seeking behaviors that lead to college success, in part because their imposter syndrome tells them they need to go it alone to prove their worth. (See Seymour & Hunter, 2019, and Leyva et al., 2021, for some of the research on these students.) Mastery assessment responds to all of these factors by decreasing test anxiety, by giving students more time to recover from a slow or rocky start in a course, and by normalizing help-seeking behavior.

I decided to try mastery assessment in my first-year writing seminar after my fall 2019 offering, when feedback on the course evaluations indicated that some students felt I hadn’t prepared them sufficiently for the math exam in the course. I could think of several students whose high school backgrounds hadn’t exposed them to some of the mathematics topics in the course. They had a much harder time in the course compared to peers who had seen some of the topics (discrete mathematics topics like probability, combinatorics, and number theory) before college. I realized I needed to level the playing field somehow, and build in some structures so that all my students could succeed in the course, something that fulfilled those principles of mastery assessment more than test corrections did.

Back in the early 2000s, when I was teaching at Harvard, we had used mastery quizzes as part of the calculus course. More recently, I had mentioned this technique to a Vanderbilt colleague in computer science who was also facing students with varied mathematical backgrounds. She tried mastery quizzes in her course and found it very useful, and so this spring, when I returned to the classroom to teach during the pandemic, I added a set of mastery quizzes to my course. It was a fair amount of work (more on that later), but I consider it a great success, and I wanted to share some details here on the blog for others who might be considering this approach.

This spring, I gave my students three mastery quizzes during the semester, each covering a different set of learning objectives. For each quiz, I provided my students a handout listing those learning objectives and providing a set of practice problems aligned with those objectives. For instance, for the first quiz, here were the learning objectives:

- Modular Arithmetic
- Determine if two integers are congruent module a given integer
*m*. - Generate a set of integers all congruent to a given integer
*x*modulo a given integer*m*. - Simplify or solve a modular arithmetic equation.
- Calculate
*x*MOD*m*, given integers*x*and*m*.

- Determine if two integers are congruent module a given integer
- Prime Numbers
- Determine if a given number is prime.
- Find the prime factorization of a given composite number.
- Determine if two given numbers are relatively prime.
- Find numbers that are relatively prime to a given number.

Writing these learning objectives wasn’t too hard, since I mostly cut-and-paste from the learning objectives I had articulated for the math exam in this course in previous years. I took roughly a third of the learning objectives from the exam and spun them off into this mastery quiz, doing the same for the second and third quizzes. There was some refinement of objectives that happened as I tried to sharpen them for the mastery quizzes, and this was helpful for me to be clearer about my expectations for my students’ learning.

The first offering of each mastery quiz was given during class time, taking about half an hour. Each quiz had eight questions, one for each of the learning objectives, and students needed to answer at least seven of these questions correctly to pass the quiz. If a student didn’t pass that first offering, they could review their quiz, either on their own or with my help, and then take a different version of the quiz later during my office hours. Students could attempt each quiz an unlimited number of times up until a day before the final exam, and if a student passed all three of the quizzes, they were exempt from taking the final exam, receiving a 100% on the final.

This may seem like quite a reward, but (a) the final exam only counted for 15% of the student’s grade in the course since there were also three substantial writing assignments and (b) through their work on the mastery quizzes, the students convinced me that they had indeed mastered the learning objectives I had set out for them. In fact, one of the most rewarding aspects of the mastery quizzes was seeing students who initially struggled with the material review their quizzes, work their practice problems, and come to office hours until the math made sense and they passed their mastery quizzes. This set-up also motivated me to find more creative and more effective ways to explain the math content of the course to my students, since the students who needed four or five tries on the mastery quizzes also needed different ways to make sense of the math.

The mastery quizzes involved a fair amount of work on my end, however. Not only were students wanting to meet with me for offices more often, either to review or retake their quizzes, but I had to write multiple versions of each quiz. I got faster at doing so over time, and now I have a bank of quizzes and quiz questions I can use in future offerings of the course, but it definitely added to my workload this spring, depending on the quiz. For instance, 15 of my 16 students passed the first mastery quiz on the first try, and the remaining student passed on their second try, which meant I only had to create two versions of that quiz. The second quiz, however, was passed by just two students on the first offering, with some students taking five tries to pass. I calibrated the third quiz better, since two students passed on the first try, seven on the second try, and the remaining seven students on the third try.

The mastery quiz approach can be varied in different ways to fit particular teaching contexts. Perhaps the students are given a limited number of opportunities to pass a given quiz, either a limited number of attempts or a limited time window during which to pass the quiz. The reward for passing might be altered, particularly if the final exam covers learning objectives not addressed by the mastery quizzes. Perhaps students receive some bonus to their final grade for passing their mastery quizzes (e.g. 10 or 20 points out of 100), or they automatically score a 100% on a section of the final exam addressing the objectives covered by the quizzes.

The “carrot” for passing all mastery quizzes needs to be sufficiently attractive to students for this approach to work well. Awarding 100% on a final exam or a large portion of a final exam may seem too great of an award, but it can be appropriate if students have already demonstrated their mastery of the associated learning objectives.

Mastery quizzes can greatly reduce the stress and anxiety that students feel about assessment, and can do more to demonstrate mastery than test corrections, given that even test corrections are sometimes wrong. They also support the notion that students need to master learning objectives by the time the course has ended, not necessary when a particular test is given. Mastery quizzes can foster more student-instructor interactions for the students who most need help in a course, and the mastery quiz approach facilitates spaced practice by students (the opposite of cramming) which has been show to enhance long-term retention of learning (Rohrer & Pashler, 2007).

My students reacted positively to the mastery quiz approach. Some students were visibly relieved when I told them there was a way they could exempt themselves from the final exam! A few students seemed discouraged when they failed a quiz for the third or fourth time, but all of them stuck it out and eventually passed every quiz. I was really proud of my students this semester putting in the time and effort needed to master the course material, and, as I noted above, it was rewarding seeing their progress.

**Addendum (8/30/21): **During an interview last week, I was prompted to reflect further on my experiment last spring with mastery quizzes. I realized two things…

One was that the reason my students struggled more with the second mastery quiz (just two students passed on the first try) was that the second quiz addressed topics (combinations, permutations) that fewer of my students had encountered prior to the course and that were harder than the topics addressed by the first mastery quiz (modular arithmetic, prime factorizations).

The second was that I want to reflect more on what I learned about my students’ struggles with these topics, particularly the ones on the second mastery quiz, so I can plan better classroom activities around these topics. After meeting individually with students struggling to pass the mastery quizzes, I had to work to understand what wasn’t clicking for those students and to devise new ways to help them make sense of the topics. I want to channel that back into my lesson plans and problem sets, to see if I can better prepare students for their first encounter with the mastery quizzes.

*Image: *“Los Alamos Pole Vaulter,” William Pacheco, Flickr CC BY-NC-SA

*References*

Campbell, R., Clark, D., & OShaughnessy, J. (2020). Introduction to the special issue on implementing mastery grading in the undergraduate mathematics classroom. *PRIMUS 30*(8-10), 837-848. __https://doi.org/10.1080/10511970.2020.1778824__

Leyva, L., McNeill, T., Marshall, B., & Guzman, O. (2021). “It seems like they purposefully try to make as many kids drop”: An analysis of logics and mechanisms of racial-gendered inequality in introductory mathematics instruction. *The Journal of Higher Education*. __https://doi.org/10.1080/00221546.2021.1879586__

Rohrer, D., & Pashler, H. (2007). Increasing retention without increasing study time. *Current Directions in Psychological Science 16*(4), 183-186. __https://files.eric.ed.gov/fulltext/ED505647.pdf__

Seymour, E., & Hunter, A. (2019). *Talking about leaving revisited: Persistence, relocation, and loss in undergraduate STEM education*. Springer: New York.