Article: Lasry (2008)

Reference: Lasry, N. (2008). Clickers or flashcards: Is there really a difference? The Physics Teacher 46(4), 242-244.

Summary: Lasry reports the results of a study contrasting the use of clickers and flashcards in facilitating peer instruction in an introductory physics course. Two sections of the course were taught in the same semester by Lasry. In one section, students responded to multiple-choice, conceptual understanding questions using clickers; in the other they responded using flashcards. In both sections, student responses to the questions were used to determine what followed the question–further explanations of the topic at hand by the instructor if most students missed the question, moving on to the next topic if most students answered correctly, or peer instruction otherwise.

Lasry administered the Force Concept Inventory to both sets of students at the start and end of the semester as an assessment of the students’ conceptual understanding. The normalized gain, (post%-pre%/100-pre%), for the clickers section was 0.486, and for the flashcard section it was 0.520, not a statistically significant difference in this case.

Lasry’s conclusion is that “using peer instruction with clickers does not provide any significant learning advantage over low-tech flashcards.” He notes that clickers might provide other advantages, such as enabling instructors to analyze student response data for the purpose of improving in-class questions over time and interesting other instructors in experimenting with peer instruction.

Comments: Lasry’s data are certainly interesting and provide some evidence that peer instruction works as well with flashcards as with clickers. However, he describes the “contributions of clickers” as being “more on the teaching side than on the learning side of the educational equation.” I find this separation of teaching and learning a little artificial. The effects on student learning that any instructional technology has depend on how the technology is implemented. There are a couple of ways of implementing clickers that have the potential to positively impact student learning that don’t appear to be addressed in this study. These factors might explain the lack of difference in learning gains between the two sections.

For example, since clickers allow an instructor to track individual student responses, they can be used to hold students more accountable for their responses than they would be using flashcards, which has the potential to increase student motivation to participate and engage with questions asked during class. It’s unlikely that student responses in the clicker section in this study were factored into student grades since tracking individual student responses in the flashcard section would have been impractical and Lasry apparently tried to keep as many aspects of each section constant as he could. If that’s the case, then students in each section would have been similarly motivated to participate, which might explain the lack of difference in learning gains. Had student responses to clicker questions been included in student grades in the clicker section, students might have performed better on end-of-semester assessments.

One of the points that Tim Stelzer made in his clicker conference keynote last November was that student participation tended to decrease over time when flashcards were used at the University of Illinois. I would be interested in finding out if there was any difference in participation in the two class sections in Lasry’s study. If there was not, then there might have been other factors, such as instructor experience or instructor-student rapport, that kept participation high in the flashcard section and offering another explanation why the clicker section didn’t exhibit greater learning gains.

Another implementation choice that has a potential effect on student learning is “agile teaching,” that is, using response data from clicker or flashcard questions during class to make teaching decisions. In Lasry’s study, response data were used to determine when to engage students in peer instruction. Such choices are likely most effective when based on accurate assessments of student learning. As Stowell and Nelson (2007) showed, the flashcard method can lead to instructors overestimating their students’ comprehension since the method makes it possible for students to see other students’ responses as they select their own responses. Clickers tend to provide more accurate feedback on student learning since they promote independent answering by students. It’s possible that in the Lasry study, the flashcard method provided accurate enough assessments for the teaching choices that were made. Other kinds of agile teaching choices might have benefited from the more accurate data provided by clickers. The impact of clickers on agile teaching is an issue that hasn’t been studied well to date to my knowledge.

Finally, another way in which clickers might provide benefits over flashcard methods is that clickers make it easy for students to see the distribution of responses to a question. Flashcards provide this distribution (in rough form) only to the instructor. Seeing the distribution of responses has a potentially motivating effect on students, particularly when students find out that most of their peers answered a question incorrectly. It’s unclear from the article the extent to which clicker or flashcard questions were used to generate “times for telling” in this fashion. It’s possible that in classes where these kinds of questions are asked more regularly, clickers have a bigger impact on student learning because of the easy display of results to the class.

4 Responses to "Article: Lasry (2008)"

Jeff Stowell says:

January 26, 2009 at 3:30 pm

Without a description of the flashcards, it’s hard to know how much students could see each others’ responses. I also agree that there is something beneficial about being able to see where a student’s response is compared to others. It may affect a students motivation, or at least their self-efficacy.

Reply
Nathaniel Lasry says:

January 27, 2009 at 5:19 pm

I completely agree that “The effects on student learning that any instructional technology has depend on how the technology is implemented”.

Let’s take 3 cases:
1- Technology is adopted with poor pedagogy
2- Technology is adopted with good pedagogy
3- Technology is not adopted but emulated with low tech and good pedagogy.

From the above quote, we will agree that technology without proper implementation is no good. Indirectly, this is also an admission that some may use technology with either no positive effects or even negative effects (eg. I’ve seen clickers used to efficiently drill students on rote memorization tasks, nomenclature recall etc.).

My data show that a proper implementation of the technology yields good gains, in fact almost identical to those Eric Mazur had the 1st time he implemented the approach at Harvard (a side by side comparison between Mazur’s 1st implementation at Harvard and my 1st implementation in a 2yr college is published last Nov in the American Journal of Physics, 76(11) p. 1066-1069 ).

In the Clicker-Flashcard paper, the data also show that the clicker is not a ‘magical’ learning device. It is not about the clicker but how you use it. Clickers are VERY useful in giving a precise tally, in gathering anonymous votes and can even collect confidence data. Most importantly, I can archive all this data and analyze it from semester to semester. That’s really important to me as a teacher, but it doesn’t change how my students will be engaged or how well they will learn (hence the teaching vs learning difference).

I did not give credit for answers, mostly because it is counter-productive. Indeed, a study done by James (Am J Phys Volume 74, Issue 8, pp. 689-691) showed this quite well. Here is an excerpt of the abstract:
“In the high stakes classroom where students received little credit for incorrect CRS responses, it was found that conversation partners with greater knowledge tended to dominate peer discussions and partners with less knowledge were more passive. In the low stakes classroom where students received full credit for incorrect responses, it was found that students engaged in a more even examination of ideas from both partners. Conversation partners in the low stakes classroom were also far more likely to register dissimilar responses, suggesting that question response statistics in low stakes classrooms more accurately reflect current student understanding and therefore act as a better diagnostic tool for instructors.”

It is possible to give participation credit only (independent of whether the question is answered correctly or not). I chose not to do this because it would have not been possible with flashcards.

About the anonymity of clicker responses in comparison to flashcards. Students raise flashcards all at once. Very few will turn to their neighbor to find out what they are voting. There definitely is NO anonymity with respect to the teacher. That is, I know if student X voted A,B,C,D or E in real-time. In fact, I think this provides a slight advantage in favor of flashcards. I have seen many students click a little too fast. Students are much more cautious when they have to choose answer they will be showing the prof… Not enough of an advantage to get me to stop using clickers.

About seeing the distribution of answers: Having consulted with Mazur and tested a few schemes, it is counter-productive to show students the answers after the 1st vote. They’ve all seen ‘who wants to be a millionaire’. The audience is almost always right… But the distribution is shown to students when they vote after the discussion. With flashcards I did this too, although distributions were more of a ball park estimate at times.

As much as I am madly in love with my clickers and will continue to use them as long as I teach, I still think its important for us to come to terms with the fact that it is not about the gizmo. Bottom line: pressing a button provides no major advantage on raising a flashcard when it comes to engaging students. When properly used, BOTH work equally well.

Since proper use is what we’re after, the take home message is focus on the pedagogy not the technology.

Reply
Ian Beatty says:

January 29, 2009 at 1:07 pm

Hi, Derek.

It occurs to me that the nature of the questions being asked might be crucial for whether flashcards are equivalent to clickers or not. Consider, for example, emotionally charged “values” questions… Perhaps the anonymity of clickers would be significant there.

(Some CRSs also permit all-that-apply question types, of course.)

Cheers,
:ian

Reply
derekbruff says:

January 30, 2009 at 5:11 pm

Thanks for these comments. I’ll agree with Nathaniel that technology without proper implementation is not helpful. I, too, have seen instructors use clickers in unproductive ways. Thus, I also agree that the take-away for instructors interested in teaching with clickers is to focus more on the pedagogy than on the technology.

I would argue, however, that when investigating the different effects clickers and flash cards can have on student learning, it’s worth paying attention to implementation choices that highlight functional differences in the two technologies. So, for instance, since clickers allow instructors to track individual student responses over time (and thus include clicker questions in student grades) and flash cards do not, it’s worth investigating that effect this capability might have on student motivation (and thus, student learning). In this study, this feature of clickers wasn’t implemented, so the study doesn’t shed any light on how this feature might affect student motivation.

I should clarify here that when I speak of including clicker questions in student grades, I’m referring to either grading clicker questions on accuracy (whether or not students answered correctly) or just grading them on effort (giving full credit for both right and wrong answers) or some combination of the two (giving full credit for correct answers and partial credit for incorrect answers). The choice of how to factor clicker questions into student grades is an important one, and I’m eager to read the study by James exploring this choice. (Thanks for pointing it out to me, Nathaniel.)

Nathaniel also makes a good point that using flash cards makes it easier for instructors to know how each student answers during class. Some classroom response systems make this easy to do, too, but this isn’t a feature I see many instructors using. It’s quite possible that this feature of flash cards creates the kind of accountability for student responses that including clicker questions in student grades creates, potentially explaining the “no difference” result in this study. The class size (n=40 or so) may be a factor here, as well.

The affective (motivational) issues at play when using clickers or flash cards are interesting and important ones. I look forward to future studies exploring these in more depth.

Finally, I’ll second Nathaniel’s point that showing students the initial vote to a clicker question can be counter-productive. If students see that most of their peers agree on a question, they’re probably less likely to engage in meaningful peer instruction. On the other hand, if the initial vote is split among two or three alternatives, seeing this distribution might encourage more discussion. This is another topic that would be worth exploring in future studies.

Reply

Leave a Reply Cancel reply