Students, Please Get Out Your Phones

Inside Higher Ed published a great piece last week by Nick Roll looking at changes in classroom response systems. At many campuses, such systems are shifting from using dedicated “clicker” devices to leveraging students’ own mobile devices, including phones, tablets, and laptops. My institution is rolling out a bring-your-own-device (“BYOD”) system (Top Hat) across campus this fall and I played a pretty major role in making that happen, so I thought I would share a little behind-the-scenes perspectives here on the blog…

Classroom Response Systems: History and Benefits

Classroom response systems are instructional technologies that allow instructors to rapidly collect and analyze student responses to questions posed during class. Wired versions of these systems have been used since the 1960s, but it was the advent of wireless systems using radio frequency communications that led to rapid adoption of classroom response systems in college and university classrooms in the early 2000s. Vanderbilt faculty members have used classroom response systems since 2002, thanks to a College of Arts & Science Venture Fund grant awarded to a group of instructors from multiple departments. In 2007, most colleges and schools at Vanderbilt standardized on Turning Technologies as their classroom response system vendor, and that was the status quo for several years.

When used to support active learning pedagogies like peer instruction, classroom response systems provide a number of benefits, including increased student participation and engagement during class, useful feedback on student learning to both students and instructors, and, in some cases, greater learning outcomes and student success. These benefits are supported by empirical research[1] and by student surveys. According to a March 2016 Dean of Students “digital learning” survey of Vanderbilt undergraduates, when students consider classroom response systems, they appreciate the ability to see other students’ answers (and, thus, compare their own answers to their peers’), the ways in-class polling engages them in learning during class, the immediate feedback on learning that polling activities provide, and the opportunity to test their knowledge and skills during class through polling activities.[2]

The Move to BYOD

Usage of classroom response systems at Vanderbilt is widespread, with adoption by faculty teaching in undergraduate, graduate, and professional student programs. In fact, classroom response systems are used in all of Vanderbilt’s colleges and schools, with the possible exception of the Blair School of Music. In the March 2016 Dean of Students survey, 77% of undergraduate respondents reported that they used a classroom response system in at least one course in the last year, and19% reporting use in at least three courses in the last year. Most students (80%) reported using the system provided by Turning Technology, but some students reported using systems by other vendors, including Poll Everywhere (21%) and Top Hat (21%).

The reports of Poll Everywhere and Top Hat usage are representative of a technological shift occurring in classroom response systems. In 2007, when Vanderbilt standardized on Turning Technology clickers, relatively few students owned smart phones or other mobile devices they could easily use in the classroom, so using dedicated clicker devices was the most practical approach to using classroom response systems. As smart phone ownership among students (at Vanderbilt and elsewhere) has increased in the last several years, a number of “bring your own device” (BYOD) options have become available for classroom response systems. BYOD systems leverage students’ personal mobile devices (phones, tablets, laptops), allowing students to respond to polling questions via Web browsers, text messaging, or dedicated mobile apps. Turning Technologies has a BYOD option called ResponseWare, and new vendors like Poll Everywhere and Top Hat have emerged that are BYOD-native.

BYOD classroom response systems offer a number of advantages over clicker-based systems:

BYOD systems leverage students’ existing mobile devices, eliminating the cost and logistics associated with student-purchased clickers, which can run close to $50 each.
Since they accept responses via the cloud, BYOD systems don’t require installation and maintenance of radio receivers across multiple classrooms on campus.
Students are known to forget their clickers from time to time, but students almost always have their phones with them (and charged).
Given the input capabilities of mobile devices like phones and tablets, BYOD systems offer better support for free-response polling questions than clickers. This opens up a range of pedagogical applications for classroom response systems.

One common concern expressed about BYOD systems is the potential for distraction that mobile devices pose in the classroom. However, many instructors who use BYOD systems report that giving students something engaging and on-topic to do with their mobile devices during class leads to less use of phones and tablets for off-topic pursuits such as Facebook or ESPN. Other concerns about BYOD systems are logistical in nature—What about students who don’t own a phone or laptop or tablet? Will dozens or hundreds of students going online concurrently overwhelm classroom Wifi access points? See below for evidence that these were not likely to be significant concerns at Vanderbilt.

BYOD at Vanderbilt

Faculty interest in BYOD systems was high, with some faculty already making the switch from clicker-based systems to BYOD systems and other faculty adopting BYOD systems who had never used clickers. With at least three BYOD systems in use across campus, there were concerns about students having to sign up for accounts in multiple systems, which led to discussions about standardizing on a single system across campus, as we had done with clickers back in 2007.

Standardizing would offer several advantages. Instead of instructors paying for access to systems out of their research funds or asking students to pay for individual system licenses, Vanderbilt could negotiate a campus-wide license that resulted in lower per-student costs. Funding a BYOD classroom response system centrally would be consistent with how Vanderbilt provisions other critical technologies, such as our course management system. Also, instructional technologists and other support staff would only need to learn one system (and liaise with one vendor), making technical and pedagogical support for instructors teaching with classroom response systems more efficient and effective.

Furthermore, a centrally funded and supported BYOD system would make it easier for instructors around campus to experiment with the kinds of active learning pedagogies supported by these systems. Some instructors are hesitant to ask students to make a financial commitment to a system (by purchasing a clicker or an individual license) they don’t plan to use regularly through the semester. If Vanderbilt had already paid for access to a BYOD system, however, instructors could easily try out in-class polling for a class session or two, or use it selectively on only the days when it made the most sense. Lowering the barrier to entry in this way would mean more Vanderbilt faculty using educational technology in intentional ways to enhance face-to-face learning experiences for their students.

Students are supportive of a move to BYOD. According to the March 2016 Dean of Students survey, 71% of undergraduate respondents indicated a preference for bringing their own device when using a classroom response system. Twelve percent indicated no preference, and only 16% preferred to use dedicated devices (clickers).

That same survey also provided evidence that device ownership is not likely to be a problem at Vanderbilt. Of the students who responded to a question about device ownership, 98% indicated they owned smart phones and 99% indicated they owned laptops. (By comparison, only 31% indicated they owned tablets.) In most professional schools, these numbers are 100%, given laptop requirements in places like Nursing and Law. For the small percentage of students without personal mobile devices, the Jean and Alexander Heard Library has a laptop and iPad loaner program.

Several years ago, the School of Nursing moved to a BYOD system, ResponseWare, but quickly returned to using clickers when it was found that the Wifi infrastructure in classrooms wasn’t sufficient to the task. Vanderbilt Information Technology (VUIT) has significantly expanded Wifi coverage in the years since, and VUIT representatives felt confident that our classroom Wifi access points can handle BYOD systems at this point. Instructors already teaching with BYOD systems have reported no problems with Wifi connectivity.

In short, Vanderbilt was well positioned to move to a campus-wide BYOD classroom response system.

Vendor Selection

In August 2015, I received approval from the Provost’s Office to assemble a working group to consider options for a campus-wide BYOD classroom response system license. The Center for Teaching (CFT) reached out to faculty and staff known to be actively using or supporting classroom response systems at Vanderbilt, inviting them to participate in the group. To cast a wider net, the CFT mentioned the working group repeatedly in its start-of-semester broadcast publicity (email newsletter, blog, etc.) and in the CFT section of the “EdTech Roadshow” presentation shared at most college and school faculty meetings in the fall. Eventually, 44 faculty, staff, and students joined the working group, representing seven of Vanderbilt’s schools and colleges.

The group met once in October 2015 to assemble a list of BYOD response system features that would be useful to Vanderbilt instructors (see below), then invited several leading BYOD response system vendors to campus to provide demonstrations and answer questions. Based on these sessions, as well as additional research on the vendors conducted by CFT staff, members of the working group provided feedback on the response system options. Top Hat was received very well by the working group, but it was the system least used at Vanderbilt at the time. The CFT decided to conduct a Top Hat pilot in the spring semester, recruiting four faculty instructors from the working group to use Top Hat’s system in their spring courses.

In late February 2016, approximately six weeks into the Top Hat pilot, the working group reconvened to share experiences using the various systems in their classrooms. This discussion, along with strengths and weaknesses of the systems identified during and after the February meeting, informed a proposal submitted to Academic Affairs. The proposal was received positively, but a decision was deferred until Vanderbilt’s next course management system was selected. That decision (Brightspace by D2L) was finalized in December 2016, which cleared the way for this fall’s adoption of Top Hat as Vanderbilt’s BYOD classroom response system.

I’m not going to share publicly the feature-by-feature vendor comparisons that were included in the proposal, but I will share the list of classroom response system features the working group identified as valuable back in the fall of 2015, in case this list is useful for other campuses working through similar decision processes. Some features were more important than others, of course, but here’s the complete list:

Feedback from the system to students letting them know their votes were recorded
Option for students to respond via a dedicated mobile app
Ability to create on-the-fly questions
Integration with the course management system’s grade center
Ability to hide responses until later in a class session
Linking responses with student identities
Descriptive statistics, histogram for numeric-response questions
Ability to embed polling questions in PowerPoint
Free-response questions with upvoting
Mechanisms to make sure only those physically present can vote
Word cloud visualization for free-response questions
Ability to track attendance across the semester, not just single classes
Options for importing questions from a different system into the new one
Text wall / list of responses for free-response questions
Ability to respond via SMS/text messaging for multiple-choice and free-response question types
Side-by-side comparison of responses from two questions (or the same question asked twice)
Allowing students to respond anonymously
Engagement features – competition, etc.
Ability to handle 200+ responses
Capacity to handle mathematical notation
Ability for students to revisit questions after class
Placemark questions in which students click points on a provided image
Ability to create and export reports
Segmentation – slicing responses from Question B according to Question A
Single sign-on – integration with VUnet authentication
Tools for sharing questions and/or content among instructors
Instructor app allowing control of polling from a mobile device
Floating toolbar, allowing polling on top of any presentation software

Adoption and Implementation

After the Top Hat adoption was approved by the Provost’s Office, the CFT moved forward with Top Hat and campus partners to get the system ready for use this fall. That involved integration with Vanderbilt’s single sign-on authentication system and with our new course management system, Brightspace. The CFT also worked with Top Hat to roll out training and support offerings, including online training sessions, in-person training sessions, and a set of on-demand resources. As I write this, we have a few dozen instructors (including me!) gearing up to use Top Hat this fall, with CFT and Top Hat staff standing by to provide support.

I’m really excited by this adoption. As noted above, a BYOD system opens the door to various free-response polling questions that have real pedagogical value. And having a centrally funded and supported system means it will be a lot easier for instructors to try out a BYOD classroom response system and see how it might enhance their teaching. I’ve got my fingers crossed that we won’t have significant technical problems this fall, but mostly I’m proud to have helped make this happen at Vanderbilt. My center now supports Brightspace and Top Hat, which provide instructors core tools for support learning both in and out of class. I’m looking forward to seeing the creative ways our faculty use these tools this fall.

Footnotes

[1] See “Examining the Benefits and Challenges of Using Audience Response Systems: A Review of the Literature” by Robin Kay and Ann LeSage (2009) in Computers & Education, 53, 819-827.

[2] Concerns students had about classroom response systems included the use of polling primarily or exclusively for attendance or participation grades, as well as the loss of class time and frustrations resulting from occasional technical difficulties.