Practice, Feedback, and Generative AI
Cross-posted from the Intentional Teaching Patreon, as part of the 2024 slow read of Intentional Tech.
Back in the summer of 2016, Emily and I were on a date in lovely Centennial Park here in Nashville, Tennessee. The summer evening was pleasant, the park’s replica Parthenon was majestic, and the paths circling the ponds and green spaces of the park were filled with… people playing Pokémon Go. Remember Pokémon Go? That game where you caught virtual pocket monsters on your phone while traveling about the actual world. Centennial Park must have been a Pokémon hotspot that summer, because I’m pretty sure Emily and I were the only people at the park not trying to catch them all.
With this new augmented reality app taking the country by storm, I braced myself for the inevitable response from higher education and the technophiles therein about how Pokémon Go will change the future of teaching and learning. How can we use Pokémon Go to connect with and engage our students? What can we learn from Pokémon Go about using augmented reality in and out of the classroom? Why are Pokémon Go players so addicted to the game and how can we sprinkle a little of that magic on our educational apps? None of those are actually terrible questions, but I was pretty skeptical about Pokémon Go’s potential for disrupting higher education. (So was John Warner, for the record.)
There’s always some new technology hitting the scene that attracts a certain kind of instructor the way a shiny object is said to attract crows. (That’s probably just crow folklore, by the way, says corvid researcher Kaeli Swift.) Some shiny new technology makes itself known, and this instructor can’t wait to see what they can do with it in their teaching. You probably have a colleague like this. You might be that colleague, in fact. One of my goals in writing Intentional Tech was to provide instructors like this with something of a roadmap to why they might use a new technology.
Can the technology make visible “thin slices” of student learning? Can it help students map their internal knowledge organizations? Can it connect students with authentic audiences for their work? These are things that technology can do (and sometimes do very well), so it’s useful to keep them in mind when evaluating new tech for educational use. As I say in the book, our pedagogical goals need to guide our adoption and use of technologies. My working title for the book was even More Than Just Shiny Objects: Using Technology to Support Student Learning.
Back in 2016 the shiny object might have been Pokémon Go. In 2023 the shiny object was most definitely generative AI. To be fair, at the start of 2023, most faculty didn’t see ChatGPT and similar tools as exciting new technologies to be pressed into service of student learning. They were far more worried about the potential of these tools to render our existing assessments of student learning null and void, given how readily ChatGPT could produce a B- response to pretty much any essay question. But even twelve months ago, there were plenty of instructors excited to see what they could do with generative AI in their teaching.
I was one of them, of course! I don’t welcome all shiny new technologies with open arms. (No thank you, Second Life. You’re great, Fitbit, but not for general educational use. Um, good luck with that Metaverse, Mark Zuckerberg.) But I am, on average, more positive than negative about teaching with technology. And it was clear to me early on that ChatGPT had some educative potential. So I did what I always do with new technologies I’d like to investigate for teaching purpose, I went through my Intentional Tech principles. Could I use ChatGPT to create times for telling? Make student learning visible? Help students learn from and with each other?
Most of my investigations didn’t go anywhere, although thinking of ChatGPT as an authentic, if entirely artificial audience seemed to have some merit. It wasn’t until I spoke at Yale University’s spring teaching forum in May 2023 that it became clear that the teaching principle that would best inform generative AI use was that of practice and feedback. As I wrote about in my newsletter, I heard Yale divinity faculty member Tisa Wenger talk about her spring semester ChatGPT experiment. She had co-taught a course on the history of Christianity in the Americas with Erika Helgen, and they asked their students to give ChatGPT one of several possible prompts, all of which asked the chatbot to write a sermon or speech making a particular argument from a particular historical perspective, then use what they were learning about the history of Christianity to critically evaluate the chatbot’s output.
This, I felt, was brilliant. It made use of ChatGPT’s strengths, which include being able to write in particular genres and make reasonable logical arguments, as well as its weaknesses, which its inability to deal with particulars and its tendency to just make stuff up. And every student could have their own unique sermon or speech to evaluate! You get a sensible but historically inaccurate sermon! And you get a sensible but historically inaccurate sermon! (Apologies to Oprah.) Bonus: Everyone gets a little AI literacy, that is, learning about how these tools work, what they’re good at, and how they’re problematic.
This application of generative AI to teaching wasn’t unique to the good folks at the Yale Divinity School, of course. Lots of instructors started figuring out that having students question generative AI and then critique the results was an appropriate response to the concerns about students using AI to do their homework. For instance, in July 2023 Robert (Grading for Growth) Talbert shared plans for his fall discrete structures course in which students would ask Google Bard to solve math problems. Google Bard is, as I understand it, one of the weaker large language models available now, so it compensates by providing not one but three potential responses to a question. It’s hedging its bets against hallucinations, I guess. But three possible answers to a math problem? Robert quite sensibly planned to have his students respond to the prompt “Which one, if any, is right, and why?”
There’s a similar potential for generative AI in fields that rely on case studies, like nursing. I found a discussion paper in the journal Nurse Education in Practice by Lucija Gosak and colleagues that evaluated ChatGPT’s ability to diagnose a fictional patient given a detailed description of that patient. I replicated their ChatGPT interaction, and while I can’t verify any of the chatbot’s diagnoses, I can imagine asking nursing students to do so. And then there are faculty like Dayanjan Wijesinghe, a pharmacy faculty member at Virginia Commonwealth University, who are well ahead of the curve. He and his colleagues built a tool that “utilizes OpenAI’s ChatGPT-4 API to construct a conversational AI, acting as a simulated patient, to provide a dynamic, interactive platform for future pharmacists to practice and refine their communication abilities.” Instead of using the general purpose ChatGPT for this kind of practice, Wijesinghe’s students can go to a custom-built chatbot designed intentionally for this pedagogical application.
Using generative AI for this kind of practice and feedback isn’t the only pedagogical use of this technology, of course. I’m planning to explore the use of Adobe Firefly and other AI image generators a bit later in this year’s Intentional Tech slow read in support of multimodal assignments. These teaching principles can help us approach new technologies like generative AI with a bit more grounding and perhaps avoid some of the hype (positive or negative) that can accompany flashy new technologies. That said, don’t ask me to do an Intentional Tech analysis of Pokémon Go. I can only stretch this framework so far.