Generative AI in EducationActivity Examples: Students Critically Evaluating AI

Building AI literacy will be important for our students as they engage with generative AI tools. Two of the articles below document how educators engaged in inquiry learning with their students to develop fluency with and critical evaluation of generative AI tools. We've also included additional resources that represent different viewpoints of generative AI.

For each reading below, we've included the estimated reading time (), the resource citation (), a main idea quote () pulled from the resource, and key questions to consider () as you think about why and how to implement these technologies.

How to cheat on your final paper: Assigning AI for student writing

Reading Time: 34 minutes

Fyfe, P. (2022). How to cheat on your final paper: Assigning AI for student writing. AI & Soc. https://doi.org/10.1007/s00146-022-01397-z

Main Idea Quote:

For the course’s final module on AI, students had to write an essay that integrated the output from a text-generating language model into their own writing—but without revealing which was which. Styled as the “Professor Fyfe Turing Test,” students were encouraged to try and make indistinguishable what essay text came from the AI or from themselves. I imposed no quotas for the amount of AI-generated text nor strictures about the ways it appeared. In the essay’s final section, students composed exclusively in their own voices (i.e. without AI), reflecting on the following questions given in the assignment prompt:

How easy or not was it to write this way? What worked or what didn’t? How did the AI-generated content relate to your own? How did it affect what you might have thought about or written? Do you feel like you “cheated”? To what degree is this paper “your” writing? Do you expect a reader would notice GPT-2’s text versus your own? Would you use this tool again, and in what circumstances? And, ultimately, what ideas about writing, AI, or humanness did the experiment test or change?

Finally, all papers included an Appendix in which students provided a “revealed” version of their essay with the AI-generated text highlighted...

Collectively, my class did not conclude one way or another that writing with AI was tantamount to plagiarism. They split down the middle on the question, with all sorts of qualifications about their votes. But this assignment was never meant to settle a debate. Instead, it immersed students within the debate to broaden and deepen their perspectives, and even to raise other questions we had not yet collectively discussed. As one student perceptively judged: “Overall, the experiment was able to test multiple ideas … It was interesting to experience these ideas after having read about them.” The “hands-on” experience let students explore the arguments for themselves, as well as test out new configurations of writing beyond the potentially limiting concept of humans vs. AI. Furthermore, the assignment proved useful not only in reckoning with text-generating AI, but in having students reflect on the values of their own expression. The assignment’s framework of AI-powered plagiarism led students to assume that they would somehow avoid the intellectual work of writing. But the opposite happened, reinforcing or expanding students’ definition of what that work encompassed. As the author Robin Sloan reports of his own experiments writing with machines, “the goal is not to make writing ‘easier’; it’s to make it harder” (2016). In other words, such experiments refuse to let us take our ideas about writing for granted, pushing us to rearticulate or re-envision them. And, happily, even though this required more work and some occasional aggravation, students were engaged in ways surpassing my expectations.

Key Questions to Consider:

Do critical evaluation and use of a tool go hand-in-hand? Can students evaluate generative AI tools without implementing them?
What other aspects besides cheating and plagiarism would be important for your students to explore?
This experiment used GPT-2, the precursor to ChatGPT. How might students' experience and attitudes be different with a more powerful AI tool?

How I approach AI-literacy in the writing classroom

Reading Time: 7 minutes

Dumin, L. (2022). How I approached AI-literacy in the writing classroom. Medium. https://medium.com/@ldumin157/how-i-approached-ai-literacy-in-the-writing-classroom-d04e7031c87f Links to an external site

Main Idea Quote:

For starters, what do I mean by “AI literacy”? For me, this means teaching students both what AI can AND can’t do well. I see both sides of this as important because if we only focus on one half of the discussion, students may miss some very important points from the other side.

...But what did happen was purposeful discussions in class almost daily about the newest news on AI. We opened up ChatGPT and Elicit.org and asked it the same question about which was the faster animal while hunting. The question was meant to be silly to see what the programs could do with it.

...By showing actual examples and then discussing the output, I hoped to train my students on what to look for in good output. This involved going back to the critical thinking skills that so many of us are working to teach our students. Using AI to find information still requires as much critical thinking as using Google, but the skills need to be tweaked so that students understand that they need to question the output rather than accept it at face value.

Key Questions to Consider

What uses of generative AI could you critically evaluate with your students?
How can you encourage individual and whole-class reflection with your students?

Five ways to use Social Annotation with and against ChatGPT

Reading Time: 5 minutes

Dean, J. (n.d.). Five ways to use Social Annotation with and against ChatGPT. Hypothesis. https://web.hypothes.is/blog/five-ways-to-use-social-annotation-with-and-against-chatgpt/#annotations:_Ahs7rFUEe27yUN0kHRngg Links to an external site

Main Idea Quote:

5. Annotating the writing of AI chatbots

Annotation can be used to fact-check, comment on, and critique the products of AI writing tools. The ShareGPT browser extension lets users make a stable URL of a ChatGPT session that can be annotated with Hypothesis. Let’s teach students to be critical readers (and perhaps editors) of ChatGPT outputs. What mistakes (or “hallucinations”) does the bot make? What does it get right? Does it show any biases? How is its writing formulaic? How can students improve upon what they are given by AI? If AI lacks an understanding of what it’s writing, can students add the meaning?

Key Questions to Consider

What skills do students need in order to critically evaluate the writing of AI chatbots? How can you help them develop these skills?
How can you use social annotation to encourage critical evaluation of AI outputs?

Resources to Prompt Conversation

In addition to the assignment examples above, we are providing resources that approach generative AI from a critical perspective. Consider how these might align to your course outcomes and how you might use these with your students.