A Field Guide to Applied AI Literacy
WCUPA Teaching & Learning Center
Entry 4.1
Assessment in an AI World
Assessment in an AI World
Assessment in an AI World
Making discussion prompts less vulnerable to AI
Why the standard “post and reply to two classmates” format is almost perfectly designed for AI to complete, and five redesign moves that require something AI can’t easily provide.
The concept
Discussions prompts that used to work great before generative AI (post 150 words on this week’s topic and reply to two classmates) don’t really work anymore. AI is more than capable of generating a plausible responses to posts it hasn’t even read.
There are a few modifications to the standard discussion prompt that can make it less vurnerable to generative AI though. By asking for specifics that AI might struggle to provide, you can make it more difficult for students to use generative AI to do their work for them. It’s nearly impossible to make a discussion prompt that is fully protected from AI, but understanding why some prompts fail and where others don’t is an important step in the redesign process.
The vulnerability spectrum — prompt types and why they fail
AI handles easily
Open opinion prompt
“What do you think about this week’s topic? How does it connect to what we’ve covered so far?”
No specific knowledge required. AI produces a balanced, reasonable, well-structured response on any topic. Indistinguishable from competent student work.
AI handles easily
Summary or analysis prompt
“Summarize the main argument of this week’s reading and explain whether you found it convincing.”
AI knows the readings. It summarizes accurately and generates plausible opinions. This prompt also doesn’t require having read this week’s specific version or any annotation the student made.
AI struggles somewhat
Application prompt
“Find a real-world example of this concept from current events and explain the connection.”
AI can generate plausible examples, but they can be generic or slightly off. Note that models tied to paid subscriptions are much better (even really good) at responding to these prompts compared to free models.
AI struggles somewhat
Personal experience prompt
“Connect this concept to something from your own professional or personal experience.”
AI generates plausible-sounding personal experiences. They lack specificity, but students can add minimal details to make them convincing. Better when combined with a follow-up that asks for elaboration.
AI struggles more
Time-dependent + peer-reactive prompt
“After reading at least two classmates’ posts, respond to one specific claim a peer made that you disagree with or would complicate. Quote the sentence directly.”
Requires waiting for peers to post, reading their actual words, and engaging with a specific claim. AI cannot fabricate this without the student at least reading others’ posts first.
AI struggles more
Specific-text anchor prompt
“Choose one sentence from this week’s reading that surprised, confused, or changed your thinking. Quote it exactly, give the page number, and explain your reaction.”
Requires the student to have the reading in front of them. AI can approximate but the page number requirement and the specificity of “this week’s version” creates friction. Pairing with an unusual or newly added reading increases resistance further.
Four redesign moves
1
Anchor to a specific textual moment
Require students to identify and quote a specific passage (or provide a timestamp from a podcast) before they respond. This forces engagement with the actual text rather than knowledge about the topic.
Before
“What is the author’s main argument in this week’s reading, and do you agree?”
AI knows this text. It can summarize accurately without the student having opened it.
After
“Find one sentence in this week’s reading that you think does the most work in the author’s argument. Quote it with page number, then explain why that sentence specifically carries so much weight.”
Requires the reading to be open. The page number and the “that sentence specifically” framing make generic responses obvious.
2
Require verifiable personal specificity
Personal experience prompts are only resistant to AI when they require details that could be verified or followed up on: a specific employer, a named place, a real event with a date. Vague personal connection prompts are easy to fake.
Before
“Connect this week’s concept to your own professional experience.”
AI generates a convincing generic professional experience. Students add their name and submit it.
After
“Describe a specific situation from your own workplace or community — name the organization or context — where this concept would have changed a decision that was actually made. What was the decision, and what would have been different?”
Named organization, real decision, real outcome. Specific enough that a follow-up question (“Tell me more about that decision”) becomes a meaningful verification.
3
Build in timing dependency
If students can complete the full discussion before their peers have posted anything, the peer interaction is performative. Requiring the initial post by midweek and substantive replies after creates a genuine sequence — students who do replies before reading peers are exposed by their inability to engage with specific claims.
Before
“Post your response by Sunday. Reply to two classmates.”
All work can be done in one sitting, in any order, with no real peer engagement required.
After
“Initial post due Wednesday. Replies due Sunday — must respond to a classmate who posted after you did. Your reply must quote something specific your classmate wrote.”
Forces reading peers’ actual posts. The “posted after you” requirement means the content wasn’t available when the initial post was written.
4
Ask for the difficulty, not the answer
Prompts that ask “what did you find hard?” or “where did your thinking change?” are harder to fake than prompts asking for conclusions. AI produces fluent conclusions about any topic. It struggles to convincingly simulate confusion, resistance, or changed thinking — because those states require having actually started somewhere different.
Before
“What did you learn from this week’s material? What was the most important takeaway?”
AI produces a polished summary of the most important takeaways. No confusion, no process, no evidence of a student who was anywhere other than confident.
After
“Describe one moment in this week’s material where your initial reaction was wrong, or where you had to revise what you thought you understood. What did you think first, and what changed?”
Requires a starting position and a revision. AI can simulate this, but specific starting positions are harder to fake convincingly — especially if followed up.
AI prompt — stress-test your existing discussion prompts
Step 1 — see what AI produces on your current prompt
I am a student in an online course on [course topic]. Here is this week’s discussion prompt: “[paste your current prompt]”
Complete this discussion post as a competent student would. Write in a natural student voice. Don’t add caveats — just produce what a student might submit.
[Run this. Read it. Ask yourself: could a student submit this? Would you know?]
Step 2 — generate an AI-resistant redesign
Here is my original discussion prompt: “[paste prompt]”
The goal of this prompt is to assess whether students [what you actually want to measure — e.g., “have read and engaged critically with the week’s material”].
Redesign this prompt using two or three of these strategies: anchoring to a specific textual moment, requiring verifiable personal specificity, building in timing dependency, or asking for difficulty rather than conclusions. Show me the redesigned prompt and explain what each change does.
Caveats
- None of these moves eliminate the possibility of AI use — they raise the cost and lower the quality of AI-generated responses. A student determined to fake the work can still do so. The goal is that doing so takes more effort than doing the actual work.
- Overly complex prompts can produce student anxiety instead of genuine engagement. One or two of these moves per prompt is usually enough. Stacking all five produces a prompt that’s exhausting.
- The timing structure (midweek initial, weekend replies) only works if you communicate it clearly and consistently enforce it. If late initial posts go unaddressed, the sequence collapses and students revert to doing everything the night before it’s due.
- Redesigning discussion prompts is only useful if you’re reading and responding to the discussion. Prompts that require specificity and peer engagement but receive only a completion grade signal to students that the quality of engagement doesn’t matter.
Reflection questions
- Run your current discussion prompt through AI right now. What does it produce? Is that meaningfully different from what you’d accept from a student?
- What are your discussion boards actually assessing — topic knowledge, reading engagement, peer interaction, or something else? Does your prompt design match that goal?
- Have you redesigned your reply prompt in the last few semesters, or only the initial post prompt? What would change if you gave the reply equal design attention?
- Which of the five moves would require the least redesign effort to implement in your current course — and which would have the most impact?
