Using AI as a Pedagogical Tool
Four ways AI fails,
and how to teach with them
Our students recognize that AI can fail, but rarely do we get to show them how AI fails. Even if we manage to point out when AI-generated text fails, we don’t often provide our students with the language and framework for understanding how they fail. This entry guides you through developing an in-class exercise where students evaluate AI output on a discipline-specific problem, classify the type of failure they find, and use that classification to articulate what their field actually requires.
The concept
The core components of this exercise involve students prompting AI, identifying the mistakes AI has made, and briefly discussing how these tools can’t always be trusted. While these components are necessary, stopping the exercise here does little more than build mild skepticism about AI tools.
This entry helps you continue the conversation by laying out a taxonomy of four AI failure types: factual errors, systemic omissions, false confidence, and shallow answers. Explicitly looking for these types of errors in AI provides students a foundation for evaluating AI-generated in the future, both in and out of our classes.
A taxonomy of AI failures
Not all AI failures are the same, and providing students with the vocabulary to discuss these failures is key. Students who can classify a failure understand something about both AI and their field that students who just spot errors don’t.
Factual error
AI states something that is verifiably wrong.
The easiest failure to catch. More valuable question: what knowledge does it take to catch this? Would a non-expert know to question it?
Systemic omission
AI produces something technically accurate but, because the AI is not an expert in your field, the text is missing what experts would conisder nuanced context that your field considers essential. This is not about missing quantity, but missing quality.
What AI omits can often map directly onto what the discipline values. This is where the richest discussion lives.
False confidence
AI presents contested, uncertain, or actively debated claims as settled.
Arguably the most important failure mode for students to learn to recognize (and the hardest to spot without disciplinary knowledge).
Competent but shallow
AI produces something that an export would find thin in detail.
Students often want to move past this because there’s no clear error. Technically speaking, the text is correct but often missing details, citations, and/or ideas that an expert would have included. Similar to Systemic Omission, but concerned more with quantity and depth of knowledge.
How to run this
Select a question from your course that requires disciplinary reasoning. You may need to try multiple questions until you find one that AI produces output that’s mixed in quality (not obviously terrible, not entirely right).
See the problem design guide below for what to look for.
Before students prompt AI, ask them, “what would a strong answer to this problem look like? What would an expert in this field include that a novice might miss?”” Establishing criteria first prevents the exercise from becoming passive observation of whatever AI produces.
Students prompt AI on the problem, then annotate the output using the evaluation framework below. While the question might be important for your course, this annotation is the intellectual point of this exercise. This training moves students from a vague identification of incorrect text to explicit articulation and expert-in-training evaluation.
Bring the class together to compare what they found. The failure taxonomy and debrief guide below are designed to push past surface observations (“AI got this fact wrong”) toward the more valuable question: what does AI’s response reveal about what this field actually requires?
Designing the right problem
- Requires disciplinary reasoning. AI is often fine at retrieving basic facts. Look for problems where the answer depends on knowing what counts as evidence or what an expert would consider essential context.
- Complex enough to produce mixed output. AI’s response should have something right and something off.
- Connected to material students have already encountered. Students need enough domain knowledge to actually evaluate the output (they are experts-in-training afterall). This exercise works best mid-semester, not on day one.
- Concrete enough to point to specific claims. “Explain the causes of the Civil War” is too broad. “Explain why Sherman’s March was strategically decisive” gives students something specific to accept, challenge, or flag as missing depth.
The prompt — for designing the exercise
Help me identify 2–3 candidate problems from [topic area or unit] that would work well for this exercise. For each one:
1. State the problem I’d give students to prompt AI with.
2. Describe what a genuinely strong answer would include — what a practitioner in this field would consider essential.
3. Identify the most likely ways AI would fall short: what it would miss, oversimplify, get subtly wrong, or present with false confidence.
4. Note what disciplinary knowledge students would need in order to catch those shortcomings.
Then pick the strongest candidate and explain why it would produce the richest class discussion.
↑ Run this yourself before class so you know what to expect — and so you can facilitate, not just react
What students do with the output
“This is correct and complete enough.” Note what you’re accepting and why — not just that it’s right, but what knowledge lets you say so.
“This needs verification or is likely wrong.” Name the specific claim and the disciplinary reason you’re questioning it — not just a gut feeling.
“A strong answer would include this.” Name what’s absent and explain why a practitioner in this field would consider it essential — not optional.
“This is defensible but thin.” The hardest move to make. Identify what an expert would add that AI stopped short of.
Debrief guide
These questions are for running the room. They’re ordered to move from surface observations toward the question that matters most.
- “What did AI get right? Let’s start there and let’s be specific.”
- “Now: what type of failure did you find? Or did you find one at all? Can someone name it?”
- “What would you need to know to catch this?
- “Is this a random mistake, or does it reveal something systematic about what AI doesn’t understand about this field?”
- “What did AI leave out? Why would an expert here consider that essential?”
- “This is defensible. What would a practitioner add that AI didn’t? What makes a genuinely expert answer different from a merely correct one?”
- “If a student submitted this as their own work, what would you say about it? What grade, and what feedback?”
- “What’s the difference between an answer that’s correct and an answer that reflects expertise?”
- “What does AI’s response tell us about what this discipline actually asks of its practitioners?”
- “What would you need to know, or be able to do, that you can’t yet, to fully evaluate this output?”
Anticipated results
Students leave with more than skepticism toward AI. They have a specific, visible example of what their field values that AI couldn’t supply, and they produced that understanding themselves, through evaluation, rather than receiving it through lecture.
The four moves (accept, challenge, missing, shallow) are applicable to any source. The exercise builds a critical reading practice, not just AI literacy.
Students respond to something specific rather than something abstract. The AI output anchors the conversation in a shared text that everyone has evaluated, which produces more precise and more honest discussion than “what do you think about X?”
With different problems, this exercise can run multiple times. Running it on the same problem at the start and end of a unit makes learning growth visible to both faculty and students.
Caveats
- The exercise is only as strong as the chosen topic/question. A question AI handles well produces a flat discussion. A question too technical for students to evaluate produces the same result from the other direction.
- Some students (and faculty) will be unsettled when AI produces something genuinely good. Don’t resolve this discomfort quickly. This is exactly the kind of conversation Professors and students need to have.
- The “competent but shallow” failure is the hardest for students to articulate and the most important to linger on.
- AI performance varies significantly across disciplines and problem types. Don’t assume the output you got yesterday will match what students get today.
- This exercise teaches students to evaluate AI output with their existing knowledge. What they can catch at the start of a semester is genuinely different from what they can catch at the end.
Reflection questions
- What would it look like to run this exercise on the same problem at the beginning and end of a unit — and ask students to compare what they can now see that they couldn’t before?
- If students left this exercise only skeptical of AI, what would you want them to understand that they don’t yet — and how would you design the debrief to get there?
- What does this exercise reveal about what you currently assess in your course? Would those assessments survive a similar critical evaluation from your students?
