AI-generated content is never 100% reliable and should always be critically evaluated before using.
Understanding how AI generates responses is helpful when considering it’s limitations. As we discussed in Section 1, AI models predict and generate outputs based on patterns in the data they were trained on, not through independent reasoning or human-like cognition.
Some important limitations to consider:
- AI Hallucinations – AI at times generates incorrect information. One hallmark of AI generated text, is the citation of non-existent sources.
- AI Biases – AI generated content will reflect biases present in the texts on which it has been trained (e.g., norms & values, language)
- Lack of Contextual Understanding – AI does not have a representation of ideas analogous to a human’s ability to make meaning of the concepts underlying the text it generates nor does it have an understand based on experience in the real world. Instead, AI generates text based on statistical relationships to the words you enter within a given chat window. For example, it will create “a random response with no basis in fact” in respond to questions such as “could [this text] have been written by AI?”
- Inconsistencies in Responses – AI-generated answers vary even with similar prompts.
Anthropic’s course in AI Fluency, introduces the idea of discernment, as one skill critical to analyzing and evaluating AI’s outputs. After generating something with AI, it is critical to stop and carefully reflect on what was produced.
- Check for Factual Accuracy – Verify specific claims, statistics, dates, and references. AI can confidently present incorrect information, so cross-reference with credible sources, especially in areas where you lack deep expertise.
- Identify Potential Hallucinations – Look for fabricated citations, non-existent studies, or invented quotes. Pay special attention to specific names, publications, or technical details that seem too convenient or precise.
- Examine for Bias and Perspective – Consider what viewpoints or voices might be missing from the output. Ask: Whose perspective does this reflect? What assumptions are embedded in this response?
- Test for Logical Consistency – Check if different parts of the response contradict each other or if the reasoning follows logically from premise to conclusion.
- Review for Writing Quality – Look for vagueness, redundancy, or sections that sound polished but lack substance. AI often produces text that appears sophisticated but may be hollow or circular in reasoning.
- Assess Appropriateness for Purpose – Determine if the tone, depth, and approach match your intended use and audience.
Examples of AI Making Mistakes:
There are many popular threads of mistakes made by AI.