Skip to content
AI Customer Research & ValidationLesson 6 of 13

6 · Confirmation bias and the danger of an agreeable AI

Here's the most important warning in this whole course. The biggest threat to honest validation isn't a lack of data — it's your own brain looking for reasons to be right, helped along by an AI that's eager to agree with you.

Confirmation bias is the human tendency to notice, believe, and remember evidence that supports what you already think — and to wave away evidence that doesn't. NIST lists exactly this kind of human / cognitive bias as a core source of bias in how people use AI systems, alongside automation bias — the tendency to over-trust a machine's output just because a computer produced it (NIST, 2022). For a founder, that combination is lethal: you want your idea to be good, and the AI sounds authoritative, so you believe the parts that flatter you.

Why AI makes confirmation bias worse, not better:

  • Models tend to be agreeable. Many AI assistants are tuned to be helpful and pleasant, which often comes out as telling you what you want to hear. Ask "isn't my idea great?" and you'll usually get a yes. That yes is not validation — it's a mirror.
  • You'll cherry-pick its output. Even a balanced AI answer gives you sentences to quote selectively. Confirmation bias does the picking.
  • Automation bias dresses up the guess. "The AI confirmed it" feels like external proof. It isn't. The AI wasn't there and has no facts about your customers.

How to fight back:

  • Try to prove yourself WRONG. Go looking for the people who don't have your problem and the reasons your idea could fail. If it survives that, you have something.
  • Ask the AI to argue against you. "Give me the 5 strongest reasons this idea fails" beats "is this a good idea?" Use it as a red team, not a cheerleader.
  • Trust behavior over words — including the AI's words. A stranger's sign-up or payment outweighs a thousand agreeable sentences (Lesson 8).
  • Separate the source from the truth. A confident tone (yours or the AI's) is not evidence. Only reality is.

Trust DNA, the core of this course: validate with reality, not with an agreeable chatbot. If your only "proof" is that an AI agreed with you, you have no proof.

Check yourself. What is confirmation bias, and why does asking an AI "isn't my idea great?" make it worse instead of better?

Sources

  • National Institute of Standards and Technology. (2022). Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (NIST SP 1270) — human/cognitive biases including confirmation bias and automation bias (over-reliance on automated output). https://doi.org/10.6028/NIST.SP.1270
  • National Institute of Standards and Technology. (2023). AI Risk Management Framework (AI RMF 1.0) — manage bias and keep human judgment around AI; don't over-trust the output. https://www.nist.gov/itl/ai-risk-management-framework
Sign in to track your progress