Agreement is not verification: when three AIs agree

You ask ChatGPT to evaluate a business idea. It sounds promising. You ask Claude. Same conclusion. You check with Perplexity. It agrees too.

Three different AI systems. Three similar answers. You feel reassured. But what exactly have you verified?

Different models, same mistake

Researchers studying more than 350 AI models found that their mistakes can be remarkably similar. On one benchmark, when two models were both wrong, they gave the same wrong answer about 60% of the time.

Different AI systems can rely on overlapping information, share similar assumptions or reproduce the same misconceptions. So what feels like independent confirmation can be an echo chamber with three different logos.

Agreement is not verification.

Stefanos Karakanis

Why leaders should care

I see real value in comparing models, challenging their reasoning and exploring alternative perspectives. But agreement is not verification. Three models agreeing can increase our confidence without increasing the quality of our evidence.

That becomes especially consequential when AI influences hiring, investments, business strategy or decisions affecting people’s lives.

Try this instead

The next time several AI systems agree, don’t immediately ask a fourth. Ask: “What evidence would prove all three of you wrong?” Then check that evidence independently.

Sometimes the greatest risk isn’t getting the wrong answer. It’s becoming so confident in that answer that we stop looking for the truth.


I coach and advise leaders through AI change. If the human side of AI is the hard part for you or your team, let’s talk. The first call is free.

Cover of The AI Decision Check

Free guide

The AI Decision Check

A 60-second pause, five lenses and a 30-point scorecard to make better decisions with AI, plus a flight plan for AI agents.

Keep reading