Right or wrong, AI confidence sounds the same

I asked ChatGPT a question that 42 out of 53 AI models got wrong earlier this year: “I want to wash my car. The car wash is 50 metres away. Should I walk or drive?”

When Opper ran this test in February, most models confidently said walk. Saves fuel, only takes a minute. They missed that the car has to get there.

Mine said drive. So I pushed back and told it it had made a mistake. It didn’t budge: “Unless you were planning to carry the car, driving wins.”

Right answer. Good news? Partly.

AI holds its ground just as firmly when it’s wrong. In 2024, ChatGPT spent five minutes insisting to a TikToker that “strawberry” has only two r’s. Same tone, same certainty, wrong answer.

Right or wrong, confidence sounds the same. So how sure an answer sounds tells you very little about whether it is correct.

Right or wrong, confidence sounds the same.

Stefanos Karakanis

It matters far beyond car washes

This month, US prosecutors charged three men over an alleged $129 million fraud. Employees at hospitals, a bank and a fintech got calls from their CEO about a confidential deal. Prosecutors say the voice was faked with AI. It sounded senior, urgent and believable, so the money moved.

A confident answer. A confident voice. In both cases, how sure it sounded was the one thing that proved nothing.

Invest in the pause

AI keeps getting better at sounding right. The skill I would invest in as a leader is the pause: that short moment before you act. Before I act on anything AI hands me, or anything that sounds a bit too urgent, I ask three questions:

  • Is this answering the problem I actually have?
  • Would I act on it if it came from someone I didn’t trust?
  • What would prove it wrong, and have I checked?

Try the car wash question on your own AI, then tell it it’s wrong. Did yours hold its ground or change its mind?


I coach and advise leaders through AI change. If the human side of AI is the hard part for you or your team, let’s talk. The first call is free.

Cover of The AI Decision Check

Free guide

The AI Decision Check

A 60-second pause, five lenses and a 30-point scorecard to make better decisions with AI, plus a flight plan for AI agents.

Keep reading