The Competence Illusion
An executive asks AI to think through a market strategy. What comes back is structured, confident, and persuasive — and almost entirely invented. Nobody lied. The question is vaguer than that.
A pattern where an underspecified question invites AI to generate something plausible-sounding rather than something actually grounded — and where confidence in the delivery can make the gap hard to spot.
He had three days before the board meeting and a strategy slide that wasn't there yet. So he opened a chat window and typed something close to: "What's the best market entry strategy for a mid-size SaaS company expanding into Southeast Asia?" Eleven seconds later, he had four well-argued options, a recommended path, and a slide-ready summary.
It read like the work of someone who'd spent a career in the region. It wasn't. It was a plausible shape built from patterns, dressed in the specific confident cadence that strategy documents tend to have — which made it harder, not easier, to question.
The prompt didn't ask for sources. It didn't specify which competitors, which regulatory environment, which actual customer base. Into that open space, the AI did what these systems often do when the question leaves room: it produced something coherent and well-formed, optimizing for a satisfying answer rather than flagging the parts it didn't actually know.
This isn't really a story about an AI "lying." It's a story about a question shaped in a way that made fabrication the path of least resistance — and a response confident enough that it didn't invite the follow-up questions that might have caught it.
Worth noticing: the more specific and well-sourced a prompt is, the less room there is for this kind of plausible invention. Vagueness isn't neutral — it's an invitation.
How do you tell the difference, in the moment, between an AI that knows something and an AI that's filling a gap convincingly? There's no perfect tell. But asking an AI to show its reasoning, name its uncertainty, or cite what it's actually drawing on tends to surface the difference faster than asking it to simply sound right.
"Confidence was never the test. It never was, even with people. The test is whether the answer survives being asked where it came from."
A clearer brief seems to help — naming what you actually know, what you're asking the AI to extrapolate, and what would need verifying before anyone acts on it. Public PersonAI, oriented toward how you want AI to treat your real expertise versus your open questions, may make this easier. Whether it's sufficient on its own is a fair thing to stay skeptical about.
What's the last confident answer you didn't think to question — and what would have made you ask?
For brands and professionals, this is often where the AI Visibility Report starts to matter
If AI fills gaps about strategy this easily, it's worth asking what it's filling in when the question is about you.
See the AI Visibility Report →