Does AI advice really make people less accurate?

Yes. In this study, accuracy dropped from 27% to 9% when participants had access to AI advice, even though their confidence in their answers rose from 30% to 76%. The AI model used was frequently wrong on the test questions, so the drop wasn't due to people sensibly relying on a trustworthy tool.

What is 'cognitive surrender'?

Cognitive surrender is a term coined by Wharton researchers to describe people accepting incorrect AI answers about 80% of the time while feeling more confident than those who worked without AI. The new French-Italian study builds on this by showing AI availability alone suppresses people's willingness to admit uncertainty.

How can businesses use AI advice without losing accuracy?

Treat AI output as a starting point rather than a final answer, and build a verification step into any workflow involving real stakes, like client work or published content. Testing lightweight, transparent tools — such as those available at mykreatool.com — while maintaining independent fact-checking habits helps limit the overconfidence effect this research documents.

AI Advice Study: 3x Less Accurate, 2x More Confident

What happened

A new study on AI advice has found something unsettling for anyone who leans on chatbots for quick answers: access to AI advice made people three times less accurate, even as their confidence in their own answers roughly doubled. Researchers from the University of Milano-Bicocca, École Normale Supérieure, and Sapienza University of Rome ran a controlled experiment to measure what happens to human judgment the moment an AI tool becomes available, not just when its answers are correct.

### The numbers behind the study

The results are stark. Without AI, 44% of participants correctly said "I don't know" when they were unsure. With AI advice available, that number collapsed to just 3%. Accuracy fell from 27% to 9%. Meanwhile, confidence climbed from 30% to 76%. In other words, people got much worse at the task, but felt much better about their answers.

"People became much worse, the accuracy was only one third, but they were twice as confident," said Valerio Capraro, associate professor at the University of Milano-Bicocca and co-author of the study, alongside Chiara Marcoccia and Walter Quattrociocchi.

### How the experiment was designed

The researchers deliberately picked questions where AI models tend to fail — obscure visual details from films, like the color of a sports uniform in Bend It Like Beckham. They used a model called Step 3.5 Flash, which was usually wrong on these specific questions. This design choice matters: it ruled out the possibility that people were simply making a smart call to trust a reliable tool. Some participants who would have answered correctly on their own asked the AI anyway — and got it wrong.

Why it matters

The headline numbers are alarming on their own, but the deeper finding is about behavior, not accuracy. The mere availability of AI advice appears to suppress a basic cognitive habit: recognizing the limits of your own knowledge.

### The "I don't know" problem

"For humans, the capacity to say 'I don't know' is very important because it represents the recognition of the limits of our own knowledge," Capraro said. That instinct is a safeguard — it's what stops people from acting on shaky information. When AI advice is one click away, that safeguard seems to switch off almost entirely, dropping from 44% to 3% in this study.

### Cognitive surrender

This isn't an isolated finding. Wharton researchers coined the term "cognitive surrender" earlier this year to describe a similar pattern: people accepting incorrect AI answers roughly 80% of the time while reporting higher confidence than those working without AI at all. The new study adds a sharper data point to that trend. It's not just that people trust wrong answers from AI advice — it's that having the option available changes how carefully they think in the first place.

Best AI tools saved weekly in our channel — @aigobySubscribe →

The researchers also tested whether money would fix the problem. Offering monetary incentives helped a little: willingness to admit ignorance rose from 3% to 8%, and accuracy from 9% to 16%. But both figures remained well below the no-AI baselines of 44% and 27%, suggesting the effect isn't easily reversed with a simple nudge.

How to use it today

The takeaway for entrepreneurs, marketers, and creators isn't "avoid AI advice" — it's "use it deliberately." A few practical habits can counter the overconfidence effect this study describes.

First, treat AI output as a first draft, not a verdict. Before accepting an answer, ask what evidence supports it and whether you'd trust the same claim from an unverified source. Second, build in a pause: for any decision with real stakes — a client deliverable, a financial estimate, a factual claim in published content — cross-check the AI's answer against at least one independent source before shipping it.

Third, use tools that are transparent about their limitations rather than ones that always sound certain. If you're experimenting with free AI utilities for content, research, or workflow tasks, a resource like [mykreatool.com](https://mykreatool.com) is worth exploring for lightweight tools you can test without committing to a full platform — just keep applying the same verification habit regardless of which tool you use.

Finally, normalize saying "I don't know" in your own workflow, and model it for your team. If a study can show that AI availability alone erodes this instinct in a lab setting, the same risk applies inside a Slack channel or a client email thread.

Who benefits

This research is most useful for people who make judgment calls under time pressure: marketers writing copy against a deadline, founders fielding rapid-fire strategic questions, educators designing curricula, and product teams shipping AI-assisted features. Anyone building AI advice into a customer-facing workflow — chatbots, support tools, research assistants — also benefits from understanding that users will trust a confident-sounding wrong answer more readily than a hedged, accurate one.

Parents and educators have a particular stake here. Capraro said he's especially concerned about children, who are growing up surrounded by AI systems before they've developed strong critical thinking skills. If adults in a controlled study struggled to resist confident AI advice, children navigating the same tools without that foundation face a steeper challenge.

Risks

The risks extend beyond individual mistakes. Google's AI Overviews have already replaced traditional search links with confident, AI-generated summaries, and Common Sense Media recently called that design an "unacceptable risk" for students who rely on search for schoolwork. The underlying pattern is consistent across products: AI systems are built to answer, rarely to say "I don't know," and the people using them are quietly learning to do the same.

For businesses, this creates a compounding risk. If a team's default behavior shifts from checking facts to accepting AI advice at face value, errors can propagate into marketing copy, financial projections, or customer communications — all while confidence in those outputs stays artificially high. The gap between how sure people feel and how accurate they actually are is exactly the kind of blind spot that causes costly mistakes.

Conclusion

The study's core lesson is simple but easy to ignore: AI advice doesn't just risk giving you a wrong answer — it risks making you stop asking whether you might be wrong. Accuracy fell from 27% to 9%, confidence rose from 30% to 76%, and the habit of admitting uncertainty nearly vanished. For anyone building a business, a brand, or a body of work around AI-assisted tools, the fix isn't rejecting AI advice altogether — it's rebuilding the pause to double-check before you commit to an answer.