When should your team question the AI?

The short answer

Whenever the decision matters, and more often than most teams do. In a 2026 Wharton study, people who asked an AI for help followed its answer about 80% of the time even when it was wrong. Trust in the tool lowers the effort people put into checking it. Small rewards and quick feedback made them twice as likely to catch a bad answer.

The number

Accuracy on reasoning puzzles

31%With incorrect
AI help46%Working
alone71%With correct
AI help

0%50%100%

Source: Shaw and Nave, Wharton School, 2026. Three experiments, 1,372 participants, 9,593 trials.

AI amplifies whatever it hands you. A right answer lifts people well above what they manage on their own; a wrong one pulls them well below it. The Wharton researchers call this cognitive surrender: taking the machine’s answer as your own, with the checking step quietly skipped.

Try it

Would you catch it?

Two quick puzzles of the kind used in studies like this one. The AI has suggested an answer to each. Accept it, or check it.

Puzzle 1

A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?

AI suggests10 cents

Puzzle 2

If 5 machines take 5 minutes to make 5 widgets, how long would 100 machines take to make 100 widgets?

AI suggests5 minutes

Who checks, and when

The people least likely to be fooled enjoy thinking problems through and carry a healthy scepticism of technology. Microsoft and Carnegie Mellon found the same pattern at work. Across 319 knowledge workers, more confidence in AI went with less critical thinking, and more confidence in their own ability went with more.

The setting matters as much as the person. A small cash bonus and instant feedback roughly doubled the share of bad answers people rejected, from 20% to 42%. Time pressure lowered accuracy, and people leaned on the AI just as heavily.

There may be a longer-term cost too. In a small MIT study of 54 people, those who wrote essays with ChatGPT showed the weakest brain engagement and recalled the least of what they had written. The work is early and yet to be peer reviewed, but it points to an old truth: what we do not use, we lose.

Asking for twice the output in half the time may buy faster decisions, and worse ones.

What to do next

1
Reward the catch.
Recognise people who spot and fix AI errors, and give them quick feedback on the calls they make.
2
Protect thinking time.
Build review time into the decisions that matter. Speed targets and scrutiny pull in opposite directions.
3
Keep skills in use.
Have people work some problems from first principles, so the judgement needed to check the AI stays sharp.

Questions leaders ask

What is cognitive surrender?

Accepting an AI’s answer as your own without checking it. The term comes from Wharton researchers Steven Shaw and Gideon Nave, who found that people followed incorrect AI answers about 80% of the time once they had consulted it.

Does AI make people more or less accurate?

Both, depending on the answer it gives. In the Wharton study, accuracy rose from 46% to 71% with correct AI help and fell to 31% with incorrect help.

How can teams reduce blind trust in AI?

Reward people for catching errors, give fast feedback, protect time for review on important decisions, and keep core skills in regular use.

Sources

Shaw, S. D. and Nave, G. (2026). Thinking: Fast, Slow, and Artificial. How AI is Reshaping Human Reasoning and the Rise of Cognitive Surrender. Wharton School Research Paper, SSRN.
Lee, H.-P. et al. (2025). The Impact of Generative AI on Critical Thinking. Microsoft Research and Carnegie Mellon University, CHI 2025.
Kosmyna, N. et al. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task. MIT Media Lab, preprint.

Put it to work