The Rise of Gen AI and “Cognitive Surrender”
By Dr Dawn H Nicholson.
We make thousands of decisions every day, either alone or in tandem with others. Some decisions are made automatically, others need our individual or collective time and investment. Stanovich and West (2000) labelled the systems used to make such decisions as ‘System 1’ and ‘System 2’:
- System 1 is quick and automatic, requiring little or no effort, with no real sense of voluntary control.
- System 2 is different, the mental activities associated with it require effort.
Individuals engaged in System 2 have agency; they need to make choices and can act independently. This requires concentration. System 2 requires activation, whereas System 1 cannot be turned off (Kahneman, 2011).
This Dual System processing has been the accepted ‘norm’ for many years. But now the time has come for a rethink, since we are adding something new into the decision-making mix – Gen AI. We need to understand what this new ‘partner’ means for our decision-making ability and outcomes.
One hint of what we might be setting ourselves up for is encapsulated by an earlier quote from Professor Hannah Fry: "As humans, we are lazy, and we take cognitive shortcuts. Misplacing our trust in machines is a mistake that all of us are capable of doing." (Leprince-Ringuet, 2020).
Tri-System Decision Processing
A new working paper from a Wharton School team summarises research suggesting that this is an entirely legitimate concern. This new research takes Dual System processing and adds a third processing dimension to create a Tri-System theory. The third element of the system is "artificial cognition that operates outside the brain" – i.e. Generative AI. System 3 can supplement or supplant internal processes, introducing novel cognitive pathways.
Cognitive Surrender
A key prediction of this new Tri-system theory is the concept of “Cognitive Surrender”. This is the idea that humans adopt Gen AI outputs with minimal scrutiny, overriding their own intuition and/or deliberation. The mere existence of generative AI can change how we engage our other thinking modes: how we use our intuition; how we use our deliberation and how confident we are when it comes to our decision-making and responses.
This concept was demonstrated across three studies written up in the paper. The researchers found that simply having the option to turn to AI for decision-making assistance led to study participants surrendering their own thought processes, instead allowing the AI to think for them. In turn, this led to them simply adopting the AI-generated answers, ignoring their own System 1 and System 2 processes. Furthermore, adopting the answers from Gen AI increased the confidence of the individual participants’ responses.
Participants in the research were given the option to use Gen AI (ChatGPT) in figuring out their responses to a range of logic and reasoning tests. It was not a requirement for them to use it, yet over 50% of participants chose to use ChatGPT. More concerningly, once they chose to consult ChatGPT, then ChatGPT’s answer was adopted over 80% of the time, even when the answer given by ChatGPT was incorrect – something which the researchers manipulated experimentally. The researchers named this phenomenon “Cognitive Surrender” – a scenario where the decision-maker ceases to construct their own answer, instead adopting one generated by an external Gen AI system – in this case, ChatGPT.
Confidence in Decision-Making Outputs
The researchers also found that AI outputs could inflate the participants’ confidence, even when they were incorrect. This is concerning since previous research by Arnott (2006) identified confidence biases, which lead to an overinflated sense of our own prowess as decision-makers, as particularly damaging in decision-making processes. Not only do they increase a person’s belief in their own ability as a decision-maker, but they also curtail the search for new information relating to the decision task.
Conclusions and Implications
This new research suggests that as humans, we risk becoming over-reliant on AI, losing our own capacity to think. An extreme view? Maybe. But Cognitive Surrender does suggest the possibility of abdication of critical evaluation, where we humans relinquish all cognitive control and simply adopt the AI’s judgement as our own. Accuracy is surrendered to System 3 accuracy, and as we all know – or should know by now - Gen AI is not accurate 100% all of the time.
As Business Psychologists, we need to carefully consider the implications of this research in organisations. Key questions should include:
- How and when is AI integrated into decision-making processes?
- Who is engaging with it – at which stage of the decision-making process and to what end? Do they know enough to challenge the AI?
- How does access to AI affect confidence and decision-making outcomes?
- How can decision makers be supported to use Gen AI and System 3 effectively, whilst ensuring they maintain critical thinking and accountability for the decisions made?
- What happens to us when our human thought processes – including inference, evaluation and justification are effectively ‘outsourced’ to AI systems?
One thing’s for sure. Gen AI is not going away, so the need for real thought and action is now!
About the Author
Dr Dawn H. Nicholson is Vice Chair of The ABP and Head of University Accreditation, with almost three decades of HR and consulting experience and a PhD in Decision-Making Psychology. A Chartered Psychologist and ABP Fellow, she has led Business Psychology programmes at Kent and Arden universities and remains active in the Business Psychology industry. As Biz Psych Cup Lead, she champions collaboration between academia and business. Passionate about guardianship and growth of the field, Dawn works to raise standards, support emerging talent, and promote the value of Business Psychology. She also enjoys countryside walks with her two lively cocker spaniels.
References
Arnott, D. (2006). Cognitive biases and decision support systems development: A design science approach. Information Systems Journal, 16(1), 55–78. https://doi-org.chain.kent.ac.uk/10.1111/j.1365-2575.2006.00208.x
Kahneman, D. (2011). Thinking, Fast and Slow. London: Penguin.
Leprince-Ringuet , D. (2020). AI's big problem: Lazy humans just trust the algorithms too much
https://www.zdnet.com/article/ai-needs-to-be-controlled-but-lazy-humans-may-not-be-up-to-the-job/
Shaw, S. D., & Nave, G. (2026). Thinking Fast, Slow, and Artificial: How AI is Reshaping Human Reasoning and the Rise of Cognitive Surrender https://osf.io/n84tx
Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate? Behavioral and Brain Sciences, 23(5), 645–665. https://doi-org.chain.kent.ac.uk/10.1017/S0140525X00003435