Boards are becoming comfortable with using AI as a tool for drafting, summarising and searching – but the technology has moved on. The latest AI systems don’t simply retrieve and generate – they reason, working through problems step-by-step to construct complex arguments. Most leaders are still catching up, unaware of what’s possible and underprepared for what can go wrong. 

INSEAD recently brought this conversation into focus at a Lifelong Learning webinar with Miguel Sousa Lobo, Associate Professor of Decision Sciences at INSEAD. The conversation, moderated by INSEAD alumnus Kevin Pereira (MBA’14D), Managing Director at Blu Ltd – Artificial Intelligence, focused on what reasoning AI models means for leaders.

The risks leaders aren’t seeing

• Organisations may be familiar with obvious AI risks such as a system confidently stating something false as true. The emerging risks are more subtle: the newest models are less prone to obvious hallucinations, yet Lobo flags something more insidious – higher-order delusions. The more capable the system, the more confidently it can support a wrong conclusion with coherent, well-structured logic.
• Some models are built to agree with you. This isn't a glitch, it's a design choice. Certain AI systems are trained to reinforce user confidence rather than challenge it. Think of instances when ChatGPT responded with "a great question" or praised a piece of work that was, objectively, not great. That tendency to flatter rather than interrogate remains when the stakes rise. In a boardroom context, AI can shape decisions, validate poor strategy and erode the culture of scrutiny that good governance depends on.

Where human judgement matters

As AI systems become more capable, the humans overseeing it need to raise their game. Not by learning to code nor mastering prompt engineering – but by honing their judgement. Know when to push back on a well-reasoned but wrong conclusion. Understand how these systems have been trained, what they optimise for and where they are structurally inclined to mislead.

The need for human oversight doesn’t decrease as AI improves, it’s increasing. What’s changed is where judgement matters most: in oversight, validation, governance and in understanding how these systems are shaped by their training and incentives. The question for leaders is no longer “Can we trust this tool?” It's “Do we have the frameworks and the governance to challenge it when it's wrong?” 

Right now, most boards are only asking the first one.