It’s becoming a mistake to believe that a “human-in-the-loop” (HITL) strategy for managing the risks of agentic artificial intelligence (AI) within organisations is viable.
As agentic AI expands, integrates and improves, relying on HITL oversight to avoid disastrous AI mistakes is betting on a system that’s already reaching breaking point, argues Anna Collard, senior vice-president of content strategy and CISO advisor for KnowBe4 Africa.
The urgent pivot required boils down to a “human-on-the-loop” (HOTL) model – where humans move from being active participants in every digital transaction to becoming the strategic architects of the systems themselves.
The HITL model assumes that a person can effectively supervise a machine-speed agent. However, Collard points out that this ignores the reality of human biology.
“Asking an employee to monitor a high-speed AI agent and react immediately when a transgression occurs is pretty much impossible,” Collard explains. “In the HITL model, you’re asking the human to be the pilot of a jet that moves at Mach 10. They literally can’t move the stick fast enough to avoid a mountain.
“Furthermore, having to do many approvals or reviews can lead to what psychologists call ‘vigilance decrement’ – where our attention moves from being heavily taxed to eventually just shutting down.”
Moving ‘on-the-loop’: The architect of constraints
In a HOTL model, the human moves from the centre of the action to the perimeter of the strategy. Collard describes this new role as the “architect of constraints”.
“The human’s role then isn’t to check every output, but to build the guardrails that define an agent’s jurisdiction,” says Collard. “We move from being administrators to sense-makers. In the pilot analogy, the human moves to the Control Tower. They aren’t holding the controls for every take-off and landing, but they set the flight paths, define the safety parameters, and intervene only when the radar shows a systemic conflict.
If an organisation deploys an AI agent to manage logistics, the human architect ensures that the code-based constraints prevent that agent from accessing financial systems. As our own CTO often says: if your agent is meant to make ice-cream, the system constraints must ensure it cannot build a bomb.”
In this framework, the human doesn’t attempt to audit every outcome, a task that would simply recreate the bottleneck. Instead, they operate at the governance layer, stepping in only when real-time monitoring or runtime guardrails detect a deviation from intent. Their role is to manage the ‘drift,’ ensuring the agent’s systemic goals remain anchored to the organisation’s values, while the automated safety net handles the millisecond-by-millisecond enforcement.
Tactical support: AI mentors for the analogue brain
While the HOTL model provides the strategic framework, organisations still face the immediate tactical threat of AI-powered attacks. Because humans cannot be expected to detect these machine-speed threats unassisted, the secondary theme of this new era is the use of AI to fight AI.
This is where AI Defence Agents (AIDA) become a critical tool for the “On-the-loop” architect. Rather than relying on traditional security awareness training alone, AIDA acts as a real-time coach.
Psychologist Hermann Ebbinghaus famously proved that humans forget nearly 50% of new information within an hour, and up to 80% within a month. If security awareness training is delivered as an annual “one-and-done” event that knowledge quickly decays.
“AIDA bridges the gap between ‘knowing’ and ‘doing’ by providing continuously optimised, individualised security training, including a real-time nudge at the exact moment of risk,” explains Collard. “Nudges act as Spaced Practice, reinforcing knowledge exactly when it’s needed, which prevents the cognitive decay that leads to mistakes.
“By interrupting the user’s instinctive ‘System One’ autopilot and triggering their analytical ‘System Two’ thinking only when and if it matters, these AI mentors allow the human to maintain their role as the ultimate decision-making fail-safe.”
AIDA’s power lies in how it enables organisations to move from centralised control to distributed intelligence. Instead of relying on a small security team to monitor everything, tools like AIDA embed guidance directly into the workforce at scale.
“The goal of the agentic era isn’t to replace human judgment, but to protect it,” Collard concludes. “By moving humans ‘on-the-loop’, we remove them from the exhausting task of monitoring and elevate them to the role they were designed for: strategic oversight and nuanced decision-making.”