Kathy Gibson reports – Somethings artificial intelligence (AI) makes mistakes – and the consequences could be dire.
But when this happens, organisations can’t simply pass off the mistake as a technical error and move on. Because when the AI messes up, the responsibility falls to the company.
Malcolm McDonald, CIO of Youth Employment Services (YES), told delegates to the AI Expo in Sandton last week that organisations need to factor in this accountability when they develop their AI policies.
“Consider the stakes,” he says. “not every organisation is the same, so the stakes must determine your policy.”
For instance, he says, a corner café probably has recoverable mistakes, so a minimal policy would be needed. But a listed company with vulnerable data has existential consequence and this must be reflected in its policy.
At the same time, he cautions against developing long, complex documents as a policy will be useless if it’s not read and used.
Instead, he advocates the OODA loop whose invention is credited to the fighter pilot John Boyd, who aimed to react to any circumstance in 40 seconds.
OODA stands for “observe, orient, decide, act”, and forms the basis of every decision.
The first step, observe, is the input from any of the senses.
The second, orient, is the longest and slowest step, where the observations need to be positioned within the person’s worldview. “Your culture, instinct, upbringing, circumstance, education – everything you are is art of orientation,” McDonald says.
The last two steps, decide and act, then need to happen quite quickly, he adds.
The key is to get people through the orient phase as quickly as possible, as that informs what the decision and action will be.
“A policy document is an orientation tool for the operations of your business,” McDonald says. “The objective is to orient staff so they know exactly what to do when they encounter a situation.”
He stresses that the policy document should only be as long or complex as necessary.
“For instance, our policy is six pages. This covers all the scenarios that we need to cover for now.”
YES also has only human-in-the-loop processes because the policy determines that humans must be involved in any process that affects a human life.
McDonald warns that humans should always double-check work produced by AI.
“Formatting is not evidence,” he says. “AI presents wrong answers as confidently as right ones; and fabricated citations and hallucinated figures can be beautifully formatted.
“So verify before you forward anything – the audit trail ends at your desk.”
This is necessary because the people in the company remain accountable. “The AI is never accountable.”
He recommends that companies reference global standards when crafting the AI policies, including ISO/MED 42001:2023, NIST AI RMF, OECD AI Principles and IEEE 70002021.
“And make the policy user-friendly,” he stresses.