Kathy Gibson reports – There is no doubt that AI is here to stay, and that organisations around the world are using it to improve their processes and competitiveness.
But there is an awful lot that you won’t get told about actually implementing AI before you sign up, says Barry Buck, chief technology officer at Saucecode and architect of Roboteur.
“Lately I have found myself in a lot of meetings where the topic of discussion is not whether AI can do the job, but whether companies can justify the cost of keeping it running,” he says.
“These meetings are probably the most honest conversation people have about AI adoption at scale; when they discuss all the things that the companies selling AI transformation didn’t tell you.”
The sales pitch is all about reinventing how the organisation works, with terms like transformation, agentic, the end of the job title.
“It’s only when the invoice arrives that you see the full implications of AI consumption,” Buck explains.
“And, after a decade of confident demos, I am convinced that every enterprise buyer has been burnt.”
This issue can be summed up in the new buzzword, tokenomics, where users start adding up the cost of AI consumption on top of their licensing fees.
Yes, AI can be credited with massive savings, Buck says. “We’ve all seen the numbers – millions of records, millions of rands and trillions of nanoseconds saved.
“But we aren’t being told upfront about the difference between what is being sold and the reality of running the systems.”
How AI applies the brakes
There are two brakes to AI adoption that buyers are generally not told about upfront, Buck believes. The first is money and the cost of intelligence; the second is geography.
“When it comes to the money, AI is being sold like transformation, but it is billed like electricity consumption. Except no-one can read the meter.
“The problem is, there is no clear definition of what a credit is worth: it depends on the prompt, the scan, the retries. The only way to measure it is to see the invoice. So how can you deterministically plan for what your process will cost in production.”
And, once it’s in production, organisations are committed – and have to pay up.
“Safety costs credits too,” Buck points out. “You assume your credits are used for the model to work, but in production it spends a lot of time being supervised. By the time a prompt is safe enough to run in an enterprise there is often very little budget left to do the work.”
Buck recommends moving work out of the prompt, with developers steering away from big prompts that often fail and end up being expensive.
Instead, he suggests using AI to do extracts and sanity checks, with an engine to apply the rules. “Rules are free, repeatable and auditable. Tokens are none of these things.
“In this way, the bulk of calls can be moved to less expensive models, with the frontier reserved for the steps that earn it.”
The other brake is the border, he says.
“Enterprises all have sensitive information. But, as far as we can tell this month, no major vendor runs a current frontier model on a GPU inside South Africa.
“Read your contract then read the footnote.”
Since frontier models typically run offshore, Buck proposes a solution for ensuring no personal information leaves the country.
“The frontier model says what the next step should be, and shapes the playbook on data that identifies no-one; and a senior person signs off on it.
“So the frontier model proposes; a human disposes; and a sovereign model executes.”
Both of these brakes happen at run-time, he adds, recommending a hybrid solution that uses vendor-led frontier models with high token spend only where it makes sense in order to maintain data sovereignty while reining in costs.
Security conundrum
Cybersecurity is a big concern for AI adopters, Buck says.
“Hackers have had access to these tools since 2023,” he says. “They don’t have to worry about procurement processes, cross-border reviews or no credits meetings. So your fraud team is competing with an organisation that went agentic in 2023.”
This is not a call for recklessness, Buck says. But customers need to be as organised as the hackers.
“In my experience, the answer is hybrid. You have to evaluate the threat, and innovate how you apply the technology.”