In many parts of the world, including major technology hubs in the US, there’s a years-long wait for AI factories to come online, pending the buildout of new energy infrastructure to power them.
Now Emerald AI, a US startup, is developing an AI solution that could enable the next generation of data centres to come online sooner by tapping existing energy resources in a more flexible and strategic way.
“Traditionally, the power grid has treated data centres as inflexible — energy system operators assume that a 500-megawatt AI factory will always require access to that full amount of power,” says Varun Sivaram, founder and CEO of Emerald AI. “But in moments of need, when demands on the grid peak and supply is short, the workloads that drive AI factory energy use can now be flexible.”
That flexibility is enabled by the startup’s Emerald Conductor platform, an AI-powered system that acts as a smart mediator between the grid and a data centre. In a recent field test in Phoenix, Arizona, the company and its partners demonstrated that its software can reduce the power consumption of AI workloads running on a cluster of 256 Nvidia GPUs by 25% over three hours during a grid stress event while preserving compute service quality.
Emerald AI achieved this by orchestrating the host of different workloads that AI factories run. Some jobs can be paused or slowed, like the training or fine-tuning of a large language model for academic research. Others, like inference queries for an AI service used by thousands or even millions of people, can’t be rescheduled, but could be redirected to another data centre where the local power grid is less stressed.
Emerald Conductor coordinates these AI workloads across a network of data centres to meet power grid demands, ensuring full performance of time-sensitive workloads while dynamically reducing the throughput of flexible workloads within acceptable limits.
Beyond helping AI factories come online using existing power systems, this ability to modulate power usage could help cities avoid rolling blackouts, protect communities from rising utility rates and make it easier for the grid to integrate clean energy.
“Renewable energy, which is intermittent and variable, is easier to add to a grid if that grid has lots of shock absorbers that can shift with changes in power supply,” says Ayse Coskun, Emerald AI’s chief scientist and a professor at Boston University. “Data centres can become some of those shock absorbers.”
Emerald AI is a member of the Nvidia Inception program for startups and an NVentures portfolio company.
Using the grid to Its full potential
Electric grid capacity is typically underused except during peak events like hot summer days or cold winter storms, when there’s a high power demand for cooling and heating. That means, in many cases, there’s room on the existing grid for new data centres, as long as they can temporarily dial down energy usage during periods of peak demand.
A recent Duke University study estimates that if new AI data centres could flex their electricity consumption by just 25% for two hours at a time, less than 200 hours a year, they could unlock 100 gigawatts of new capacity to connect data centres — equivalent to over $2-trillion in data centre investment.
The International Energy Agency projects that electricity demand from data centres globally could more than double by 2030.