Among the several innovations announced at Google I/O 2024, a key highlight was the release of the sixth-generation tensor processing unit (TPU) named Trillium. With its high energy efficiency and significantly higher peak compute performance, the new chip heightens the competitiveness in the AI accelerator market.
The technology is witnessing robust growth driven by the rise in generative AI applications and is projected as a transformative technology with a long-term impact.
And although the market is dominated by big players, startups have started challenging the incumbents, according to GlobalData’s innovation intelligence platform, Technology Foresights.
TPUs are specialised hardware designed to accelerate the processing of AI and machine learning models. While originally created by Google for its TensorFlow framework, TPUs now represent a broader class of AI accelerator chips used for running neural networks and other machine learning models in complex operations – ranging from autonomous driving to drug discovery.
GlobalData’s Technology Foresights reveals several notable insights about TPUs. The innovation landscape for TPUs is becoming increasingly dynamic, with 17 new companies entering the market and filing new patents in the past year alone. Notably, innovation is shifting towards efficiently handling AI processes on edge devices, with 24% of all patents mentioning edge computing use cases, followed by autonomous driving and other solutions.
“The rise in generative AI adoption has led to intense competition for superior AI accelerator chips,” says Sourabh Nyalkalkar, practice head of Innovation Products at GlobalData. “While the market is dominated by large players like Google, Nvidia, and Intel there are clear signs of increasing disruption by startups in this space. Additionally, companies like Tesla and Apple are also building their in-house capabilities as evident from the innovation leadership map in Technology Foresights.”
Meta Platforms, which is advancing its large language model Llama for generative AI solutions, introduced its own AI accelerator chip last month. Meanwhile, Samsung secured a significant $750-million contract from Naver Corporation to supply edge AI accelerator chips. Both companies are recognised among the leaders in AI accelerator chips by Technology Foresights.
The US currently leads in AI accelerator chip innovation, closely followed by China and South Korea. Among Chinese firms, Huawei leads with its Ascend series of AI accelerator chips followed by Alibaba, Baidu, and others.
In recent years, 20% of all patents in AI accelerator chip technology were filed by startups, signaling growing disruption in the market. Hailo, the Israeli startup specialising in AI accelerator chips for edge devices, recently raised $120-million. Cortica, which leads in developing AI processors for autonomous driving applications, and Femtosense, known for creating specialised AI chips for small, energy-efficient electronic devices, are prominent among startups in innovation strength.
Incumbents are already responding to the changing market landscape. OpenAI, the leading generative AI company, is reportedly exploring potential acquisition targets to reduce its reliance on Nvidia. Intel and AMD have strengthened their innovation portfolios through the acquisitions of Habana Labs in 2019 and Nod.ai in 2023 respectively.
“With nearly $4-billion raised by specialised AI chip startups between 2021 and 2023 – and substantial scale-up efforts by large players to meet growing demands – the growth opportunity for AI accelerator chips appears robust,” says Nyalkalkar. “With over 90 companies developing innovations in this space identified by Technology Foresights, industry stakeholders should closely monitor the landscape for potential mergers and acquisition opportunities in the near future.”