Programmable AI Silicon: The Key to Handling Growing AI Workloads
AI is influencing every aspect of technology—hardware, software, automation, and more. At the recent Silicon Catalyst Spring 2025 Portfolio Company Update, experts discussed the rising energy demands of AI and expressed confidence that innovations in silicon hardware will address these challenges.
But what form will this hardware innovation take? At the ITF World 2025 conference in Antwerp, Luc Van den hove, CEO of imec, emphasized that AI’s future depends heavily on hardware breakthroughs, particularly programmable AI silicon.
The Challenge of Diverse AI Workloads
As AI advances toward agentic models (focused on decision-making) and physical AI (in robotics and autonomous vehicles), hardware must handle a variety of workloads efficiently and sustainably. Developing dedicated AI hardware traditionally takes much longer than producing new algorithms, which means software often outpaces hardware availability.
Van den hove explained that simply adding brute compute power worked well for first-generation large language models (LLMs). However, next-gen AI will involve reasoning models with diverse and complex workloads. A one-size-fits-all hardware approach will not suffice.
Agentic AI requires different processing units than physical AI, with some models needing CPUs, others GPUs, and many lacking the right processors altogether. This diversity calls for a flexible hardware solution.
Current Hardware Limitations and Energy Concerns
AI workloads often run on suboptimal hardware architectures, leading to high energy consumption. With the complexity and variety of AI tasks increasing, energy demands are set to rise exponentially.
Adding to the challenge is the unpredictable nature of AI workloads—they can change rapidly with new algorithms. Hardware development cycles, which take years, struggle to keep pace.
The Problem of Stranded Hardware Assets
Custom chips designed for specific AI models risk becoming obsolete quickly due to fast-moving software innovation. This creates stranded hardware assets, a costly issue exacerbated by the growing complexity and expense of chip production.
Only a few companies have the resources to develop custom AI chips. For the broader market, a new approach is needed.
Programmable AI Silicon: A Flexible Solution
Van den hove proposes that silicon hardware must become nearly as programmable as software. Instead of multiple specialized computers, one reconfigurable device could adapt to varied AI workloads by dynamically changing its internal configuration.
This approach means software would define silicon functionality, breaking from traditional rigid hardware design processes. Imagine a processor built from modular supercells—stacked semiconductor layers optimized for different tasks, integrated in 3D to minimize data traffic energy loss. A network-on-chip would coordinate these supercells, reconfiguring them quickly to suit evolving algorithm needs.
By splitting requirements across different chiplets instead of one monolithic chip, hardware from different suppliers could be combined seamlessly.
Benefits and Industry Impact
- Enables more companies to design hardware tailored to specific AI workloads
- Boosts innovation and market differentiation
- Makes hardware development more affordable and agile
- Promotes compatibility and performance through standards like RISC-V
Given AI’s impact across sectors—from drug design to robotics and autonomous driving—the future of AI depends heavily on these hardware innovations.
The Imec Technology Forum (ITF) World 2025 event runs from May 20-21, 2025, in Antwerp, Belgium. For those interested in the latest developments in AI hardware, it offers valuable insights into where the industry is heading.
For professionals seeking to deepen their AI knowledge, Complete AI Training offers up-to-date courses covering AI algorithms, tools, and applications relevant to today’s tech landscape.
Your membership also unlocks:
