A KAIST research team has published the first systematic analysis of the computational and energy costs of AI agents, finding that these autonomous systems consume up to 136.5 times more energy per query than conventional generative AI. The study, led by Professor Minsoo Rhu of the School of Electrical Engineering, quantifies real-world response latency, GPU idle time, and data-center-scale power demand, signaling that AI infrastructure efficiency is now as critical as model capability.
How AI agents differ from conventional LLMs
AI agents extend beyond simple question answering. They plan, use external tools like web search and code execution, and coordinate multi-step reasoning on their own. AI Agents & Automation of this kind require far more large language model (LLM) invocations than chain-of-thought reasoning alone. Each tool interaction triggers additional calls to the model, which sharply increases computational load.
The team found that response time can rise by up to 153.7 times compared to simple CoT tasks. During execution, GPUs remain idle for as much as 54.5 percent of the total time while external tools run their processes. This idle window means expensive hardware isn't being fully used, creating a new form of inefficiency as AI systems tackle more complex work.
The data-center power equation
Using a 70-billion-parameter LLM comparable to current commercial services, the study measured an average consumption of 348.41 watt-hours per query. That's 136.5 times higher than a basic generative AI query. To put that in perspective, the researchers modeled a scenario where AI agent requests reach 13.7 billion per day-equivalent to Google search traffic. In that scenario, data-center power demand would hit roughly 198.9 gigawatts, about half the average power consumption of the United States.
"This study is the first to quantitatively show not only how AI is becoming more intelligent, but also how much electricity and cost are required to implement and sustain that intelligence," Professor Rhu said. Existing AI data centers under construction operate in the single-digit gigawatt range, making the projected demand a significant infrastructure challenge.
Co-design is the next frontier
The findings shift the competitive focus from building smarter models to building more efficient systems. The researchers argue that future progress depends on jointly optimizing AI semiconductors, data-center design, and power infrastructure. This co-design approach aims to reduce operating costs while making AI services sustainable.
"Research and investment in this direction will be essential to dramatically reduce the cost for end users to access AI services while building sustainable AI infrastructure," Rhu said. The study was presented in February at the 32nd IEEE International Symposium on High-Performance Computer Architecture (HPCA), with KAIST PhD student Jiin Kim as first author. The team has released the AI agent implementations and benchmarks as open source.
Why this matters for IT and development
The data shows that AI for IT & Development operations must now account for energy consumption at the level of physical infrastructure, not just software engineering. For teams deploying AI agents in software development, research, or workplace automation, idle GPU time and soaring latency directly affect cost and system design choices. The shift toward efficiency-first AI infrastructure means IT architects will need to evaluate agent workflows not only by what they accomplish, but by how much power they draw per task.
Your membership also unlocks:
