NVIDIA Releases Dynamo 1.0, Inference Software Adopted Across Cloud Providers and Enterprises
NVIDIA released Dynamo 1.0 on March 16, open source software designed to manage AI inference workloads across data centers. The software integrates with major cloud providers including Amazon Web Services, Microsoft Azure, Google Cloud and Oracle Cloud Infrastructure, along with adoption from enterprises like ByteDance, PayPal and Pinterest.
Dynamo functions as a distributed operating system for what NVIDIA calls "AI factories" - data centers running inference at scale. It coordinates GPU and memory resources across clusters to handle requests of varying sizes and priorities arriving unpredictably.
Performance Gains and Technical Approach
In industry benchmarks, Dynamo boosted inference performance on NVIDIA Blackwell GPUs by up to 7x, according to the company. The software reduces token costs by managing memory more efficiently and routing requests to GPUs that already contain relevant data from earlier processing steps.
The system adds traffic control mechanisms and moves data between GPUs and lower-cost storage to reduce wasted computation. For long-context requests common in agentic AI systems, Dynamo can offload temporary memory when it's no longer needed.
Ecosystem Integration
NVIDIA integrated Dynamo with popular open source frameworks including LangChain, SGLang and vLLM. The company also released standalone modules like KVBM for memory management and NIXL for GPU-to-GPU data movement.
Cloud infrastructure providers CoreWeave, Nebius and Together AI said the software reduces deployment complexity. Chen Goldberg, executive vice president at CoreWeave, said supporting Dynamo allows the company to offer "a more seamless, resilient environment for deploying complex AI agents."
Who's Using It
Adoption spans multiple categories. AI-native companies Cursor and Perplexity use the platform. Inference endpoint providers including Baseten, Deep Infra and Fireworks integrated it. Global enterprises including AstraZeneca, BlackRock, Coupang, Instacart, Meituan, Shopee and SoftBank Corp. deployed it.
Pinterest said in a statement that Dynamo optimization helps the company "expand the seamless and personalized experiences we deliver, powered by high-performance AI infrastructure" to hundreds of millions of users.
Dynamo 1.0 is available now as free, open source software. Operations teams managing AI infrastructure should consider how inference orchestration affects GPU utilization rates and token costs. Learn more about AI for Operations and explore the AI Learning Path for Operations Managers to understand how these systems integrate with broader operational strategy.
Your membership also unlocks:
