South Korean researchers solve memory bottleneck in large-scale AI training
The Electronics and Telecommunications Research Institute (ETRI) has developed a technology that addresses one of the fundamental constraints in training large AI models: memory capacity limits. The system, called OmniXtend, uses standard Ethernet to share memory across multiple servers and accelerators, treating distributed resources as a single memory pool.
GPU memory has long been a bottleneck in AI training. As models grow larger and data volumes increase, computational efficiency drops sharply when memory runs out-a problem researchers call the "Memory Wall." Existing solutions rely on high-speed serial interfaces like PCIe, which limit how far apart devices can be and how many can connect together.
OmniXtend works differently. It disaggregates memory resources that were traditionally tied to individual machines and makes them accessible over the network. This allows data centers to dynamically allocate memory capacity without replacing hardware.
How it performs
ETRI demonstrated the system using large language models. In tests where memory capacity was insufficient, inference performance dropped significantly. When the team expanded memory using Ethernet, performance recovered by more than twofold-matching systems with adequate local memory.
The researchers built two key components: an FPGA-based memory expansion node and an Ethernet-based memory transfer engine. In real-world tests, multiple devices formed a shared memory pool and accessed each other's memory in real time.
Because OmniXtend uses conventional Ethernet switches rather than specialized high-speed interfaces, it scales more easily across large data centers. Memory can be expanded without replacing servers, reducing deployment and operational costs.
Next steps and adoption
ETRI presented the technology at RISC-V Summit Europe in May 2025 and RISC-V Summit North America 2025. The institute is leading the Interconnect Working Group under the CHIPS Alliance of the Linux Foundation, working to establish open-source standards for AI networking and memory expansion.
The team plans to commercialize OmniXtend by partnering with data center hardware and software companies. Potential applications include AI training servers, memory expansion devices, and network switches.
Future work will extend the technology to high-reliability embedded systems like automotive and maritime applications, and expand it to work across different accelerator types including neural processing units, GPUs, and CPUs.
Kim Kang Ho, Assistant Vice President of the Future Computing Research Division at ETRI, said the institute will "significantly expand research on memory interconnect technologies centered on neural processing units and accelerators" and "continue to advance the technology and strengthen international collaboration to ensure its adoption in next-generation systems."
The research was funded by South Korea's Ministry of Science and ICT and the Institute of Information and Communications Technology Planning and Evaluation under the "Research on Memory-Centric Next-Generation Computing System Architecture" project.
For researchers working on Generative AI and LLM infrastructure, this addresses a practical constraint that affects model scaling decisions. The work also reflects broader efforts in AI for Science & Research to build more efficient computing systems.
Your membership also unlocks:
