SOCAMM2 Targets AI Servers With HBM-like Bandwidth at Lower Cost and Energy Use

SOCAMM2: High-bandwidth, low-power memory for AI servers

NVIDIA has paused promotion of its first SOCAMM module and shifted focus to SOCAMM2, working with Samsung Electronics, SK Hynix, and Micron on sample testing. Earlier deployment targets of 600,000-800,000 units were shelved after technical setbacks. The bet now is on a more capable, more flexible successor.

For product teams, the appeal is clear: near-HBM bandwidth targets at lower cost and power, plus the flexibility of a field-replaceable module. It complements HBM rather than replaces it-HBM stays on-package for peak throughput; SOCAMM2 adds scalable system memory without locking you into the motherboard.

What SOCAMM2 is

SOCAMM is a CAMM-based, LPDRAM memory module for AI servers. It aims to deliver HBM-like throughput targets with lower cost and power by pairing LPDDR-class chips with a serviceable module form factor. Compared with DDR5 RDIMMs, it claims a smaller footprint and roughly one-third lower power use.

Documentation for NVIDIA's GB300 NVL72 lists support for up to 18 TB of SOCAMM based on LPDDR5X with an aggregate bandwidth of up to 14.3 TB/s. Due to motherboard changes in "Blackwell Ultra," commercialization was delayed, but the approach is still expected to appear with the Vera Rubin platform as a non-onboard memory option for the Vera CPU.

Key technical highlights

• Interface stability: SOCAMM2 retains the 694 I/O architecture from SOCAMM1, simplifying evolution for controllers and boards.
• Higher speed: Data rate rises from 8,533 MT/s to 9,600 MT/s (~12.5% uplift) for stronger throughput on AI workloads.
• LPDDR6 potential: SOCAMM2 is expected to support LPDDR6, pending verification and ecosystem readiness.

Why LPDDR6 matters

LPDDR6 shifts to a 24-bit channel (two 12-bit sub-channels) with independent 4-bit CA buses. Burst length moves to BL24, producing a 288-bit packet: 256 bits for data and 32 bits for error handling, memory tags, or DBI. Bandwidth per device targets 38.4 GB/s with about 20% lower power versus LPDDR5.

It adds Efficiency Mode so one sub-channel can access all 16 banks while the other sleeps under light load. A new ALERT line allows DRAM to actively report 15 types of errors to the processor, improving integrity and serviceability. For more on the standard, see JEDEC LPDDR6.

Reality check: LPDDR6 is still in technical verification. Compatibility, cost, and stability at scale remain open items, which is why vendors haven't formally locked it in for SOCAMM2 yet.

Integrating SOCAMM2 in your server roadmap

Hardware implications

• Motherboard layout: Trace length, signal integrity, and module service access need revisiting versus DDR5 RDIMMs. Plan for a different z-height envelope and keepouts.
• Memory controller: Confirm SOCAMM2 channel mapping, timing, and training sequences in silicon. Older controllers may not support the speed and features.
• Thermals: LPDDR-based modules reduce power but concentrate heat. Validate cooling paths, airflow direction, and liquid-cooling block fit.
• Serviceability: Vera Rubin's non-onboard design target fits SOCAMM2's field-replaceable approach, helping reduce maintenance windows and upgrade costs.

Software and firmware

• Drivers: Enable multi-channel parallelism and tune page policies to reduce contention under mixed training/inference loads.
• Schedulers and frameworks: Calibrate prefetching and batch sizing in TensorFlow/PyTorch to align with SOCAMM2 bandwidth and latency profiles.
• Telemetry: Expose error signals (including LPDDR6 ALERT when available) to system health services for proactive remediation.

Chip compatibility

NVIDIA's GB300 series and AMD's MI300 series are referenced as targets for adaptation. For GB300 NVL72, published specs indicate up to 18 TB SOCAMM (LPDDR5X) and 14.3 TB/s aggregate bandwidth, the result of chip-memory co-design to reduce loss and crosstalk. Expect similar co-validation requirements for SOCAMM2 designs.

Vendor landscape and supply

• Micron: Early mover; announced delivery of LPDDR5X-based SOCAMM modules to customers. Advantage depends on iteration speed and cost curve.
• Samsung and SK Hynix: Advancing SOCAMM2 with emphasis on thermal reliability (Samsung) and speed/latency tuning (SK Hynix). Both engaged in NVIDIA sample tests; aiming for mass production as early as next year.
• Domestic players (e.g., Foresee): Introducing SOCAMM2 variants optimized for local server environments, including reduced module height by removing the LPCAMM2 top trapezoid to improve chassis and liquid-cooling compatibility.

Commercial outlook

Challenges: Controller compatibility, OS/framework tuning, initial cost premiums (especially with LPDDR6), and yield learning curves. If software lags hardware, performance benefits go unused.

Demand tailwinds: TrendForce projects the AI server sub-market to reach $298B in 2025, with AI servers accounting for 70%+ of total server value. Memory demand grows in lockstep. See TrendForce.

Positioning: SOCAMM2 sits between DDR5 and HBM-better bandwidth and efficiency than DDR5 at a lower cost and easier scaling than HBM. Fits mid-to-high-end AI training, inference, and cloud nodes that need large, flexible system memory.

What product leaders should do now

• Pick the lane: Map workloads to memory tiers-HBM for on-package compute; SOCAMM2 for scalable system bandwidth; DDR5 for general purpose.
• Prototype early: Secure SOCAMM2 samples from two vendors to de-risk supply and compare thermals, timing margins, and RAS features.
• Controller verification: Run SI/PI models at 9,600 MT/s; validate training sequences, margining, and error handling at temperature corners.
• Thermal design: Test with worst-case concurrency. Confirm airflow/liquid interfaces and keep module service access clear.
• Software tuning: Align driver policies, NUMA placement, and framework prefetch/batch sizes with observed bandwidth and latency.
• Cost model: Build BOM scenarios for LPDDR5X and LPDDR6 variants; include yield ramps and service cost reductions from field-replaceable modules.
• Validation plan: Add long-soak, mixed-workload tests; enable telemetry for early fault detection (including ALERT when LPDDR6 lands).
• Sourcing: Lock multi-vendor options (Micron, Samsung, SK Hynix; plus domestic where relevant). Track LPDDR6 readiness before committing.

Bottom line

SOCAMM2 gives AI servers a practical path to higher bandwidth and lower power without the lock-in and cost profile of HBM. The value shows up only if hardware, firmware, and software are tuned together-and if you validate early with at least two suppliers.

If your next revision targets GB300-class systems or Vera Rubin, start board and driver planning now. Keep an eye on LPDDR6 maturity; adopt when the cost and stability cross your thresholds.

Upskill your team

Need a fast way to align product, hardware, and ML teams on memory-aware AI system design? Explore role-based programs at Complete AI Training.