- Micron Introduces Dense 256GB LPDDR5x Module Directly for AI Servers
- Eight SOCAMM2 modules can increase server memory capacity up to 2TB
- AI inference workloads increasingly shift performance bottlenecks to system memory capacity
Modern large language models (LLMs) and inference pipelines increasingly demand huge memory pools, forcing hardware vendors to rethink server memory architecture.
Micron has now introduced a 256GB SOCAMM2 memory module aimed at data center systems where capacity, bandwidth and power efficiency all influence overall performance.
The module leverages 64 monolithic 32GB LPDDR5x chips, forming a dense LPDRAM package that meets the growing memory footprint required by contemporary AI workloads.
Article continues below
Scaling Memory Capacity for AI Server Platforms
The module also increases the maximum available memory per processor configuration. When eight of these SOCAMM2 modules are installed in an eight-channel server processor, the total capacity can reach 2TB of LPDRAM.
This figure exceeds the previous generation of 192 GB modules by approximately a third, allowing systems to support larger pop-ups and more demanding inference tasks.
Micron describes the SOCAMM2 design as more efficient than conventional server memory modules.
“Micron’s 256GB SOCAMM2 offering provides the most power-efficient CPU-attached memory solution for AI and HPC,” said Raj Narasimhan, senior vice president and general manager of Micron’s Cloud Memory business unit.
“Our continued leadership in low-power memory solutions for data center applications has uniquely positioned us to be the first to offer a monolithic 32 GB LPDRAM chip, helping to drive industry adoption of higher-capacity, more power-efficient system architectures.
The company says the new LPDRAM module consumes about a third of the power of comparable RDIMMs while taking up only a third of the physical footprint.
Its reduced power demand and smaller module size enable higher rack density in large data center deployments, while lower memory power reduces thermal load and infrastructure costs.
The SOCAMM2 architecture also follows a modular design intended to simplify maintenance and future expansion.
The form factor supports liquid-cooled server systems and can accommodate additional capacity as memory requirements increase along with model complexity and dataset scale.
Micron states that the 256 GB SOCAMM2 module can influence certain inference operations in unified memory architectures.
The company reports a more than 2.3x improvement in time to first token during long-context inference when the module is used for key-value cache offloading.
In standalone CPU workloads, the LPDRAM configuration is said to deliver more than 3x performance per watt compared to traditional server memory modules.
The company’s LPDRAM portfolio includes components from 8 GB to 64 GB and SOCAMM2 modules ranging from 48 GB to 256 GB, and customer samples of the 256 GB SOCAMM2 module are already shipping.
Follow TechRadar on Google News And add us as your favorite source to get our news, reviews and expert opinions in your feeds. Make sure to click the Follow button!
And of course you can too follow TechRadar on TikTok for news, reviews, unboxings in video form and receive regular updates from us on WhatsApp Also.
