Groq reportedly shifts next-gen LPUs from SRAM to 3D-stacked DRAM

NVIDIA-acquired Groq is reportedly planning to use 3D-stacked DRAM in its next-generation language processing units (LPUs), moving away from the SRAM used in its current generation. The reported shift favors memory capacity over speed as models grow.

Groq’s current-generation LPUs pack 500 MB of SRAM and deliver bandwidth of 150 TB/s. Earlier this year, NVIDIA announced Groq LPX racks as part of the Vera Rubin ecosystem, designed to boost inference performance. The racks entered full production in August.

Early results cited in the report show performance of 3,400 tokens per second when Groq LPX racks are paired with NVL72 servers. The report describes the move to 3D-stacked DRAM as a plan for next-generation chips; it does not give a launch date or further specifications for that memory.