NVIDIA and Princeton Unveil ReplaySSM to Accelerate Hybrid-Model Inference

NVIDIA and Princeton University's Dao AI Lab published ReplaySSM on October 12, 2026, a technique that addresses memory-bandwidth bottlenecks in hybrid language models by caching recent inputs instead of storing full recurrent state at every decoding step. The authors report kernel speedups of 1.46× to 1.80× and whole-decode-step speedups of 1.20× to 1.47× across four models at batch size 256, according to the paper ReplaySSM: Cache SSM Inputs, Not State available on arXiv and the NVIDIA Developer Blog post by Michelle Horton.