November 21, 2026 · The Decoder
Nvidia researchers say huge context windows are often unnecessary
Nvidia researchers argue language models rarely need their full context windows, which are slow, expensive and error-prone. In tests, periodic summaries plus targeted keyword search usually matched full-context results at a fraction of the cost. They also propose a model class that works from short notes instead of long context.
Why it matters: Labs have competed on ever-larger context windows, but this suggests memory management, not raw window size, drives agent quality and cost. It lines up with Meta FAIR's learned-memory work and Gemini's Tool Retrieval: the field is shifting toward retrieving only what is needed. Coming from the leading GPU vendor, it also signals that inference efficiency matters as much as brute-force compute.