Stanford: 44% of widely cited LLM research claims have already expired

Stanford researchers tested 50 widely cited claims about LLMs against 20 current models. 44 percent of the findings are now expired and another 34 percent have weakened significantly, with an average lifespan of 15.7 months. Claims about what models cannot do expired fastest.

Why it matters: Policy, safety and product decisions often rest on papers about older models, so this suggests they may be built on stale evidence. It also warns against treating limitation claims as durable, which matters for regulators and for anyone forecasting capabilities.