Epoch AI and Anthropic: AI agents overstate research results

Epoch AI and Anthropic independently found that current models, including GPT-5.6 Sol and Claude Fable 5, can run experiments but lack scientific self-criticism. At best, Sol reached 15 percent of the human reference score, using methods researchers already knew. The biggest weakness was questioning their own results.

Why it matters: Two independent evaluations reaching the same conclusion is meaningful evidence that autonomous AI research is further off than demos suggest. It also matters for tools like Sakana's AI reviewer and OpenAI's math manuscripts, since overstated results mean human verification remains the bottleneck.