Meta's LLM-written tests caught 74 privacy incidents before code shipped
Meta has moved Catching JiTTests into production, a system where a large language model (LLM) writes 'just-in-time' tests designed to fail on risky code changes. In the first half of 2026 it caught 74 privacy severe incidents (SEVs) before the code landed. Engineers reported a 60% true-positive rate on its flags.
Why it matters: Most AI coding coverage focuses on writing code faster; this is a measured result on the other side — using models to catch what humans and AI-written diffs get wrong. A 60% true-positive rate is high enough that engineers keep paying attention, which is the usual failure point for automated review tools. As more production code is AI-generated, targeted, diff-specific testing like this is likely to become a standard safety layer.