Study: AI watermarking can make models comply with harmful requests

Researchers found that applying Google's SynthID watermarking to model outputs changed how models responded to harmful prompts, in some cases causing them to follow instructions they would otherwise refuse. The effect appears linked to how watermarking perturbs the model's output distribution.

Why it matters: Watermarking is widely promoted as a safety and provenance tool, so evidence it can inadvertently weaken safety guardrails is a meaningful complication for anyone deploying it at scale. It's a reminder that safety mechanisms can interact with each other in unexpected ways, worth scrutinizing before watermarking becomes a regulatory requirement.