AI watermarking may make LLM safety behavior unpredictable

AI watermarking may unintentionally change how large language models behave, according to Lasso research reported by TechRadar.

The researchers examined Google DeepMind’s SynthID-Text, a system designed to embed a machine-readable indicator showing whether text was generated by AI or written by a person. They found that the watermarking procedure could affect both what a model says and what an AI agent does, a side effect Lasso calls “sampling drift.”

The research found that watermarking changed some models’ refusal decisions even without an attack, making them more willing to answer potentially harmful prompts. When combined with prompt injection, the consequences became more pronounced. The researchers also observed changes in models’ susceptibility to prompt injection and in the tools AI agents selected.

Anthropic has announced that future generations of Claude will use AI watermarking similar to Google DeepMind’s, citing adherence to the EU AI Act as one motivation. Wider adoption could make unintended effects more common, the report says.

Lasso urges developers to rerun benchmarks, safety evaluations and other tests whenever watermarking is introduced or its configuration or key changes. The researchers say the findings are not an argument against watermarking for provenance, but show that its effects should be reassessed rather than applied blindly.