This news article is an AI-generated translation from Dutch to English.
Anthropic recently announced that every text the chatbot Claude generates will now carry an invisible watermark. The measure is meant to bring transparency to our world full of AI-generated content. But how robust is the technology?
Since Anthropic announced the measure, a few questions keep coming up. Does this actually work? Is it a good idea? And perhaps the most frequently asked: can the watermark be bypassed?
To understand how a watermark invisible to the naked eye works in AI text, you first need to know how a language model builds sentences. "A language model predicts one word — or actually one token — at a time, based on the preceding text, and works with probability calculations," explains Paul Van Eecke, professor of computational linguistics at the VUB. "For the word 'dog,' the model would, for example, assign a high probability to 'barks' and a negligibly small probability to 'computer.' Importantly, the model doesn't always pick the most probable word, because that would lead to dull, predictable text. Instead, a word is chosen from the probability distribution each time, where more probable words also have a greater chance of actually being selected."
It's precisely in that choice that a watermark intervenes. When several candidate words are roughly equally probable, a model normally chooses randomly between those options. In a watermarked model, that choice isn't truly random, but happens through what Van Eecke calls a "pseudo-random process": to the user it looks like chance, but in reality the outcome is fixed. "If you have that particular key, you can actually predict the assignment of words," he says.
The dice that isn't dice
To clarify that principle, Van Eecke reaches for an analogy with the board game Monopoly. "Monopoly has one random element, namely the dice: each turn you roll the dice. You could also play Monopoly without dice, but with, say, a button that you press, and that button gives you the next decimal of pi each time." For the players themselves, nothing changes: the game still works, and no one notices the difference between an actual roll and a digit from pi. "But if you know there's a code, and you know that code, and you know pi, then you can see which games were played using the pi-button and which weren't." Translated to language models: at every point where there's ambiguity between multiple words, the choice is subtly steered by a hidden, model-specific key. "Whoever holds that key can, after the fact, determine with very high certainty from a sufficiently long text whether it was written by that specific model," says Van Eecke.
Ryan Ott, machine learning engineer at the Belgian AI company ML6, recognizes that mechanism and adds another image often used in the technical literature: "It's often explained using a green list and a red list. In a non-watermarked text, you'd expect roughly an equal split between the two lists. In a watermarked text, that balance shifts slightly, without a reader noticing." Anthropic didn't invent this technique. It builds on SynthID, a method Google DeepMind already published in a research paper in 2024. "A few years ago I attended a presentation at a conference where Google DeepMind introduced this way of watermarking," says Van Eecke. Ott confirms this. "Anthropic is the first to roll out the technique broadly and publicly across all its output, with explicit reference to the European AI Act as the motivation," says Ott.
Why now, and why Anthropic?
That watermarking is suddenly being discussed everywhere initially surprised Ott. "The discussion in the media grew quickly, but I do think it's logical in a sense." According to him, the explanation lies in the interplay of two factors: the hard deadline imposed by the AI Act, and the fact that Anthropic is one of the few players rolling out the technique across all its output right away.
The AI Act does indeed require AI providers to ensure that all output from their chatbots — so video, photo, audio, and text as well — can be detected as artificially generated or manipulated in a "machine-readable format," as Article 50 of the AI Act literally stipulates. Ultimately, all AI providers will have no choice but to build in watermarking, so that they can demonstrate with their own detection software that images or texts were generated or manipulated by their AI software. The AI Act does provide an exception for, for example, an "assistive function for standard editing": think of a spelling or grammar check. In such a case, Europe doesn't require detection to be possible.
Anthropic is the first to communicate so openly about how it will deploy watermarking and to roll it out globally. This is in contrast to Google, which did develop SynthID but doesn't actively deploy it everywhere, and OpenAI, which is still holding off.
AI providers benefit themselves as well
Still, according to both Van Eecke and Ott, other interests are also at play that have little to do with regulation. AI companies are gradually facing a shortage of fresh, non-AI-generated training data. Training on their own output only leads to a loss of quality. "So these companies have a very strong incentive to be able to recognize what they themselves have generated, so that they don't accidentally end up training on their own texts," says Van Eecke. Ott confirms this: "I think for them it might actually be a very pleasant side effect. I assume we're close to the point where fresh text is running out, if we haven't reached it already." Miguel Degeneffe of the Hasselt-based AI services company AI-D goes even further and calls it the main reason: "The internet is full of AI-generated content. With that huge amount of AI junk on the internet, also known as AI slop, they can't keep training."