I measured what watermarking actually does to AI writing. The answer isn’t what either side thinks.

Anthropic started watermarking Claude’s text output in August and extended it to additional older models on November 3rd per EU AI Act compliance. SynthID-Text. No opt-out.

Some users claim watermarking degrades output quality. Anthropic says zero impact. I think we’re somewhere in between, and I have the data to show it.

I run RVI (rvi.run), a tool that measures linguistic variance in text. Sentence length distribution, vocabulary diversity, and structural patterns. THIS IS NOT an AI detector, it’s more like a thermometer. It doesn’t care who wrote the text, it just measures how much variety is present.

In August, before watermarking, I ran 100 structured prompts (10 genres × 10 each) across five Claude models via API. Clean baseline. On November 5th I ran the same prompts against seven models, including ones I had baselines for.

Sonnet 4.6, the cleanest before-and-after:
Aggregate RVI: 59.0 (August) -> 59.43 (November). Median: 59 -> 59. No change.

Anthropic’s claim holds… On the surface.

But broken out by category:
• Scientific review: −3.3 points
• Business writing: −2.4
• Narrative fiction: +6.2
• Personal essay: +2.6

The November outputs showed less variety in factual writing and more in creative writing. The aggregate looked flat because the effects canceled out.

This makes mechanical sense. SynthID-Text biases word choices using a keyed random pattern. In factual text there’s often only one right word… the watermark constrains an already narrow field. In fiction, where “overcast,” “grey,” “leaden,” and “dim” are all valid, the keyed randomness adds variation on top of the model’s natural distribution.

The watermark isn’t neutral. Aggregate metrics completely miss the story.

If you’re generating technical documentation or research, watermarking measurably constrains your output variety. Not by a lot, but not zero. If you’re writing fiction or personal essays, it may actually be adding variety.

The dataset is 1,100 scored essays across 11 Claude models and growing. It’s all measured through RVI at rvi.run. I’ll be adding analysis results to the site soon along with other models.

Fun fact, Adam Smiths’ “Wealth of Nations” is the lowest scoring document in my human test set at a score of 33… not a lot of information in that work it seems.