OpenAI's textGrain watermark is meant to tag AI-generated text for the EU's AI Act, but the company's own numbers show it's surprisingly fragile. Swap just 25% of the words for synonyms, and detection crashes from 92% to 17%.
What happened
OpenAI just launched textGrain, a watermark designed to tag AI-written text so it can comply with Article 50(2) of the EU AI Act. That rule is about "provenance" — basically, a way to prove where a piece of text came from. The watermark is an invisible statistical pattern woven into word choices. Only OpenAI's secret key can detect it. Think of it like a secret handshake hidden in the rhythm of a sentence. You can't see it, but a machine with the right decoder can.
The unusual part? OpenAI published the failure curve right at launch. That's rare. Most companies hide how easily their security can be broken. Here's what the curve shows: at baseline, detection is 92%. Swap 10% of the words for synonyms, and detection falls to 66%. Swap 25%, and it crashes to 17%. Math and short passages barely watermark at all.
To make this concrete, a developer built an interactive explainer with an "attack lab." You can take a watermarked passage, swap words for synonyms, run a translation round-trip (translate to another language and back), or switch to math-like text — and watch the detector's p-value collapse in real time. A p-value is just a confidence score: the lower it is, the less sure the detector is that the watermark is there.
OpenAI's own documentation says it plainly: "the absence of a detected watermark does not prove human authorship." That's the sentence that matters most.
Why this matters more than a tech demo
This isn't just a nerdy experiment. The EU AI Act is trying to make sure people know when text is AI-generated. Watermarking is one proposed method. If a simple synonym swap can drop detection from 92% to 17%, then watermarking alone can't carry the weight of a regulation. And OpenAI knows it — that's why they published the curve.
What it means for you
You don't need to be a coder to care. This affects anyone who writes, reads, or gets accused of using AI. Let's walk through a few everyday scenarios.
At home: your emails and messages
Say you use ChatGPT to draft a birthday speech or a tricky email to your landlord. That text might carry a watermark. But if you tweak a few words — swap "happy" for "glad," change "important" to "key" — the watermark can vanish. So if someone runs a detector on your email and says "no AI watermark," that doesn't mean you wrote it yourself. It just means the watermark isn't detectable anymore. On the flip side, if you wrote it yourself, there's no watermark to begin with. The detector's silence tells you almost nothing.
At work: content and communications
If you're a content writer or a communications manager, clients might start asking for "watermark-free" AI text. Now you know it's easy to do: swap 25% of words and detection drops to 17%. But that also means you shouldn't trust a client who says "our detector proves this is human-written." The absence of a watermark is not proof. For teams using AI to draft internal docs, the watermark might be present, but it's fragile. You can't rely on it for compliance.
In business: EU AI Act compliance
If your company sells into the EU, Article 50(2) might affect you. The rule asks for provenance — a way to show where text came from. OpenAI's textGrain is one attempt. But the failure curve shows it's easily defeated. So if your legal team is planning to rely on watermarks alone, you'll need a backup. Maybe human review, or a different provenance method. The upside: OpenAI's transparency gives you real data to work with.
In study: teachers and students
Teachers are already using AI detectors. Some students get falsely accused. This news makes that worse: a missing watermark doesn't prove a student wrote the essay. And a present watermark doesn't prove they didn't — they could have used AI and then swapped words. So detectors are unreliable in both directions. The honest takeaway: don't treat a watermark check as a verdict. It's a weak signal.
For creativity and income: freelancers and creators
If you're a freelance writer, you might get accused of using AI. This news gives you a defense: watermarks are easily removed, so absence of a watermark means nothing. But it also means if you do use AI to draft, you can hide it. That's an ethical gray area. For creators selling AI-assisted content, the market might demand disclosure. Watermarks won't enforce that.
And if you're looking for more free AI tools to experiment with, MyKreaTool has a solid collection. It's a handy companion when you want to test how AI text behaves.
How to try it right now
You can see this for yourself in just a few clicks. The tool is free and runs in your browser. Here's the step-by-step.
1. Open the interactive explainer. Go to Movahedi's OpenAI textGrain explainer and attack lab. It's free, no signup required. There's also a French version linked inside.
2. Find the watermarked sample passage. The page starts with a short intro and gives you a passage that has OpenAI's textGrain watermark.
3. Swap 10% of the words. Use the attack lab's synonym swap tool. Click to replace 10% of the words with synonyms. Watch the detection score drop from 92% to 66%.
4. Swap 25% instead. Reset and try 25%. The score falls to 17% in about 10 seconds.
5. Run a translation round-trip. This is fun: translate the passage into another language, then back to English. The watermark often breaks.
6. Switch to math-like text. Replace the passage with formulas and numbers. The detector struggles because math doesn't have the same word-choice patterns.
7. Watch the p-value. The explainer shows a live p-value. It's a confidence score. As you attack the text, the p-value collapses. That means the detector is losing confidence that the watermark is there.
8. Read OpenAI's original post. For the official details, check OpenAI's original post. It explains the EU AI Act context and the failure curve.
That's it. You don't need to install anything. Just play with the lab and see the numbers for yourself.
Upsides and what changes
The biggest upside is transparency. OpenAI published the failure curve at launch. That's not normal. Most companies hide their weaknesses. By sharing the numbers, OpenAI gives everyone — regulators, developers, writers — a realistic picture. You can plan around it.
What changes? First, watermarking alone won't satisfy the EU AI Act. Regulators will need to combine it with other methods, like metadata or human review. Second, people will stop treating "no watermark" as proof of human authorship. That's a good thing. It reduces false accusations. Third, content creators get a clearer picture: if you use AI, you can easily remove the watermark. That's an ethical choice, not a technical barrier.
The bad news? Bad actors can also remove watermarks. So the watermark won't stop someone who wants to pass off AI text as human. It's a speed bump, not a wall.
Limitations
Here's the honest part. The watermark only works if you have OpenAI's secret key. You can't check it yourself. It fails on short text and math. A simple synonym swap or a translation round-trip can erase it. And the absence of a watermark doesn't prove human authorship — that's OpenAI's own warning. So as a forensic tool, it's weak. It might help with casual provenance, but it can't stand up to someone who wants to remove it. Also, the interactive explainer is a demo; real-world text might behave differently depending on length and style. Don't treat it as a guarantee either way.
Conclusion
OpenAI's textGrain watermark drops from 92% detection to just 17% after a 25% synonym swap. Math and short passages barely register. And OpenAI itself says a missing watermark doesn't prove human authorship.
Your one action today: the next time someone tells you an AI-authorship detector "proved" something, remember that number — 17%. Or better yet, go try the attack lab yourself at Movahedi's explainer. It's quick, and you'll never trust a watermark check the same way again.
Comments 0