How do I remove invisible zero-width characters from text I copied from a website?
In the whitespace cleaner, turn on 'Remove zero-width separators (ZWSP, WJ, BOM)'. It deletes the zero-width space, the word joiner and the byte order mark: characters you cannot see that stop codes and names from matching. The rule is off by default, so these characters are only removed when you ask for it.
How to tell your text has them
The usual sign: two codes or names look identical, yet the Duplicate-line remover does not treat them as repeats, search does not match them, and the difference survives even with whitespace ignored. These characters are not spaces, which is why no whitespace option catches them.
To actually see them, turn on 'Show invisible characters in the preview' in the cleaner. A zero-width space appears as [ZWSP], a word joiner as [WJ], and a byte order mark as [BOM].
A worked example
Two lines copied from a product page: SKU-1001 with a zero-width space at the end, then SKU-1002.
With the default rules: nothing to clean, and a message says the text is the same as the input.
With the zero-width rule on: one change, and the first line becomes a clean SKU-1001.
In a text with all three characters, a BOM at the start, a zero-width space and a word joiner, the counter reaches 3.
Here is the practical effect: a two-line list, SKU-1001 with the hidden character and then a plain SKU-1001, has nothing removed by the duplicate remover. After cleaning, it removes one line as you would expect.
Steps
- Open the Whitespace cleaner and paste your text.
- Tick 'Remove zero-width separators (ZWSP, WJ, BOM)' and watch the counter beside it.
- Check the preview, then copy the result or press 'Apply to input'.
Why it is off by default
Because the zero-width space has a legitimate job: it marks where a line may break in languages that do not separate words with spaces, such as Thai. In Arabic or English text copied from a website it is usually useless, but the tool does not make that assumption for you.
And why ZWNJ and ZWJ have a separate rule
These two look like part of the same family, but they carry meaning. The zero-width non-joiner splits parts of a Persian word without a space, and the zero-width joiner controls how letters connect and glues combined emoji together. We took the Persian word میخواهم next to a code containing a zero-width space and turned on only the zero-width rule: one character was removed from the code, and the Persian word stayed intact. The ZWNJ and ZWJ rule exists but is off, and turning it on shows a warning.
Common questions
Does the rule remove direction marks such as RLM?
No, the tool never touches direction marks.
Does it remove ordinary spaces?
No, this rule removes those three characters only. Spaces have their own rules.
Is my text sent to a server?
No, everything runs inside your browser.