You paste a name into a form and it is rejected. You search a document for a phrase you can see on the screen and get no result. Two spreadsheet cells look identical and refuse to match. In almost every case the cause is the same: the text contains characters you cannot see.
The two usual suspects
Non-breaking spaces (U+00A0). Word processors and web pages use them to stop a line breaking in an awkward place — between a number and its unit, for example. They look exactly like an ordinary space and are a completely different character. So John Smith typed by hand and John Smith pasted from a web page are two different strings as far as search, sorting and code are concerned.
Zero-width characters (U+200B and friends). These have no width at all. They come from chat apps, content management systems, and text that has been through a rich-text editor. A code or a username that looks perfect fails validation because there is an extra character in it that nothing renders.
There are others — soft hyphens, byte order marks, directional marks left over from bidirectional text — but those two account for most of the damage.
How to tell
You cannot, by looking. That is the whole problem, and it is why "just retype it" is the advice people eventually fall back on.
What you can do is count. Paste the text into the character counter: if the count is higher than the characters you can see, something is hiding in there. To find out exactly what, paste it into the invisible character detector, which names every hidden character it finds and how many times each appears.
The fix
The text cleaner removes all of them in one pass and tells you what it removed:
- Paste the text.
- Leave Fix non-breaking spaces and Remove invisible characters ticked.
- Read the report — it lists each fix and how many times it fired.
- Copy the result, or send it straight to another tool.
The report is the part worth paying attention to. "47 non-breaking spaces replaced" tells you the text came out of a web page or a word processor, which usually means the next export from the same source will need the same treatment.
One important exception
Not every invisible character is junk. The zero-width non-joiner (U+200C) and zero-width joiner (U+200D) decide how letters connect in Persian, Arabic, Hindi and several other scripts. Removing them from text in those languages is not a cleanup — it changes how words are written, and native readers will see the damage immediately.
For that reason the cleaner leaves joiners alone by default. They are removed only if you tick the option that names those scripts explicitly. If you are cleaning text you cannot read, leave that option off.
Stopping it at the source
- Paste as plain text —
Ctrl+Shift+VorCmd+Shift+Vin most applications — drops the formatting and most of the exotic characters with it. - Export as CSV rather than copying a column when you are pulling data out of a spreadsheet.
- Clean once, at the point of import, rather than chasing the same characters through every downstream system.
Related tools
- Text cleaner — fix spaces, blank lines and invisible characters, with a report
- Invisible character detector — name and count what is hiding in your text
- Character counter — the quickest way to notice something is there