A single non-breaking space character looks identical to a regular space but breaks word-wrapping, breaks string comparisons, and causes form submissions to pass "not empty" validation on fields that look empty to the user
The previous articles on this site covered invisible Unicode characters and security risks, whitespace as syntax in Python/YAML, and trailing whitespace in Git diffs. This article addresses the specific invisible characters that cause the most real-world bugs — not as security exploits (the previous article) but as accidental data quality issues introduced by copy-pasting from PDFs, Word documents, and web pages.
The invisible character taxonomy: not all whitespace is equal
Unicode defines dozens of whitespace and whitespace-like characters, most of which look like regular spaces but behave differently:
Regular space (U+0020): the standard spacebar space. Breaks words for line-wrapping, trims with standard .trim(), matches \s in regex.
Non-breaking space (U+00A0, in HTML): visually identical to a regular space, but prevents a line break at that point. Commonly used in HTML for formatting (between a number and its unit: "100 km" to prevent line breaks splitting "100" from "km"). When copy-pasted from a web page into a form or database, it arrives as U+00A0 — which:
- Most string
.trim()implementations don't remove (unlike regular spaces) - Most string comparison
===treats as different from U+0020 - Passes a simple
field != ""validation check on a "blank" field containing only
Narrow no-break space (U+202F): used in some European number formatting (French-style thousands separator: 1 000 000). Copy-pasting French-formatted numbers from documents or government websites introduces this character between digit groups.
Zero-width space (U+200B): has zero visible width but is present in the string. Common source of "my string comparison says these are different but they look the same" bugs. Frequently introduced by some CMS systems, chat platforms, and rich-text editors.
Zero-width non-joiner (U+200C) and zero-width joiner (U+200D): control whether adjacent characters are rendered as joined or separate glyphs in scripts like Arabic and Devanagari. Can appear in text from those scripts when copy-pasted.
En space, em space, thin space, and other typographic spaces: typographic whitespace characters used in professional typography, occasionally appearing in text pasted from InDesign, Word, or high-quality web typography.
Where these characters come from
The most common sources of accidental invisible character introduction:
PDF copy-paste: PDFs don't store text as a plain string — they store character positions and rendering instructions. When you copy text from a PDF, the PDF viewer reconstructs a text string from the positioned characters, which often inserts non-breaking spaces or other characters where the original document used special spacing.
Word processor paste: Microsoft Word and Google Docs use non-breaking spaces for certain formatting purposes (between numbers and units, in headings). Pasting from these into a plain text context brings those characters along.
Web page copy-paste: HTML entities render as non-breaking spaces in browsers. Copying text from a web page copies the rendered version, which replaces with U+00A0. Forms, APIs, and databases then receive the invisible character.
Autocorrect and mobile keyboards: some mobile keyboards insert smart apostrophes (U+2018/U+2019) and smart quotes (U+201C/U+201D) instead of standard straight apostrophes and quotes. When used in technical contexts (code, configuration, command-line arguments), these characters break parsers expecting ASCII.
The bugs these characters cause
String comparison failures: "café" === "café" may be false if one contains a combining accent character (e` + combining acute) and the other contains a precomposed character (é). Both look identical; they're different code point sequences. Unicode normalization (NFC vs NFD) is the solution.
Search/filter mismatches: a database record containing a non-breaking space in a "city" field won't match a user's search for that city typed with a regular space — identical to the human eye, different bytes in the database index.
Validation bypasses: a required field containing \u00A0 (non-breaking space) looks empty to the user, passes field !== "" validation (because the field is not an empty string — it contains a character), and fails .trim() !== "" checks if .trim() doesn't strip non-breaking spaces (which many implementations don't by default).
Number parsing failures: a price copied from a European document as 1 000,00 where the thousands separator is a narrow no-break space (U+202F) will fail parseFloat("1 000,00") — returning NaN in JavaScript — because neither the narrow no-break space nor the European decimal comma are standard parsing tokens.
Detecting and fixing invisible characters
In code: use a regex that includes all Unicode whitespace classes, not just ASCII space. In JavaScript: /\p{Z}/gu (the Unicode general category "separator") matches all Unicode space separators, not just U+0020. In Python: str.strip() does handle some Unicode whitespace but not all; re.sub(r'\s+', ' ', text) with the re.UNICODE flag (default in Python 3) is broader.
In databases: periodically audit text fields with: SELECT * FROM table WHERE column LIKE '%' || CHAR(160) || '%' — or the equivalent for your database — to find records containing non-breaking spaces.
In forms/APIs: normalize input on receipt — trim all Unicode whitespace classes, normalize Unicode (NFKC normalization replaces compatibility characters with their canonical equivalents, handling most common invisible-character issues), and validate after normalization.
How to use the Whitespace Cleaner on sadiqbd.com
- Paste text from any source (PDF, Word, web page) before using it in technical contexts — the tool removes or normalizes the full range of invisible characters, not just ASCII spaces and tabs
- For bulk data cleaning: use the tool on lists before importing them to databases, CRMs, or spreadsheets — names, addresses, product codes, and identifiers are common vectors for invisible character contamination
- When "trim" doesn't fix it: if a string looks correct but still fails comparisons or validation, invisible characters are a likely cause; cleaning through this tool reveals what's actually in the string
Frequently Asked Questions
If I normalize all Unicode whitespace to regular spaces, does that break anything? For most application data: no. Non-breaking spaces in user-entered names, addresses, product descriptions, and similar text are almost always artifacts of copy-paste, not intentional choices by the user. Normalizing them to regular spaces produces the intended behavior. The exception: content that was intentionally authored with non-breaking spaces for typographic reasons (professionally designed articles, marketing copy) where the layout depends on the non-break behavior. For such content, blind normalization would change the intended formatting. The typical approach: normalize input data (user-entered forms, imports), but don't process authored content through the same normalization pipeline.
Is the Whitespace Cleaner free? Yes — completely free, no sign-up required.
Try the Whitespace Cleaner free at sadiqbd.com — remove invisible characters, non-breaking spaces, and normalize whitespace in any text.
