The Pattern That Works on Your Machine
/^[a-zA-Z0-9]+$/
Straightforward alphanumeric validation. Ships to production. Then a user called Müller can't register, a Japanese customer's name is rejected, and someone with a Cyrillic username gets an error they can't interpret.
The pattern is doing exactly what it says. The problem is that a-z means the 26 letters of the basic Latin alphabet, and most of the world's names aren't written with only those.
Regex has a Unicode layer that most developers never touch, and it's where the difference between a pattern that works for English test data and one that works for real users lives.
\w Is Not "Word Character"
The shorthand classes are far narrower than their names suggest. By default, in most engines:
\w = [A-Za-z0-9_]
\d = [0-9]
\s = space, tab, newline, carriage return, form feed, vertical tab
\w excludes every accented letter, every non-Latin script, and every character outside basic ASCII. \d excludes the digit characters used in Arabic, Devanagari, Thai and many other scripts.
With the Unicode flag enabled, this changes. In JavaScript, adding u and using property escapes:
/^\p{L}+$/u // any letter, any script
/^[\p{L}\p{N}]+$/u // letters and numbers, any script
/^\p{Lu}/u // uppercase letter
Python's re module treats \w as Unicode-aware by default for str patterns. .NET does too. Java requires UNICODE_CHARACTER_CLASS. PCRE requires the u modifier. Go's regexp supports \p{...} directly.
The behaviour differs by language, and assuming is where the bug comes from. Check what your engine does with \w before relying on it for anything user-facing.
The Regex Tester lets you check a pattern against real input:
- Enter your pattern and flags.
- Paste test strings — including non-ASCII ones.
- See what matches.
The habit worth building: always include non-ASCII test cases. A name with an umlaut, one in a non-Latin script, an emoji, an accented character in both composed and decomposed form. If your pattern only ever sees John Smith, it hasn't been tested.
Unicode Property Escapes
The \p{...} syntax matches by Unicode property, which is far more precise than character ranges.
General categories:
\p{L} any letter
\p{Lu} uppercase letter
\p{Ll} lowercase letter
\p{N} any number
\p{Nd} decimal digit
\p{P} punctuation
\p{S} symbol
\p{Z} separator (includes various spaces)
\p{C} control and format characters
\p{M} combining mark
Scripts:
\p{Script=Latin}
\p{Script=Cyrillic}
\p{Script=Han}
\p{Script=Arabic}
\p{Script=Devanagari}
Negation uses a capital P: \P{L} matches anything that isn't a letter.
Practical examples:
// Name validation that works internationally
/^[\p{L}\p{M}'\- ]+$/u
// Any decimal digit in any script
/\p{Nd}+/u
// Strip control and format characters
str.replace(/\p{C}/gu, '')
// Detect any CJK content
/\p{Script=Han}/u
That name pattern includes \p{M} for combining marks, which matters for the next section.
Combining Marks and Normalisation
The same visible character can be represented two different ways in Unicode.
é as U+00E9 (single precomposed character)
é as U+0065 + U+0301 (letter e + combining acute accent)
They look identical. They're different strings. A regex matching one won't match the other, and a string comparison between them returns false.
This is why café might not equal café depending on where each string came from — one from a keyboard, one from a database, one from a file exported by a different system.
The fix is normalisation before matching:
str.normalize('NFC') // compose: prefer single characters
str.normalize('NFD') // decompose: letter + combining marks
NFC is the usual choice for storage and comparison. Normalise at your system boundary — the moment data arrives — and everything downstream compares consistently.
There's a pattern-level consequence too. . matches a single code point, not a single perceived character. A decomposed é is two code points, so:
/^.$/u.test('é') // true for the precomposed form
// false for the decomposed form
A pattern expecting one character sees two.
Emoji and the Grapheme Problem
Emoji make this worse, because many are built from several code points.
😀 U+1F600 — 1 code point
👍🏽 U+1F44D + U+1F3FD — 2 (base + skin tone modifier)
👨👩👧 3 emoji + 2 zero-width joiners — 5 code points
🇩🇪 2 regional indicator symbols — 2 code points
A "single emoji" the user sees can be five or more code points. Truncating a string at an arbitrary code point boundary can split a family emoji into three separate people, or strip a skin tone modifier, or leave half a flag.
For emoji specifically:
/\p{Emoji_Presentation}/u
/\p{Extended_Pictographic}/u
For counting or splitting by what the user perceives as a character, regex isn't the right tool — Intl.Segmenter with granularity: 'grapheme' handles grapheme clusters properly.
Anywhere you truncate user-supplied text — a database column limit, a preview snippet, a character counter — this matters. Splitting mid-cluster produces visibly broken output.
Surrogate Pairs Without the Unicode Flag
In JavaScript, characters above U+FFFF are stored as two UTF-16 code units. Without the u flag, regex operates on those units individually:
'😀'.length // 2
/^.$/.test('😀') // false — sees two units
/^.$/u.test('😀') // true — sees one code point
Character classes break similarly:
/[😀-😎]/ // without u: nonsense, ranges over surrogate halves
/[😀-😎]/u // with u: a valid code point range
The u flag is close to mandatory for any pattern that might encounter non-BMP characters. The newer v flag adds set operations and string properties on top.
Case-Insensitivity Is Language-Dependent
The i flag does simple case folding, which is correct for English and wrong in specific places.
The Turkish dotted I. In Turkish, the uppercase of i is İ (dotted capital I), and the lowercase of I is ı (dotless i). A case-insensitive match treating I and i as equivalent is wrong in Turkish locale.
German ß. Its traditional uppercase form is SS, which is a one-to-two character mapping. Case folding that assumes one-to-one doesn't handle it.
Greek final sigma. σ and ς are the same letter in different positions, both uppercasing to Σ.
For validation and matching where correctness across locales matters, locale-aware comparison via Intl.Collator is more reliable than regex case-insensitivity. For most application purposes the i flag is fine — just know where it isn't.
Validation Patterns Worth Reconsidering
Names. Don't restrict to [A-Za-z]. Allow letters from any script, combining marks, apostrophes, hyphens and spaces. Better still, apply a length limit and almost nothing else — name validation rejects more legitimate names than it catches bad input.
Email. The full RFC-compliant pattern is enormous and still doesn't tell you whether an address exists. A pragmatic check — contains an @, has something either side, has a dot in the domain — plus an actual verification email is the working approach. Note that internationalised addresses with non-ASCII local parts and domains exist.
Postcodes and phone numbers. Formats vary enormously by country. A single pattern that handles all of them either doesn't exist or is so permissive it validates nothing. Use a library with per-country rules.
URLs. Internationalised domain names contain non-ASCII characters. A pattern restricted to ASCII will reject valid URLs.
The general principle: validate loosely, normalise consistently, and verify through an actual channel where it matters. Strict format validation on user identity data reliably excludes real people.
Practical Tips
Enable the Unicode flag on any pattern touching user input.
Use \p{L} instead of [A-Za-z] for anything name-like.
Normalise to NFC at your system boundary.
Include non-ASCII test cases in every pattern you write — accented, non-Latin, emoji, and a decomposed form.
Don't truncate by code point where graphemes matter. Use a grapheme segmenter.
Check what your engine's \w and \d actually cover. It differs by language and by flag.
Prefer libraries for phone numbers, postcodes and complex formats. Regex isn't the right shape for per-country rule sets.
FAQ
Does \w match accented letters?
Depends on the engine and flags. In JavaScript it doesn't by default. In Python 3 it does for str patterns. Check rather than assume.
What does the u flag do?
Enables Unicode mode — . matches whole code points rather than UTF-16 units, \p{...} property escapes become available, and character ranges work correctly above U+FFFF.
Why doesn't my pattern match café?
Possibly a decomposed form — e plus a combining accent rather than a single precomposed character. Normalise to NFC first.
How do I match any letter in any language?
\p{L} with the Unicode flag enabled.
Why does my emoji count as multiple characters? Many emoji are multiple code points — skin tone modifiers, zero-width joiners, regional indicators. Use a grapheme segmenter to count what users perceive.
Should I validate names with regex? Very loosely if at all. Strict name validation excludes legitimate names in most of the world's scripts.
The Takeaway
Most regex bugs involving real users trace back to patterns written against ASCII assumptions. Turning on the Unicode flag, swapping character ranges for property escapes, and normalising input before matching resolves the majority of them — and testing with non-ASCII input is what makes the problem visible in the first place.
Write and debug regular expressions free with the Regex Tester at sadiqbd.com — no sign-up, instant results.
