2026-09-10 · 8 min read
- url
- idna
- dns
Internationalized domain names let applications accept Unicode labels while DNS uses an ASCII-compatible form. Labels beginning with xn-- are A-labels whose payload uses Punycode, but valid conversion also depends on the selected IDNA mapping and validation rules.
A URL parser has additional responsibilities around scheme, user information, ports, IPv6 literals, percent escapes, paths, and special schemes. Applying a Punycode function to an unparsed URL can corrupt non-host components or conceal an authority-boundary bug.
Choose the processing profile explicitly
Record whether each component follows IDNA2008, Unicode Technical Standard 46 compatibility processing, or a platform URL standard. Transitional mappings and runtime Unicode versions can change results for characters such as sharp s, joiners, and context-sensitive code points.
- Parse the URL before extracting the hostname.
- Map and validate complete labels, not arbitrary substrings.
- Reject disallowed code points and failed contextual rules.
- Enforce DNS label and complete-name length limits on the ASCII form.
Canonicalize for comparison without hiding errors
For host allowlists, cookie-domain checks, redirect policy, and cache keys, compare the canonical ASCII hostname produced by the same parser and IDNA profile used for the request. Handle a terminal root dot according to an explicit policy.
Do not normalize a malformed authority into an allowed host. Reject embedded credentials where they are unexpected, distinguish ports, and account for IPv4 and IPv6 canonicalization separately from IDNA.
Treat display names as a security surface
A technically valid Unicode hostname can resemble another brand through mixed scripts or confusable characters. Preserve the actual destination, use browser-grade display policies where appropriate, and show the ASCII form in high-risk confirmation or administrative interfaces.
- Do not use visual similarity as an identity test.
- Register and monitor relevant domain variants according to risk.
- Match TLS certificates against the network hostname using maintained libraries.
- Keep redirects, cookies, and origin checks bound to parsed hosts.
Test the complete resolution path
Use Flashman's URL tool on public examples, diff for Unicode and ASCII forms, case converter for safe case fixtures, number-base converter for code-point inspection, and regex tester only for narrow display diagnostics rather than hostname validation.
Test mixed scripts, normalization forms, dot variants, joiners, deviation characters, uppercase A-labels, trailing dots, maximum lengths, malformed xn-- labels, user information, ports, redirects, DNS resolution, certificate matching, cookie scope, and each deployed client and server runtime.