flashman
← All posts

Password Unicode and byte limit debugging

Debug password failures by separating characters from bytes, preventing silent truncation, preserving input, and testing normalization and hash-library limits.

2026-09-04 · 6 min read · Rahul Chitturi

  • passwords
  • unicode
  • security

A password that fits a character limit can exceed a byte limit after UTF-8 encoding. Emoji and many non-ASCII characters use multiple bytes, while some password-hashing libraries or legacy systems process only a bounded prefix.

Silent truncation is dangerous because distinct passwords can authenticate as the same effective input. Changing Unicode normalization between registration and login can also lock out users.

Document the complete input contract

Define minimum and maximum policy, encoding, normalization behavior, and the password hashing library's effective byte handling. Enforce a pre-hash limit explicitly and return a generic validation message without logging the password.

  • Do not silently truncate characters or encoded bytes
  • Do not trim or case-fold passwords
  • Apply identical processing at registration and authentication
  • Use a dedicated salted password-hashing algorithm

Handle migrations without weakening checks

When replacing a legacy hash, verify the old record using its exact historical input rules, then rehash the successfully authenticated password with the new policy. Store algorithm and parameter metadata with the record.

Do not use a fast general-purpose hash to avoid a library limit. Choose a maintained password-hashing implementation and configure memory, time, parallelism, salt, and output according to current platform guidance.

A Flashman workflow

Use the password generator to create non-production test values, units converter to document byte budgets, Base64 for synthetic byte fixtures, and diff to reveal normalization changes in harmless samples.

Test ASCII, accented text, composed and decomposed forms, emoji sequences, values at each byte boundary, malformed transport, legacy migration, copy and paste, and all supported clients.

Try these tools