2026-09-02 · 6 min read · Rahul Chitturi
- xml
- encoding
- debugging
An XML document may look correct in an editor and still fail with an invalid character or encoding error. The parser consumes bytes, so the XML declaration, byte order mark, HTTP metadata, and actual encoding must agree.
Copying text through a Unicode editor can hide the mismatch. The visible characters survive, but the saved bytes may change from UTF-16 to UTF-8 while an old declaration remains.
Inspect bytes before editing text
Preserve a non-sensitive failing sample and record how it was transported. Identify a byte order mark, the declaration's encoding value, the Content-Type charset, and the decoder selected by the receiving library.
- Check for UTF-8, UTF-16 little-endian, and UTF-16 big-endian markers
- Verify that no log prefix appears before the XML declaration
- Compare declared and transport charsets
- Locate the byte offset of the first parser error
Decode once at the boundary
Configure one trusted boundary to turn bytes into Unicode, then pass text or parsed nodes internally. Avoid repeated encode-decode cycles and do not repair unknown bytes by deleting every non-ASCII character.
If signatures cover the XML, preserve the original bytes and follow the signature profile's canonicalization rules. A harmless-looking encoding conversion can invalidate a correct signature.
A Flashman workflow
Use the XML formatter after correct decoding, Base64 for a safe byte fixture, diff for decoded text comparisons, and hash to label exact fixture bytes without confusing them with parsed values.
Test declarations with and without byte order marks, transport charset conflicts, non-ASCII text, truncated multibyte sequences, signatures, and every producing system.