2026-07-02 · 5 min read · Rahul Chitturi
- json
- logs
- debugging
JSON Lines files look simple: one JSON value per line. In production logs, that simplicity hides real-world problems like truncated writes, escaped nested payloads, mixed timestamp formats, and copied stack traces that break a parser only after deployment.
Debug the line format as data before changing ingestion code. The fastest path is usually to isolate the first bad record, validate it alone, and compare it with a known-good event.
Validate one record, not the whole file first
A JSON formatter expects one complete value at a time, while a JSON Lines stream contains many independent records. Copy a single failing line into a validator first. If it parses, the problem may be in line splitting, compression, encoding, or a multiline message field.
- Each line should be a complete JSON object or value
- Blank lines may be invalid depending on your importer
- Pretty-printed JSON is not JSON Lines unless each record returns to one line
- Trailing commas are still invalid inside each record
Spot truncation and escaping problems
Log shippers can truncate long events at byte limits, leaving an object without a closing brace. Nested JSON payloads may be escaped as a string inside a log record, which means they need a second decode step before you inspect their fields. Base64 payloads add another layer that should be decoded only after you confirm the outer record is valid.
Normalize timestamps before comparing incidents
JSON Lines logs often combine Unix seconds, JavaScript milliseconds, and ISO strings from different services. Convert timestamps to a common UTC view before deciding that one service lagged or emitted out of order. A timestamp unit mismatch can look exactly like ingestion delay.
A Flashman workflow
Validate the suspicious record with the JSON formatter, decode nested Base64 fields only after redaction, convert event times with the timestamp tool, and diff a working event against the failing one. This keeps the debugging loop local while preserving enough structure for a useful incident note.
When the issue is fixed, save a small redacted fixture in your ingestion tests so future parser changes cover the exact log shape that broke production.