flashman
← All posts

JSON Lines import failures: fix logs one record at a time

Fix JSON Lines import failures by checking one-object-per-line structure, escaped newlines, trailing commas, encodings, and log wrappers before parsing.

2026-07-04 · 5 min read · Rahul Chitturi

  • json
  • logs
  • debugging

JSON Lines is friendly to streaming logs because each line is a standalone JSON value. It is also easy to break with copied stack traces, pretty-printed objects, shell prefixes, or one trailing comma from an array export.

When an import job rejects a large file, reduce the problem to one failing line and validate that line as strict JSON before changing the parser.

Confirm the file is not a JSON array

A JSON array wraps records in square brackets and separates them with commas. JSON Lines stores one JSON object per newline with no outer wrapper. A pipeline that expects one format will usually fail loudly on the other, but error messages can point far away from the true mismatch.

  • JSON Lines: each non-empty line parses independently
  • No trailing comma should appear after a line
  • Embedded newlines inside strings must be escaped as \n
  • Pretty-printed multi-line objects are not JSON Lines records

Watch for log wrapper text

Application logs often prepend timestamps, severity labels, request IDs, or container names before the JSON payload. That wrapper text is useful for humans but invalid for a JSON Lines importer. Separate transport metadata from the JSON body before validating the record.

If the wrapper timestamp is important, normalize it into a field instead of leaving it outside the object.

Compare a known-good record

Diffing one accepted line against one rejected line usually exposes schema drift quickly: a string became an object, a number became null, or an escaped stack trace turned into real newlines. Validate after redaction so secrets and customer identifiers do not enter shared examples.

A Flashman workflow

Paste a single redacted record into the JSON formatter first. If it validates, diff it against the failing record and convert important timestamps for incident notes. Repeat on a small sample before running the full import again.

Because Flashman runs locally in the browser, teams can inspect internal log shapes without uploading payloads to a server-side formatter.

Try these tools