2026-09-03 · 8 min read
- http
- compression
- api-design
HTTP content codings such as gzip reduce representation size in transit. They are separate from media type, character encoding, and transfer framing, even though client libraries often process all four before application code sees a response.
A reliable integration assigns each transformation to one layer. That prevents stale headers, double decompression, corrupted text decoding, misleading size checks, and cache variants that serve the wrong representation.
Model the representation pipeline
Begin with the origin representation bytes and record each operation through compression, HTTP framing, proxy handling, client decoding, character decoding, and format parsing. Name which component owns each step.
- Use Content-Type for media type and any applicable charset.
- Use Content-Encoding for representation codings in applied order.
- Treat Transfer-Encoding as hop-level message framing.
- Do not assume Content-Length describes the decoded representation.
Bound compressed and expanded data
Enforce wire-size limits before buffering, then cap decompressed bytes and expansion ratio while decoding. A small compressed body can expand beyond application memory or parser budgets.
Apply timeouts and abort decoding when a limit is crossed. Return a controlled error without logging complete response bodies, and ensure retries do not repeatedly consume the same expensive payload.
- Count decoded bytes before text or JSON parsing.
- Reject unsupported or excessive coding chains.
- Stream large approved representations where possible.
- Keep parser depth and member limits after decompression.
Keep proxies and caches consistent
A proxy that decompresses or recompresses a body must update Content-Encoding, length metadata, validators, and cache behavior consistently. Confirm how Vary and Accept-Encoding affect cache keys on the deployed path.
Range requests, strong validators, and signatures can depend on a particular representation. Follow the platform contract instead of mixing byte ranges or hashes from compressed and decoded forms.
Build exact-byte fixtures
Use Flashman's JSON formatter after successful decoding, Base64 tool to carry synthetic compressed bytes, hash tool to identify exact representations, diff to compare headers and decoded text, and units converter for explicit limits.
Test identity and gzip responses, empty bodies, truncation, concatenated streams if supported, stale headers, proxy recompression, cache variants, decompression limits, malformed bytes, and clients with automatic decoding enabled and disabled.