How to view and query API responses and NDJSON logs

Most JSON a developer reads in a day isn't a file someone handed over. It's an API response in a terminal, or a log where every line is a JSON object. Both are easy to produce and annoying to read raw. This guide collects the commands I actually use for both, plus what to do when the output gets too big for a terminal.

All commands below were tested with curl and jq 1.7.

Part 1: API responses

Pretty-print a response

curl -s https://api.example.com/orders | jq .

-s hides curl's progress bar, which otherwise mixes into the output. If you get parse error: Invalid numeric literal at line 1, column 9 instead, look at what came back: often it's an HTML error page, or you added -i and jq is reading the HTTP headers.

Keep the headers, but separately

Headers matter when debugging (status, rate limits, caching, content type), but jq can't parse them. Write them to their own file:

curl -sS -D headers.txt -o body.json https://api.example.com/orders
head -3 headers.txt
jq '.orders | length' body.json

Or print just the status code and timing after the body is saved:

curl -s -o body.json -w '%{http_code} %{time_total}s\n' https://api.example.com/orders

Saving the body to a file is a good habit anyway: you can query it ten times without calling the API ten times.

Pull out what you need

The jq filters that cover most API work:

jq 'keys' body.json                                   # what's at the top level
jq '.orders[0]' body.json                             # one record, to see the shape
jq -c '.orders[] | {id, status, total}' body.json     # a few fields per record
jq '[.orders[] | select(.status == "pending")] | length' body.json
jq -r '.orders[] | [.id, .status] | @tsv' body.json   # tab-separated, for column/sort/awk

-c prints one compact line per result, which is what you want when piping into grep, sort or wc -l. -r prints strings without quotes.

JSON inside a JSON string

Webhooks, message queues and some logging setups wrap a JSON payload as a string inside another JSON object:

{"event": "webhook", "payload": "{\"event\":\"paid\",\"id\":\"A-100\"}"}

fromjson parses it:

jq '.payload | fromjson' event.json

{"event": "paid", "id": "A-100"}

Paginated APIs

Save each page, then merge. If every page has a results array:

jq -s '[.[].results[]]' page*.json > all.json      # one array
jq -c '.results[]' page*.json > all.ndjson         # or one record per line

In the browser

The browser's DevTools are the fastest place to look at a response your own app made: Network tab, click the request, then Preview for a collapsible tree or Response for the raw text. Right-click the request for "Copy response" or "Copy as cURL" to replay it in a terminal.

DevTools gets slow with large responses, and the tree is gone once you reload. When a response is big or you want to keep it, paste it into a viewer. In GigaJSON you can paste the body, or the whole output of curl -i (status line, headers and body): it recognises the HTTP response and opens it as an object with status, headers and body, with the body parsed. The New document dialog also has a From URL tab that fetches a URL directly, as a plain GET from your browser, so it only works for APIs that allow cross-origin requests (CORS) and don't need auth headers.

Part 2: NDJSON logs

What NDJSON is

NDJSON (newline-delimited JSON), also called JSON Lines or JSONL, is one JSON value per line:

{"ts":"2026-09-30T10:00:01Z","level":"info","msg":"request","path":"/api/orders","status":200,"ms":42}
{"ts":"2026-09-30T10:00:02Z","level":"error","msg":"request","path":"/api/orders/17","status":500,"ms":913,"err":"db timeout"}
{"ts":"2026-09-30T10:00:02Z","level":"warn","msg":"slow query","ms":1204,"query":"SELECT ..."}
not json: panic recovered
{"ts":"2026-09-30T10:00:03Z","level":"error","msg":"request","path":"/api/users","status":502,"ms":30010,"err":"upstream"}
{"ts":"2026-09-30T10:00:04Z","level":"info","msg":"webhook","payload":"{\"event\":\"paid\",\"id\":\"A-100\"}"}

The file as a whole isn't valid JSON, and it doesn't need to be. Each line can be written, read and thrown away independently, which is why structured loggers, data exports and log shippers use it. Line four is realistic too: a crash message or stack trace that bypassed the structured logger.

Filter by level, and why jq stops halfway

The obvious command:

jq -c 'select(.level == "error")' app.ndjson

{"ts":"2026-09-30T10:00:02Z","level":"error",...,"err":"db timeout"}
jq: parse error: Invalid numeric literal at line 4, column 4

jq processes the lines in order and gives up at the first one that isn't JSON. Everything after line 4, including the second error, is missing, and the parse error is easy to miss at the end of long output. In a real log that line might be at 3 a.m. and the rest of the night never gets searched.

The fix is to read each line as raw text (-R) and parse it with fromjson?, where the ? drops lines that fail:

jq -cR 'fromjson? | select(.level == "error") | {ts, path, status, err}' app.ndjson

{"ts":"2026-09-30T10:00:02Z","path":"/api/orders/17","status":500,"err":"db timeout"}
{"ts":"2026-09-30T10:00:03Z","path":"/api/users","status":502,"err":"upstream"}

To find the lines that aren't JSON, so you know what you skipped:

awk '!/^[[:space:]]*\{/ { print NR": "$0 }' app.ndjson

It prints 4: not json: panic recovered. It only checks that a line starts with {, which is enough to spot stray text.

Count, group and rank

Count entries per level, in one pass, without loading the file into memory:

jq -nR 'reduce (inputs | fromjson?) as $l ({}; .[$l.level] += 1)' app.ndjson

{"info": 2, "error": 2, "warn": 1}

-n with inputs reads lines one at a time into reduce, so this works on logs far bigger than RAM.

The slowest requests need sorting, and sorting needs all the values at once. -s (slurp) collects the stream into an array:

jq -cR 'fromjson? | select(.ms != null)' app.ndjson \
  | jq -s -c 'sort_by(-.ms) | .[:2] | map({path, ms})'

[{"path":"/api/users","ms":30010},{"path":null,"ms":1204}]

Slurping holds everything in memory. For a few hundred MB that's fine; for more, filter first so only the lines you need are slurped.

Make it readable as columns

jq -rR 'fromjson? | [.ts, .level, .msg, (.status // "")] | @tsv' app.ndjson | column -t -s $'\t'

.status // "" fills the gap where a line has no status, so the columns stay aligned.

Live tailing, compressed logs and big files

tail -f app.ndjson | jq -cR --unbuffered 'fromjson? | select(.status >= 500)'
zcat app.ndjson.gz | jq -cR 'fromjson? | select(.status >= 500) | .path'
grep '"level":"error"' app.ndjson | jq -c '{ts, path}'

--unbuffered makes jq print each match immediately, which matters when its output goes into another pipe. The grep line is a speed trick for large logs: grep discards non-matching lines much faster than jq parses them. It depends on how your logger writes JSON (spacing after colons, key order), so treat it as a pre-filter and keep the real condition in jq.

Python, when the logic grows

import json
from collections import Counter

levels = Counter()
with open("app.ndjson", encoding="utf-8") as f:
    for n, line in enumerate(f, 1):
        try:
            entry = json.loads(line)
        except json.JSONDecodeError:
            print(f"line {n}: not JSON")
            continue
        levels[entry.get("level")] += 1
print(levels.most_common())

Reading line by line keeps memory flat, and unlike jq you get the line number of every bad entry.

Convert between NDJSON and JSON

jq -s . app.ndjson > app.json      # NDJSON to one array (all lines must be valid)
jq -c '.[]' app.json > app.ndjson  # array back to NDJSON

The JSON Lines to JSON tool does the first conversion in the browser.

Looking at logs instead of grepping them

Sometimes you don't know what you're looking for yet: which fields exist, what an unusual entry looks like, whether status is always a number. For that, a table beats a terminal. GigaJSON opens .ndjson and .jsonl files as a list with one item per line. The Table view shows them as rows with a column per field, which you can sort and filter, and the Inspector shows the most common values of the selected field across the first 10,000 entries. The Query tab runs JSONPath or JMESPath over the whole log, for example $[?(@.status >= 500)]. Parsing happens in a worker in your browser, so a log of a few hundred megabytes doesn't freeze the page, and nothing is uploaded.

Two limits to know. Strip non-JSON lines first (the jq -cR 'fromjson?' command above writes a clean copy), since a broken line stops the file from opening as NDJSON. And jq in the browser handles inputs up to about 20 MB; for larger logs use JSONPath or JMESPath there, or jq on the command line. For files over a few hundred MB, see how to open a large JSON file.

A short cheat sheet

To try any of the filters on your own data without installing anything, paste it into the jq playground, or open it in the app for the tree, table and query views.