Skip to the content
GigaJSON
All tools
Format and check
View and query
Convert
JSON to code
Encode and decode
Sample files
Guides Open GigaJSON

How to view and query API responses and NDJSON logs

· 7 min read · Updated · By Amir Badi

The JSON you read on a normal workday mostly comes from two places: API responses in a terminal, and logs with a JSON object on every line. Machines write both easily, and people find both a pain to read raw. These are the commands I use for each, and what I do when the output outgrows the terminal.

I tested every command here with curl and jq 1.7.

GigaJSON displaying JSONL log records as rows with status and duration columnsGigaJSON displaying JSONL log records as rows with status and duration columns (dark theme)
These synthetic logs include a 503 response. Table view makes the records easy to compare. Screenshots follow the page theme. Click to view full size.

Part 1: API responses

Pretty-print a response

curl -s https://api.example.com/orders | jq .

-s hides curl's progress bar, which would otherwise get mixed into the output. If jq says parse error: Invalid numeric literal at line 1, column 9, look at what came back. It's often an HTML error page, or you passed -i and jq is trying to read the HTTP headers.

Keep the headers, but separately

You need the headers when debugging (status, rate limits, caching, content type), but jq can't parse them. Write them to a file of their own:

curl -sS -D headers.txt -o body.json https://api.example.com/orders
head -3 headers.txt
jq '.orders | length' body.json

Or print just the status code and timing after the body is saved:

curl -s -o body.json -w '%{http_code} %{time_total}s\n' https://api.example.com/orders

Save the body to a file anyway. You can then query it ten times without calling the API ten times.

Pull out what you need

These jq filters cover most of my API work:

jq 'keys' body.json                                   # what's at the top level
jq '.orders[0]' body.json                             # one record, to see the shape
jq -c '.orders[] | {id, status, total}' body.json     # a few fields per record
jq '[.orders[] | select(.status == "pending")] | length' body.json
jq -r '.orders[] | [.id, .status] | @tsv' body.json   # tab-separated, for column/sort/awk

-c prints one compact line per result, handy when you pipe into grep, sort or wc -l. -r prints strings without quotes.

JSON inside a JSON string

Webhooks, message queues and some logging setups wrap a JSON payload as a string inside another JSON object:

{"event": "webhook", "payload": "{\"event\":\"paid\",\"id\":\"A-100\"}"}

fromjson parses it:

jq '.payload | fromjson' event.json
{"event": "paid", "id": "A-100"}

Paginated APIs

Save each page, then merge. If every page has a results array:

jq -s '[.[].results[]]' page*.json > all.json      # one array
jq -c '.results[]' page*.json > all.ndjson         # or one record per line

In the browser

For a response your own app made, open DevTools: Network tab, click the request, then Preview for a collapsible tree or Response for the raw text. Right-click the request for "Copy response", or "Copy as cURL" to replay it in a terminal.

DevTools slows down on large responses, and you lose the tree when you reload. For a big response, or one you want to keep, paste it into a viewer. GigaJSON takes the body alone or the whole output of curl -i (status line, headers and body). It recognises an HTTP response and opens it as an object with status, headers and a parsed body. The New document dialog also has a From URL tab. It sends a plain GET from your browser, so it works only for APIs that allow cross-origin requests (CORS) and need no auth headers.

Part 2: NDJSON logs

What NDJSON is

NDJSON (newline-delimited JSON), also called JSON Lines or JSONL, is one JSON value per line:

{"ts":"2026-09-30T10:00:01Z","level":"info","msg":"request","path":"/api/orders","status":200,"ms":42}
{"ts":"2026-09-30T10:00:02Z","level":"error","msg":"request","path":"/api/orders/17","status":500,"ms":913,"err":"db timeout"}
{"ts":"2026-09-30T10:00:02Z","level":"warn","msg":"slow query","ms":1204,"query":"SELECT ..."}
not json: panic recovered
{"ts":"2026-09-30T10:00:03Z","level":"error","msg":"request","path":"/api/users","status":502,"ms":30010,"err":"upstream"}
{"ts":"2026-09-30T10:00:04Z","level":"info","msg":"webhook","payload":"{\"event\":\"paid\",\"id\":\"A-100\"}"}

The file as a whole isn't valid JSON, and it doesn't have to be. A program can write, read or drop each line on its own, so structured loggers, data exports and log shippers all like the format. Line four is realistic too: a crash message or stack trace that went around the structured logger.

Filter by level (and why jq stops halfway)

You'd start with:

jq -c 'select(.level == "error")' app.ndjson
{"ts":"2026-09-30T10:00:02Z","level":"error",...,"err":"db timeout"}
jq: parse error: Invalid numeric literal at line 4, column 4

jq reads the lines in order and gives up at the first one that isn't JSON. You lose everything after line 4, the second error included, and the parse error is easy to overlook at the bottom of a long output. In a real log that bad line might sit at 3 a.m., and you'd never search the rest of the night.

Read each line as raw text (-R) and parse it with fromjson?. The ? drops lines that fail:

jq -cR 'fromjson? | select(.level == "error") | {ts, path, status, err}' app.ndjson
{"ts":"2026-09-30T10:00:02Z","path":"/api/orders/17","status":500,"err":"db timeout"}
{"ts":"2026-09-30T10:00:03Z","path":"/api/users","status":502,"err":"upstream"}

To find the lines that aren't JSON, so you know what you skipped:

awk '!/^[[:space:]]*\{/ { print NR": "$0 }' app.ndjson

That prints 4: not json: panic recovered. It only checks that a line starts with {, which is enough to catch stray text.

Count, group and rank

To count entries per level in one pass, without loading the file into memory:

jq -nR 'reduce (inputs | fromjson?) as $l ({}; .[$l.level] += 1)' app.ndjson
{"info": 2, "error": 2, "warn": 1}

-n with inputs reads lines one at a time into reduce, so this works on logs far bigger than RAM.

Finding the slowest requests means sorting, and sorting needs every value at once. -s (slurp) collects the stream into an array:

jq -cR 'fromjson? | select(.ms != null)' app.ndjson \
  | jq -s -c 'sort_by(-.ms) | .[:2] | map({path, ms})'
[{"path":"/api/users","ms":30010},{"path":null,"ms":1204}]

Slurping holds everything in memory. A few hundred MB is fine. Past that, filter first so you only slurp the lines you need.

Make it readable as columns

jq -rR 'fromjson? | [.ts, .level, .msg, (.status // "")] | @tsv' app.ndjson | column -t -s $'\t'

.status // "" fills the gap where a line has no status, so the columns stay aligned.

Live tailing, compressed logs and big files

tail -f app.ndjson | jq -cR --unbuffered 'fromjson? | select(.status >= 500)'
zcat app.ndjson.gz | jq -cR 'fromjson? | select(.status >= 500) | .path'
grep '"level":"error"' app.ndjson | jq -c '{ts, path}'

--unbuffered makes jq print each match at once, which you need when its output feeds another pipe. The grep line speeds up large logs: grep throws away non-matching lines far faster than jq can parse them. It relies on how your logger writes JSON (spacing after colons, key order), so use it as a pre-filter and keep the real condition in jq.

Python, when the logic grows

import json
from collections import Counter

levels = Counter()
with open("app.ndjson", encoding="utf-8") as f:
    for n, line in enumerate(f, 1):
        try:
            entry = json.loads(line)
        except json.JSONDecodeError:
            print(f"line {n}: not JSON")
            continue
        levels[entry.get("level")] += 1
print(levels.most_common())

Reading line by line keeps memory flat, and you get the line number of each bad entry, which jq won't give you.

Convert between NDJSON and JSON

jq -s . app.ndjson > app.json      # NDJSON to one array (all lines must be valid)
jq -c '.[]' app.json > app.ndjson  # array back to NDJSON

The JSON Lines to JSON tool does the first conversion in the browser, and JSON to JSON Lines the second.

Looking at logs instead of grepping them

Sometimes you don't know what you're after yet. Which fields exist? What does an odd entry look like? Is status always a number? A table answers those faster than a terminal. GigaJSON opens .ndjson and .jsonl files as a list with one item per line. The Table view gives each field a column you can sort and filter, and the Inspector lists the most common values of the selected field across the first 10,000 entries. The Query tab runs JSONPath or JMESPath over the whole log, for example $[?(@.status >= 500)]. A worker in your browser does the parsing, so a log of a few hundred megabytes won't freeze the page, and the file stays on your machine.

Two limits. Strip the non-JSON lines first (the jq -cR 'fromjson?' command above writes a clean copy), because one broken line stops the file from opening as NDJSON. And jq in the browser takes inputs up to about 20 MB. For bigger logs, use JSONPath or JMESPath there, or jq on the command line. Files over a few hundred MB are covered in how to open a large JSON file.

A short cheat sheet

Task Command
Pretty-print a response curl -s URL | jq .
Body and headers apart curl -sS -D headers.txt -o body.json URL
Parse a JSON string field jq '.payload | fromjson'
Merge pages jq -s '[.[].results[]]' page*.json
Filter a log, skip bad lines jq -cR 'fromjson? | select(.level == "error")' app.ndjson
Count per field value jq -nR 'reduce (inputs | fromjson?) as $l ({}; .[$l.level] += 1)'
Follow a live log tail -f app.ndjson | jq -cR --unbuffered 'fromjson? | select(...)'
NDJSON to array jq -s . app.ndjson

To try these filters on your own data without installing anything, paste it into the jq playground, or open it in the app for the tree, table and query views.

Questions

What is the difference between JSON and NDJSON?

A JSON file holds one value, usually one big object or array. NDJSON (also called JSON Lines or JSONL) holds one complete JSON value per line, with no commas or brackets around them. That makes it easy to append to, split, grep and process one line at a time.

How do I open an NDJSON or JSONL file?

Any text editor shows it, one record per line. To read it comfortably, pretty-print it with jq (jq . file.ndjson), or open it in a viewer that understands the format. GigaJSON opens .ndjson and .jsonl files as a list you can browse as a tree or a table.

Why does jq stop with 'parse error' halfway through my log file?

One line isn't valid JSON, often a stack trace or a plain-text message mixed into the log. jq stops at the first one. Read lines as raw text and parse them yourself so bad lines are skipped: jq -cR 'fromjson? | select(.level == "error")' app.log.

How do I convert NDJSON to a JSON array?

With jq, slurp the lines into one array: jq -s . app.ndjson > app.json. The reverse is jq -c '.[]' app.json > app.ndjson. GigaJSON's JSON Lines to JSON tool does the first conversion in the browser.

More from the blog