Format JSON Lines (JSONL / NDJSON) Online

What is JSON Lines (JSONL / NDJSON)?

JSON Lines (jsonlines.org) — also known as NDJSON (Newline-Delimited JSON) — is a text format where each line is one complete JSON value, typically an object. The format is designed for streaming and append-only workloads: log files that grow indefinitely, big-data pipelines that produce records one at a time, and APIs that push events over a persistent connection.

The advantage over a single big JSON array is memory. A 100 GB JSON array file must be entirely parsed to extract any record — the closing ] is at the end. A 100 GB JSONL file can be processed one line at a time, using constant memory regardless of file size. Every major data tool (jq, pandas, Spark, DuckDB, PostgreSQL COPY, ClickHouse) supports JSONL natively for this reason.

JSONL Rules

1. One JSON value per line. Usually an object. Arrays and primitives are technically allowed but rarely used. 2. Line separator is \n (LF). Windows CRLF (\r\n) is tolerated by most parsers but not part of the spec.

3. UTF-8 encoding. No BOM. 4. No trailing newline required but conventional — files typically end with \n. 5. Empty lines are usually ignored by robust parsers but not officially allowed. 6. Each line must be complete valid JSON — no line continuations. A record cannot span multiple lines. If your JSON object contains a string with a literal newline, that newline must be escaped as \n inside the string so the object stays on one line.

Working with JSONL in Different Languages

Command Line (jq)

# Pretty-print each line of a JSONL file
cat data.jsonl | jq .

# Filter — output only lines where status is "error"
cat data.jsonl | jq -c 'select(.status == "error")' > errors.jsonl

# Convert JSONL to a JSON array
cat data.jsonl | jq -s . > data.json

# Convert JSON array back to JSONL
cat data.json | jq -c '.[]' > data.jsonl

# Count records
cat data.jsonl | wc -l

Python

import json

# Read JSONL line by line (streaming, constant memory)
with open('data.jsonl') as f:
    for line in f:
        line = line.strip()
        if not line: continue
        record = json.loads(line)
        # process record

# Write JSONL
records = [{'user': 'alice'}, {'user': 'bob'}]
with open('out.jsonl', 'w') as f:
    for r in records:
        f.write(json.dumps(r) + '\n')

# Pandas has read_json with lines=True
import pandas as pd
df = pd.read_json('data.jsonl', lines=True)

Node.js

const fs = require('fs');
const readline = require('readline');

// Stream JSONL line by line (constant memory)
const rl = readline.createInterface({
  input: fs.createReadStream('data.jsonl'),
  crlfDelay: Infinity,
});

for await (const line of rl) {
  if (!line.trim()) continue;
  const record = JSON.parse(line);
  // process record
}

// Write JSONL
const records = [{ user: 'alice' }, { user: 'bob' }];
const out = fs.createWriteStream('out.jsonl');
for (const r of records) out.write(JSON.stringify(r) + '\n');
out.end();

JSONL vs JSON Array — When to Use Each

Use JSONL when: the data is a stream of events (logs, metrics, telemetry), the file will be appended to over time, the total size may exceed available RAM, records are processed independently (no cross-record queries needed at parse time), or the consumer is a data pipeline (Spark, pandas, DuckDB, Athena, BigQuery).

Use a JSON array when: the data is bounded and small (fits comfortably in memory), the payload is being sent as a single API response, the consumer needs the entire dataset in memory anyway (rendering a table, generating a report), or the format is being consumed by generic JSON tooling (browser fetch, JavaScript libraries expecting arrays).

Common JSONL Errors

1. Pretty-printed records that span multiple lines. This is the biggest gotcha. A JSON object formatted with indent=2 spans many lines — that's not valid JSONL. Each record must be minified onto a single line before appending. Use JSON.stringify(obj) (no third argument) or json.dumps(obj) without indent.

2. Trailing commas between records. JSONL has no separator except the newline. There is no comma between lines. If you're seeing {...},\n{...}, someone tried to write a JSON array without the outer brackets — invalid.

3. Corrupted lines from concurrent writes. If two processes append to the same JSONL file without file locking, records can interleave partway through. Always use file locking (flock) or write to per-process files and merge later.

Related JSON Tools

• JSON Formatter (Parent Tool) — the underlying formatter used by every variant
• Format JSON Online — general-purpose beautifier with 2/4-space and tab options
• Minify JSON Online — strip whitespace for production payloads
• JSON Validator — validate without formatting
• JSON Diff Checker — compare two JSON objects side-by-side