JSON Key Value Extractor — Online

Why Flatten JSON to Key-Value Pairs?

Nested JSON is great for expressing relationships between data — a user has an address, an address has a city, a city has a country. But many downstream systems prefer flat data. Configuration systems like Java Properties, .env files, HashiCorp Consul, and AWS Parameter Store are all key-value stores. Database columns are flat. CSV cells are flat. Log aggregators like Datadog and Splunk index flat key-value pairs orders of magnitude better than they index nested JSON.

Flattening JSON to dot-notation pairs (address.city.country = "USA") preserves all the information in a form these flat systems can consume directly. It is also invaluable for exploring an unfamiliar API — a big paginated response can have hundreds of unique fields at various depths, and a flat list of every field with a sample value is far easier to skim than the nested source.

Common Use Cases

Config file migration. Moving from a nested YAML config to a flat Consul/etcd key store? Flatten the JSON equivalent, and each key becomes a Consul path. server.database.host maps directly to a Consul KV key.

Log ingestion. Splunk, Datadog, and Elasticsearch all index flat fields best. Instrumenting an API that returns nested JSON? Flatten before logging so every field is queryable as its own dimension. request.headers.user_agent becomes a Splunk field.

Schema documentation. Given an unknown API response, extract keys-only to generate a schema map. Combined with a type inspector, this bootstraps TypeScript interfaces or JSON Schema documents.

Environment variables. Translate a JSON config to a set of environment variables using UPPER_SNAKE_CASE: {"database": {"host": "localhost"}} becomes DATABASE_HOST=localhost. Our tool has this as a preset output format.

Debugging. Given two API responses that "look different but should be the same," flattening both and diffing the key-value lists shows exactly which fields changed — much clearer than diffing indented JSON.

Dot-Notation vs Bracket-Notation Paths

Two dominant conventions exist for encoding paths. Dot-notation: users.0.email — clean, readable, but breaks if any key contains a dot (which JSON keys can legally do). Bracket-notation: users[0].email or users[0][email] — verbose but handles special characters in keys. Our tool defaults to dot-notation because 99% of JSON has clean keys, and offers bracket mode for the pathological cases.

Numeric array indices work identically in both — the path [0] appears literally in bracket mode, or .0 in dot mode. Some tools use square brackets in dot mode too: users[0].email. This mixed style is common in Elasticsearch and MongoDB path notation.

Extracting Keys in JavaScript

// Recursive flatten to dot-notation pairs
function flatten(obj, prefix = '', out = {}) {
  for (const [k, v] of Object.entries(obj)) {
    const key = prefix ? `${prefix}.${k}` : k;
    if (Array.isArray(v)) {
      v.forEach((item, i) => flatten({ [i]: item }, key));
    } else if (v && typeof v === 'object') {
      flatten(v, key, out);
    } else {
      out[key] = v;
    }
  }
  return out;
}

const pairs = flatten({ user: { name: 'Alice', tags: ['admin', 'dev'] } });
// { 'user.name': 'Alice', 'user.tags.0': 'admin', 'user.tags.1': 'dev' }

// Just the keys (no values)
const keys = Object.keys(pairs);
// ['user.name', 'user.tags.0', 'user.tags.1']

Extracting Keys in Python (pandas)

import pandas as pd, json

data = json.loads('{"user": {"name": "Alice", "tags": ["admin", "dev"]}}')

# pandas json_normalize does dot-notation flattening natively
df = pd.json_normalize(data)
print(df.columns.tolist())
# ['user.name', 'user.tags']  — but arrays are kept as list values

# For a truly flat list including array indices, use the recursive approach
def flatten(obj, prefix=''):
    if isinstance(obj, dict):
        for k, v in obj.items():
            yield from flatten(v, f'{prefix}.{k}' if prefix else k)
    elif isinstance(obj, list):
        for i, v in enumerate(obj):
            yield from flatten(v, f'{prefix}.{i}')
    else:
        yield (prefix, obj)

pairs = dict(flatten(data))
# {'user.name': 'Alice', 'user.tags.0': 'admin', 'user.tags.1': 'dev'}

Extracting Keys with jq

# All paths as jq path expressions
echo '{"user": {"name": "Alice", "tags": ["admin", "dev"]}}' | jq '[paths(scalars)] | .[]'
# [ "user", "name" ]
# [ "user", "tags", 0 ]
# [ "user", "tags", 1 ]

# Formatted as dot-notation strings
echo '{"user": {"name": "Alice"}}' | jq '[paths(scalars) | map(tostring) | join(".")] | .[]'
# "user.name"

# Just leaf values, no paths
echo '{"user": {"name": "Alice"}}' | jq '[.. | select(type != "object" and type != "array")]'

Common Pitfalls

1. Keys with dots inside them. A JSON key like "my.field" is legal. Dot-notation flattening produces my.field.subkey which is indistinguishable from a nested structure my.field.subkey. Use bracket-notation for datasets with dot-containing keys, or escape the dots.

2. Extremely deep trees. Recursive flattening can hit JavaScript's stack limit (~10,000 levels) on pathological inputs. For guaranteed safety, use an iterative implementation with an explicit work queue.

3. Losing type information. Once flattened to a properties-file style, every value is a string. If types matter downstream, preserve them by emitting typed output (JSON pairs) or including a type suffix in the key.

4. Order preservation. Object key order in JSON is technically undefined by the spec (though preserved by every mainstream parser). Flattening walks in insertion order — do not rely on alphabetical order in the output unless you sort explicitly.

Related JSON Tools

• JSON Formatter (Parent Tool)
• JSON Tree Viewer — interactive exploration
• JSON to CSV Converter — flat spreadsheet export
• Sort JSON Keys — canonical output
• JSON Schema Validator — validate the structure