How to Open a Large JSON File

You double-click a 45 MB export and your editor stops responding. The natural conclusion is that JSON is slow to parse at that size. It is not, and measuring it points you at a completely different fix.

Parsing is not the bottleneck

A 45.1 MB document — 500,000 records of five fields — on Node 22:

Two hundred milliseconds. Whatever is making your editor hang for thirty seconds, it is not the parser — the parser finished before you let go of the mouse button.

What actually costs

The same document, measured differently:

There is the answer. A text editor asked to open that file has to lay out three and a half million lines, apply syntax highlighting to them, and build an undo buffer. A JSON viewer that renders a tree naively has to create three million DOM nodes. Browsers fall over somewhere around a hundred thousand.

Pick the tool that matches what you need

To look around the structure

A viewer that virtualises — renders only the rows currently on screen — handles this size comfortably, because the work is proportional to your viewport rather than to the file. Collapsed branches cost nothing at all; expanding is what costs, which is why a tree stays fast if you only open what you need.

The viewer here opens files up to 8 MB directly, asks for confirmation between 8 MB and 200 MB because it will use real memory, and declines above that. Parsing runs in a Web Worker so the page stays responsive while it works.

To extract specific data

Do not open it at all. jq streams, so memory stays flat regardless of file size:

# Pull one field from every record
jq -r '.[].email' huge.json

# Filter, then look at what is left
jq -c '.[] | select(.active == true)' huge.json | head -50

# True streaming for files that exceed RAM
jq --stream -n 'fromstream(1|truncate_stream(inputs))' huge.json

The plain forms still load the document into memory; --stream is the one that does not, and it is what you want past a gigabyte.

To know what is in it

Before reading any of it, find out what you are dealing with:

# Top-level shape and record count, without printing the data
jq 'if type == "array" then {type:"array", count:length} else {type:"object", keys:(keys)} end' huge.json

# The field names, from the first record only
jq '.[0] | keys' huge.json

Or paste a representative slice into the analyzer, which reports field types, how complete each field is, and where the data is inconsistent. A 500-record sample tells you almost everything a 500,000-record file would.

Things that make it worse

• Pretty-printing it first. Adds 17 MB and 3.5 million lines here, for no benefit if a tool is reading it. Only format what a person will read.
• `cat`ing it into a terminal. Your scrollback buffer becomes the bottleneck. head -c 2000 instead.
• Loading it into a browser tab you also need for other work. The tab shares a memory limit with everything in it.
• Reaching for an online tool that uploads. At this size the upload alone is slow, and your data has left the building. Everything here runs locally — the file is read with the FileReader API and never transmitted.

The structural fix

If you are generating files at this size, the format is the problem, not the file. One giant array has to be complete to be worth anything and has to be fully parsed before you can read record one.

NDJSON — one document per line — is the same number of bytes, streams naturally, works with head, grep and split, and survives truncation: I measured 502 of 503 records recovered from a file cut in half, against total loss for the equivalent array. For logs, exports and anything appended over time it is strictly better.

And if the file is large because it is one record per row with no nesting, it may not want to be JSON at all. That shape is a table, and CSV or Parquet will serve it better.