Duplicate Keys in JSON: Valid, Silent, Dangerous

This is valid JSON, and most developers are surprised by that:

{ "a": 1, "b": 2, "a": 3 }

Parse it and one value is gone:

JSON.parse('{"a":1,"b":2,"a":3}')
// { a: 3, b: 2 }

No error. No warning. a is 3, the 1 is gone, and nothing anywhere records that a value was discarded.

What the specification actually says

RFC 8259 says object names SHOULD be unique — not MUST. In the language of specifications that is a recommendation, so a document with duplicates is not malformed, and a parser that accepts it is conforming.

The RFC then says, with unusual candour, that implementations differ in how they handle duplicates: some take the last, some take the first, some report an error, and some return all of them. It explicitly warns that behaviour is unpredictable. That is the specification telling you this is a trap.

Last-wins is the common case, and it is what the tools on this site do, because matching JSON.parse matters more than being opinionated.

A detail worth knowing

The surviving entry keeps the position of the first occurrence and the value of the last:

JSON.parse('{"a":1,"b":2,"a":3}')
// { a: 3, b: 2 }   <- "a" is still first, but holds the last value

So a duplicate does not even move the key to where you would notice it. Visually the object looks untouched.

Where duplicates come from

• String-built JSON. Code that concatenates JSON by hand rather than serialising a structure. Add a field in two branches and you have two of them.
• Merged configuration. Two fragments concatenated rather than deep-merged.
• Templating. A loop that emits a key per item without checking for collisions.
• Hand editing. Copy a block, change one field, miss another.
• Deliberate attack. An attacker who knows a validator and a consumer disagree about which value wins can get one value approved and a different one used. This is a real exploit class, not a hypothetical.

How to find them

The difficulty is that the usual tools have already lost the evidence. By the time a document is parsed, the duplicate is gone — so a validator that works on the parsed value cannot see it, and will tell you the document is fine.

The check that works is on the text: count the keys in the raw document and compare against the parsed object. jq does this directly, because it keeps the pairs:

# Does any object lose a key when parsed?
jq '[paths(scalars)] | length' data.json   # paths after parsing
grep -o '"[^"]*"\s*:' data.json | wc -l    # rough count of keys in the text

# jq keeps duplicates when asked, so this prints any repeated name
jq -c 'to_entries | group_by(.key) | map(select(length > 1))' data.json

In code, the reliable approach is a reviver or a parser that surfaces pairs. In JavaScript, JSON.parse with a reviver still receives the collapsed object, so a reviver cannot detect it either — you need a parser that exposes the token stream.

How to stop producing them

1. Never build JSON by string concatenation. Serialise a real structure; a map cannot hold the same key twice, so the bug becomes impossible.
2. Deep-merge configuration fragments rather than concatenating them, and decide explicitly which side wins.
3. Validate against a JSON Schema with additionalProperties: false where it fits — it will not catch duplicates, but it catches the typo-shaped keys that often accompany them.
4. If you consume documents from elsewhere and the distinction matters for security, parse with a library that reports duplicates and reject them at the boundary.

The formatter will quietly collapse duplicates as part of re-serializing, which is a useful side effect: if the output has fewer keys than the input, you had one. The analyzer is the better check for what came out the other side, since it reports key counts per object alongside everything else about the document.