What Breaks When a Platform Changes Its JSON Shape

There are two kinds of upstream API change, and only one of them wakes you up.

The first kind removes a field your parser dereferences. You get a TypeError, an alert fires, someone opens the dashboard, and the fix ships that afternoon. Unpleasant, bounded, over.

The second kind changes something your parser does not dereference. A view_count that used to be an integer starts arriving as "1.2M". A timestamp that was ISO-8601 becomes a Unix epoch. An array that always had thirty items starts arriving with zero, because the wrapper object was renamed and your extractor now finds nothing to extract. No exception. No alert. Status 200, well-formed JSON, and a pipeline that keeps writing rows.

You find out about the second kind six weeks later, when someone asks why the numbers look odd.

This post is about designing for the second kind: why validation alone does not catch it, what a canary field is and how to pick one, why drift should degrade rather than fail, and the specific way this bit us on YouTube.

---

Three ways an upstream API changes underneath you

Ordered by how loudly they announce themselves, which is the inverse of how much they cost.

The field that vanishes

response.data.user.followerCount becomes response.data.user.followers. Your code throws on undefined, or your parseInt returns NaN and something downstream chokes on it.

This is the good case. It is loud, it is immediate, it is localised to one code path, and the stack trace names the field. You will fix it in an hour. Every incident review that focuses on this class of change is optimising the wrong thing.

The field that changes type or meaning

Quieter and considerably worse. Three variants worth naming separately, because they fail differently:

Type change. An integer becomes a formatted string. 1200000 becomes "1.2M". In a loosely typed pipeline this flows straight through to storage. Number("1.2M") is NaN; parseInt("1.2M") is 1 — and 1 is a number, so nothing downstream complains. You now have a channel with 1 subscriber in your database and no error anywhere.

Unit change. duration in seconds becomes duration in milliseconds. Every value is still an integer, still positive, still plausible. Your averages are off by a factor of a thousand and nothing in the shape of the data says so.

Semantic change. The nastiest, because the payload is correct — it just answers a different question. viewCount on a video page starts counting the whole playlist. followerCount on a business profile switches from followers to total page likes. The field name is unchanged, the type is unchanged, the value is a reasonable number. Only a human who knows the domain can tell.

The array that starts arriving empty

The quietest of all, and the one that produces the most confidently wrong data, because "empty" is a state you already handle correctly.

A platform renames the wrapper object that holds a list. Your extractor walks the response looking for the old key, finds nothing, and returns []. The endpoint answers 200. Your code path for "this channel has no videos" runs, because that is a real thing that happens, and it does exactly what it is supposed to do: nothing.

Every instrument you own says the system is healthy. Error rate flat, latency improved — empty responses are fast — and your row count quietly steps down to zero. This is the same failure geometry as a 200 with an empty body, arriving from the opposite direction: there the platform decided not to serve you, here the platform served you fine and your parser stopped understanding the answer. Both look identical from your monitoring.

---

The one that got us: videoRenderer → lockupViewModel

YouTube's watch and channel pages ship their state as a large JSON blob embedded in the HTML. For years, a video in a channel grid was a videoRenderer — or a gridVideoRenderer, or a reelItemRenderer for shorts. You walked the tree, collected every node under those keys, and mapped them.

Then YouTube rebuilt channel grids on a newer component system, and grid items started arriving as lockupViewModel. Not a renamed field inside the old shape — a different shape:

// Before
{ "videoRenderer": { "videoId": "…", "title": { "runs": [{ "text": "…" }] },
                     "viewCountText": { "simpleText": "1.2M views" } } }

// After
{ "lockupViewModel": {
    "contentId": "…",
    "metadata": { "lockupMetadataViewModel": {
      "title": { "content": "…" },
      "metadata": { "contentMetadataViewModel": { "metadataRows": [
        { "metadataParts": [ { "text": { "content": "1.2M views" } },
                             { "text": { "content": "5 months ago" } } ] } ] } } } } } }

The video ID moved from videoId to contentId. The title stopped being a runs array of text fragments and became a plain content string. View count and publish date stopped being their own named fields and became untyped strings in a flat list of "metadata parts", distinguishable only by pattern-matching the text — /view/i for one, /ago$/i for the other.

The endpoint did not error. It returned { "items": [] }, for every channel, with a 200. A channel with no public uploads is a legitimate response, so nothing in the response shape was invalid. The only signal that anything was wrong was that an endpoint which normally returns thirty items was returning zero, for everyone, all at once.

Two things we changed afterwards, both of which are the actual lesson:

Parse the new shape first, keep the old ones as a fallback. Platforms roll changes out gradually and roll them back. Our channel-videos parser now collects lockupViewModel nodes and then also collects gridVideoRenderer, videoRenderer and reelItemRenderer, deduplicating by video ID. A partial rollback on YouTube's side no longer takes the endpoint down, and the cost of keeping the dead branch is a few dozen lines.

Treat "zero items on an endpoint that always returns thirty" as an incident, not a result. More on that below, because it is the generalisable half.

---

Why schema validation alone doesn't catch this

The instinctive fix is a validator. Define the response shape in zod or Pydantic or JSON Schema, validate every payload, alert on failures.

It is worth doing and it will not save you, for three reasons.

Most drift is schema-valid. "1.2M" is a valid string. [] is a valid array. A timestamp in the wrong unit is a valid integer. A schema constrains structure; almost every expensive drift preserves structure and changes meaning.

Strict validation converts cosmetic changes into outages. If you .strict() your schemas, the day a platform adds a harmless new optional field, your endpoint starts throwing. You will loosen the schema after the second such incident, and then it catches nothing.

Validators are written against the fields you use. The lockupViewModel payload above would pass any schema we had, because our schema described our normalised output — id, title, viewCount — and the normaliser was returning an empty list, not an invalid one. Zero valid objects is valid.

Validation answers "is this well-formed?" Drift detection has to answer "is this the same thing it was yesterday?" Those are different questions and they need different instruments.

---

Canary fields: assert on the handful of things that must always be there

A canary is a very small set of fields, declared per endpoint, that a genuine response cannot plausibly lack. Not a schema. Two or three fields, checked after parsing and normalisation.

function detectDrift(def, data) {
  if (!def.driftCanaries?.length || data === null || typeof data !== 'object') return [];
  const target = Array.isArray(data) ? data[0] : data;
  if (!target) return [];
  return def.driftCanaries.filter((field) => !(field in target));
}

That is the whole mechanism. The interesting work is choosing the fields.

Choosing canaries

The rule that has worked for us: pick fields whose absence means something structural moved, and nothing else.

For a profile endpoint, id, handle and followerCount. An account without a follower count does not exist; if that key is gone, the profile block moved. For a single video: id and viewCount. For a transcript: the text itself and the ID of the post it belongs to.

Three failure modes to avoid:

• Too many canaries. A canary list that mirrors your schema fires on every optional field a platform adds. You will start ignoring the alert within a month, and an ignored alert is worse than no alert, because it consumes the attention budget that a real one needs.
• Canaries on optional data. bio, avatarUrl, location — all legitimately absent on plenty of real accounts. A canary that fires on normal data is a canary you will delete.
• Canaries on the wrapper instead of the contents. This one is subtle and we get it wrong. For list endpoints our canary is items — and { "items": [] } contains the key items, so the check passes. Our canary mechanism catches the shape where a field was removed; it does not catch the shape where a list arrives empty. That is the exact failure the YouTube change produced, and it is caught by a different instrument entirely.

Say the quiet part: canaries are cheap and they cover one of the three classes above. Do not let a green canary check convince you the other two are handled.

---

Drift should degrade, not fail

When a canary does fire, the tempting behaviour is to throw. Resist it.

A missing canary means some of what you expected is gone. It rarely means all of it. If a profile response has lost followerCount but still carries the handle, display name, bio and avatar, then failing the request throws away four fields you have in hand in exchange for a cleaner error taxonomy. Your caller wanted the data.

The rule we wrote down for ourselves, in the billing spec rather than in a wiki page: a drifted response is returned in full, flagged, and not billed for. Partial data beats no data, and charging for data you have just told yourself you are no longer sure about is the wrong instinct. The endpoint gets marked as drifted in a health table with the list of missing fields, a warning is logged with the endpoint name, and the caller gets their payload.

That combination — return it, flag it, don't charge for it — is what makes the flag usable. If drift were an error, there would be pressure to keep the canary list tiny so it never fires. Because drift is free and non-fatal, the canary list can be honest.

---

Alert on distributions, not just exceptions

This is the instrument that would have caught the YouTube change on the same day, and it is the one most teams don't have.

Log two numbers on every single request: response size in bytes and item count. Then alert on the shape of the distribution, per endpoint, not on individual values.

• An endpoint that normally returns 25–30 items returning 0, for more than a handful of consecutive calls, is an incident. Not "no results" — an incident. channel-videos returns 30 items and search returns 18–26; a sustained zero on either is structural.
• A response-size histogram that was unimodal around 40KB and is now bimodal with a spike near zero is a gate closing or a parser missing.
• A field that was non-null 99% of the time and is now non-null 3% of the time has changed, whatever the schema says.

The last one is the only practical defence against semantic and type drift. You cannot detect 1200000 → "1.2M" by looking at one response — both are plausible. You can detect it instantly by noticing that the null rate for the parsed integer went from 0.4% to 97% in an hour. Track per-field null rates for the ten fields you actually care about, compare today against a trailing baseline, and alert on the delta.

None of this needs a monitoring vendor. It needs one row per request with the endpoint name, byte count, item count, and duration, and a query you run on a schedule.

---

Recorded fixtures, and the trap of testing only against them

Parser tests should run against recorded fixtures: a real captured response, saved to disk, replayed with no network. It is the only way to test parsing deterministically, it runs in CI in milliseconds, and it makes a parser bug reproducible forever. We test every scraper this way, at the run() boundary, with no live calls.

And it is structurally incapable of detecting drift. The fixture is a photograph of the API on the day you took it. Every test passes forever, including for the eighteen months after the platform changed. A green test suite is evidence your parser still handles the old shape, and no evidence at all that the old shape is still being served.

What closes the gap is a separate, small, scheduled job that hits the real thing — a handful of known-stable targets, run on a timer, outside the test suite, allowed to be flaky, and alerting on item counts and canaries rather than on exact values. Keep it out of CI. A live check in CI blocks deploys when a platform has a bad afternoon, and a blocked deploy is how the live check gets deleted.

A useful habit alongside it: when you capture a fixture, record the date you captured it in the filename or a sibling metadata file. A fixture with no date is an assertion about the world with no expiry, and after two years nobody remembers whether it was ever right.

---

A checklist for any pipeline you don't control the top of

1. Log bytes and item count on every request. Cheapest instrument here by a wide margin, and it is the one that catches the expensive class.
2. Declare two or three canary fields per endpoint. Fields whose absence means something moved. Not a schema.
3. On a canary miss: return the data, flag the endpoint, don't bill it. Degrade, don't fail.
4. Alert on distributions. Zero items on an endpoint that always returns thirty. Null rates that move by an order of magnitude. Response sizes that go bimodal.
5. Parse the new shape first, keep the old parser as a fallback. Platforms roll back. Deduplicate the results and pay the few dozen lines.
6. Never derive a number from a formatted string without a range check. parseInt("1.2M") === 1 is the whole argument.
7. Date your fixtures, and run a live canary job outside CI. Green tests prove your parser handles the shape you recorded, nothing more.
8. Write down, per endpoint, what a normal response looks like — typical item count, typical size, which fields are never null. You cannot detect abnormal without that, and it takes ten minutes per endpoint.

The framing that makes all of this land with the people who have to fund it: schema drift is a data-integrity problem, not an error-handling problem. Error handling is about requests that fail. Drift is about requests that succeed and lie, and the cost is not downtime — it is every decision anyone made on six weeks of quietly wrong data. There is a related and even quieter version of this on the billing side, where a payload that is semantically identical looks different to a diff and charges the customer for a change that never happened.

The endpoints that never drift, incidentally, are the ones a platform is obliged to publish and versions on a public schedule — which is most of why ad-library scrapers break and the official API doesn't.