The 29-Second Ceiling: What a Gateway Timeout Does to an API's Design

Somewhere in the second month of a data API's life, a support ticket arrives that says the endpoint "sometimes works." It works for small accounts and times out for large ones. It works at 3am and fails at 2pm. Every retry the customer runs takes 29 seconds to fail.

That number is a fingerprint. Twenty-nine seconds is the default maximum integration timeout on AWS API Gateway, and if your API sits behind one, it is the hard limit on how long any synchronous request can take — no matter what your handler is doing, no matter how long Lambda would happily have run.

What makes this worth a post is not the number. It is what a hard synchronous ceiling does to product design, long before anyone thinks of it as an architectural decision. A timeout you inherited from a gateway default quietly determines which endpoints you are able to offer at all, and the constraint propagates outward into your pricing, your docs, and your customers' retry loops. It belongs to the same family as the other structural costs in why polling is a tax: a default nobody chose, charged forever.

---

Where 29 seconds comes from, and why raising it does not help much

API Gateway's REST and HTTP APIs both cap how long they will wait for your integration to respond. The default maximum is 29 seconds. AWS has since made that raisable for REST APIs through a service quota increase; HTTP APIs remain capped at 30 seconds and that one does not move.

So the first instinct — file for the quota increase — is sometimes available. It is also mostly beside the point, because the gateway is one ceiling in a stack of them:

Layer	Typical ceiling
CloudFront origin response	30 s default, extendable to a low minutes
Application Load Balancer idle timeout	60 s default, configurable
API Gateway integration	29 s default (REST raisable, HTTP capped at 30 s)
Lambda execution	15 min hard
Browser fetch with no explicit timeout	varies; ~5 min is common
An HTTP client library's default	30 s, very often
A person watching a spinner	~10 s before they assume it broke

Raise the gateway and you hit the CDN. Raise the CDN and you hit whatever HTTP client the customer is using, which you do not control and cannot see. Raise all of it and you are asking a caller to hold a TCP connection open for four minutes across the public internet on a mobile network, which is a request they will fail to honour in ways that produce no useful error on either side.

The ceiling is not really 29 seconds. It is "however long a synchronous HTTP request can be relied on across an infrastructure stack you only partly own," and that number is small and not improving.

Which reframes the problem usefully: the fix for a synchronous timeout is almost never a longer timeout.

---

What a hard ceiling forecloses

Here is the part that gets discovered late. A synchronous ceiling does not just make slow endpoints fail. It makes entire categories of endpoint impossible to offer, and you will find yourself declining feature requests for reasons you would struggle to explain to the person asking.

Anything whose duration depends on the target rather than on you. This is the killer for any API that fetches from third parties. Your own code takes 40 milliseconds. The platform you are fetching from takes between 300 ms and, occasionally, 25 seconds. You cannot make that distribution narrower, you cannot see its tail from your own metrics until you are in it, and the tail is exactly where the valuable requests live — the accounts with the most posts, the videos with the longest transcripts, the searches with the most results.

Anything paginated where the caller wants all of it. A profile with 40 posts is one round trip. A profile with 4,000 is forty. You can offer the forty round trips as forty API calls and push the orchestration onto the customer, which is a real answer, but it is a worse product and it triples their integration code.

Anything batched. "Fetch these 500 channels" is the single most requested shape in a data API and the most obviously impossible one under a sync ceiling. Five hundred targets at a conservative 400 ms each is 200 seconds of work minimum, and that is with perfect parallelism and no retries.

Anything with a variable-cost preprocessing step. Minting a proof-of-origin token, solving a challenge, warming a session, spinning up a browser context. These are usually fast and occasionally are not, and "occasionally is not" is precisely what a hard ceiling punishes.

The pattern across all four: the ceiling does not bite on the average request. It bites on the tail, and the tail is disproportionately the requests that matter. A p50 of 900 ms and a p99 of 26 seconds is a healthy-looking dashboard and a product that fails its best customers.

This is not hypothetical, and it is not a criticism of anyone's engineering. ScrapeCreators, the largest incumbent in social-data APIs, is synchronous-only behind API Gateway, and their own changelog notes that include_replies=true "may time out at 29 seconds" — an accurate, honest disclosure of exactly this constraint. (Changelogs get edited; archive the entry if you are going to rely on it.) The point is not that they made a mistake. It is that the constraint was inherited from a gateway default and then became a permanent feature of the product surface, which is how this almost always happens.

---

The four workarounds, and what each costs

Once you accept that the ceiling is real, there are four moves. Each of them is correct in some situation and each of them costs something specific.

Truncate the work

Cap the result at whatever fits — first 100 items, first N pages — and return it as if it were the whole answer.

Costs: correctness, silently. The customer gets 100 posts and does not know whether that is all of them or the first 100 of 4,000. If your response does not carry an explicit "there is more" signal, you have built a system that produces wrong analysis. If it does, you have really built pagination and should say so.

Right when: the truncation point is a genuine product decision ("recent posts" means the last 30) rather than a disguised infrastructure limit.

Chunk on the client

Push the loop to the caller. Document that they should call you once per target and manage their own concurrency.

Costs: every customer writes the same rate-limiting, retry, and partial-failure code, and most write it badly. You also multiply your request volume by the chunk factor, which changes your rate limits and, if you bill per request, your pricing.

Right when: the chunks are naturally independent and the caller genuinely wants control over ordering and concurrency.

Poll a status endpoint

Return quickly with a handle, let the caller poll until it is done.

Costs: a polling loop in every client, and a polling interval that is either wasteful or laggy. It is also the workaround most likely to be implemented badly by the API rather than the client — see below.

Right when: most of the time, honestly. This is the workhorse.

Move to a job resource

Same as polling, but the work becomes a first-class addressable object with a lifecycle, not a request that happens to be slow.

Costs: real engineering. You need a queue, workers, durable result storage, a retention policy, and an answer to "what happens if the caller never comes back." It is the most work of the four by a wide margin.

Right when: the work is genuinely long-running, or when you want it to survive a client disconnect, which is the property the other three cannot give you at any price.

---

202 plus a job resource: the shape, and the parts people get wrong

The shape is unremarkable and almost everyone gets the edges wrong, so it is worth being specific.

POST /v1/jobs
{ "endpoint": "youtube.transcript", "params": { "url": "..." } }

202 Accepted
{
  "jobId": "job_a1b2c3",
  "status": "queued",
  "estimatedCredits": 1,
  "pollUrl": "https://api.example.com/v1/jobs/job_a1b2c3"
}

Five things separate a job API that is pleasant from one that is not:

Return 202, not 200. 200 means "here is the result." 202 means "accepted for processing, the result is elsewhere." Clients and proxies both understand the difference, and using 200 for an accepted job is how you end up with a caching layer serving a stale "queued" status forever.

Give them the poll URL. Do not make the caller construct it from a template in your docs. An absolute URL in the response body means their client needs no knowledge of your routing, and it means you can move the resource later without a breaking change.

Estimate the cost up front. The caller committed to unknown work. Telling them what it will probably cost at submission time is the difference between a job API and a blank cheque, and it is the single most common omission.

Make partial results first-class. A batch of 500 where 480 succeeded is not a failure and it is not a success. It is a completed job with 480 results and 20 errors, and each error needs to say which target it belongs to. An API that returns "job failed" for that case has thrown away 480 successful fetches, and the customer will re-run all 500.

Publish the retention window. Job results live somewhere and that somewhere costs money, so they expire. If you do not say when, the caller has no way to know whether a 404 on a two-week-old job means "expired" or "you have a bug." Put an explicit expiry on the job object.

The thing this design actually buys you is not speed. A job takes exactly as long as the equivalent synchronous request would have. What you buy is that the work is no longer coupled to a connection. The client can disconnect, the mobile network can drop, the browser tab can close, and the work continues and lands somewhere retrievable. That is a different property from "slower requests allowed," and it is the one worth having.

---

Batch as a resource, not a loop

If you have built jobs, batching is nearly free, and it is worth doing as a distinct resource rather than letting people submit N jobs.

POST /v1/jobs/batch
{
  "endpoint": "youtube.channel",
  "targets": [{ "handle": "@a" }, { "handle": "@b" }, ...]
}

Three reasons this is better than a client-side loop over the single-job endpoint, none of which is "fewer HTTP requests":

You control the concurrency. You know your upstream's tolerance and your own capacity; the caller does not. Given 500 targets you can run them at whatever parallelism is actually safe. Given 500 separate submissions you get whatever concurrency their Promise.all felt like.

Failures aggregate meaningfully. One batch with a 4% error rate is a legible object. Twenty failed jobs scattered among 480 successful ones is a reconciliation problem you have handed to your customer.

It is one billable, cancellable, inspectable thing. "Cancel that batch" is a sentence you can implement. "Cancel those 500 jobs I submitted, some of which have already finished" is not.

Pick a hard cap and publish it — ours is 500 targets per batch, rejected at validation with the limit named in the error, because an unbounded batch endpoint is a denial-of-service vector wearing a friendly name.

---

The billing consequences nobody plans for

Async changes when you know the cost, and if you bill per request that is a genuine problem rather than a detail.

Synchronously, the sequence is easy: the work finishes, you know what happened, you charge. Asynchronously the caller wants a number at submission time and the truth is not available until later. Three sub-problems fall out:

Estimate versus actual. You quote estimatedCredits at submission. A batch of 500 where 40 targets do not exist costs less than quoted. Decide now whether you charge the estimate and refund, or charge on settlement — and whichever you pick, make the job object show both numbers. A cost that changes between submission and completion with no record of why is a support ticket. The general rule, which applies to any billing derived from work you performed rather than requests you received, is that the customer has to be able to predict the number before you charge it.

Failure attribution. In a 500-item batch some items fail for reasons that are yours (a proxy died) and some for reasons that are theirs (the handle is misspelled). Anyone can agree the first should not be billed. The second is a real design decision with defensible answers in both directions — you did the work, and they got no data. Ours is the blunt version: a failed item is never charged, and neither is an empty result, on the grounds that a rule the customer can predict is worth more than a rule that is maximally fair to us.

Cancellation. A cancelled batch that was 60% complete has consumed 60% of the cost. Bill 60%, or bill nothing? Both are defensible, neither is obvious, and if you have not decided before someone cancels a large batch, you will decide it under pressure with an angry customer on the line.

All three of those get easier or harder depending on which billing model you started from, and the choice is upstream of the architecture rather than downstream of it — the four models in this category and the workload each one fits is the longer version of that argument.

---

When synchronous is the correct answer

The argument above is not that everything should be a job. Most things should not.

Synchronous is right when the work is bounded by your code rather than someone else's — a cache lookup, a database read, a computed value. It is right when the caller cannot proceed without the answer, which describes most interactive UI. It is right when the response is small and the p99 is comfortably inside a second, because a job resource for a 40-millisecond operation is a poll loop and a queue in exchange for nothing.

The useful test is not average latency. It is: does the duration of this request depend on a system I do not control? If yes, the tail is not yours to bound, and no timeout you configure will make it so. If no, keep it synchronous and enjoy the simplicity.

The failure mode worth avoiding is the one where you never asked the question — where an infrastructure default set at project setup silently becomes the boundary of what your product is allowed to do, and two years later you are declining a feature request without being able to say why.

We took the other branch: our long-running work is POST /v1/jobs, batching is POST /v1/jobs/batch up to 500 targets, and results are retrieved from the job resource rather than held open on a connection. It is more moving parts — a queue, workers, result storage, an expiry policy — and it is the right trade for work whose duration belongs to somebody else's servers.