Four Ways to Price a Scraping API, and the Workload Each One Is Right For
Daily allowances, never-expiring credits, pay-for-success, and change-based billing are four different products. Which one wins depends on the shape of your month.
13 min read · 20 Aug 2026
You have three tabs open. One is a $100/month subscription with a daily unit allowance. One is a $497 pack of credits that never expire. One is a $29 pack that only charges you when a request actually returns something.
They are not three prices for the same thing. They are three different bets about the shape of your workload, and whichever one is cheapest for you is mostly determined by facts about your month that none of the pricing pages ask about.
This post is about the four models in this category, what each is genuinely good at, and how to work out which one your workload wants. We run one of them, which you should treat as a bias rather than an authority — the section that matters most to us is the one where a competitor's model is plainly better than ours.
First: marginal cost in this category varies by two orders of magnitude
Before comparing models, one fact that every pricing page in the category obscures.
The cost of serving a request depends almost entirely on whether the target requires a residential IP, and that varies wildly by platform. From our own production cost table:
| Platform | Residential proxy needed? | Marginal cost per 1,000 |
|---|---|---|
| YouTube | No | ~$0 |
| Meta Ad Library | No — official Graph API | ~$0 |
| Instagram / Twitter | Yes | ~$0.30 – $0.75 |
That is not a 20% difference. YouTube costs essentially nothing to serve — bandwidth on a box with unmetered egress — while Instagram costs real money per request because someone has to rent residential bandwidth to get it. The mechanism behind that split is an ASN lookup, and it is the single largest input into anyone's cost structure in this business.
Every vendor in the category prices flat anyway. One credit, one request, whatever the platform. Which means the pricing model you choose is layered on top of a cost structure it does not reflect, and there is a subsidy running underneath all four models discussed below. We will come back to who pays it.
Model 1 — Daily-allowance subscriptions
The shape: a monthly fee buys a number of units per day. Unused units do not carry over. EnsembleData runs this: $100/month for 1,500 units/day, scaling to $1,400/month for 50,000/day, with a 50-units/day trial.
Do the arithmetic in months, not days. 1,500/day is 45,000/month, so $100 works out at $2.22 per 1,000 — if you use every unit every day. Use 20% of them and you are paying $11.11 per 1,000 for the ones you used. At the top tier, $1,400 for 1.5M units/month is $0.93 per 1,000 fully utilised, which is competitive with anything in this post, and $4.67 at 20% utilisation, which is not.
What it is genuinely good at. Two things, and they are underrated. The first is a hard, structural cost ceiling: you cannot overspend. Not with a runaway loop, not with a misconfigured cron, not with an agent that decided to retry three thousand times. Prepaid credits will happily drain to zero in an afternoon; a daily allowance stops at midnight's number and resumes tomorrow. If you are handing API access to something autonomous, that property is worth paying for. The second is that finance likes it — a fixed monthly line item is easier to approve and defend than a variable consumption bill, which is not an engineering virtue but is a real reason teams buy.
Where it fails. Bursty workloads, which is most of this category. Campaign research, ad-spy sweeps, and monthly reporting all look like a flat line with a spike in it. Under a daily allowance you must buy for the peak day, then pay for that capacity on all the days you use a twentieth of it. A workload that runs 45,000 requests in the last three days of the month needs a 15,000/day plan and wastes 27 days of it.
Model 2 — Never-expiring prepaid credits
The shape: you buy a bucket, you spend it whenever, it never evaporates. ScrapeCreators runs this: $47 for 25,000 credits ($1.88/1k) and $497 for 500,000 ($0.99/1k), no subscription, no expiry, and — unusually — no feature gating at all, so the free tier gets every one of their 179 endpoints.
Say the uncomfortable thing plainly: at volume, this is the best deal in the category, and it is not one we can beat. $0.99 per 1,000 with credits that never expire prices utilisation risk at zero. Every other model in this post has a way to make you pay for capacity you did not use; this one does not. If your only question is cost per successful request at scale and you are willing to prepay, the arithmetic is settled and the rest of this post is about the conditions under which cost per request is not your only question.
It is also structurally hard to undercut. They have caching that makes head demand nearly free, and a new entrant undercutting $0.99/1k is starting a race with worse margins than the incumbent. Anyone in this category who tells you their per-request price beats that number is either not counting something or not going to survive counting it.
Where it costs you. You are extending an interest-free loan to a vendor. A large prepaid balance is exposure — to the company disappearing, to a pricing change you cannot now walk away from, to a platform they support breaking. Prepay what you would be relaxed about writing off, and let the discount curve be secondary until you trust them. The other cost is behavioural: nothing about a never-expiring balance creates urgency to check whether you are wasting it. Daily allowances make waste visible every night; credit buckets make it invisible until the top-up.
Model 3 — Pay only for successful responses
The shape: the billing unit is a successful response, not a request. SociaVault runs this at $29 for 6,000 credits, about $4.83 per 1,000 — two-and-a-half to five times the price per unit of the credit model above.
This is the strongest trust guarantee in the category, and it is stronger than ours. That deserves the detail, because "we don't charge for failures either" is the easy reply and it is not the same thing.
We publish two guarantees that sound equivalent: you are never charged for a failed request, and never charged for an empty result. Both are real, both are specified, both are tested. But the mechanism is a refund, not a pricing model — we charge on a cache miss, run the request, and refund in full if it throws or comes back empty. Three consequences follow, and all three favour the pay-for-success model:
One — our guarantee depends on our own classifier. "Failed" and "empty" are definitions in our code. We define empty precisely: null, an empty array, an object whose items/results/posts/comments/ads/videos array is empty, or an object with no keys. That is a good definition and it is still a definition, written by the party that benefits from edge cases falling outside it. A response that parses cleanly and is wrong — the 200 with a plausible, truncated payload — is charged, by us and by everyone else. Under a model where the billing unit is a successful response, there is no classifier standing between the outcome and the invoice.
Two — refunds do not restore your daily spend cap. We offer hard daily caps, and the cap counts gross charges: a refund puts credits back in your balance but does not give you back cap headroom, because the cap exists to limit gross activity. So a bad hour against a blocked platform can exhaust your daily cap entirely on requests that were all refunded, and your next legitimate request gets a 429. That is a deliberate design decision and it is defensible. It is also a real way in which "you were never charged" does not mean "it cost you nothing."
Three — reconciliation is two-sided. Every failure produces a charge row and a refund row. Netting to zero is correct, and it is more work than a ledger where the row was never written.
None of that makes our guarantee bad. It makes it a remedy, where theirs is a design, and a design is the stronger of the two. You pay two-and-a-half to five times per unit for it — which for a low-volume, high-difficulty workload against Instagram or TikTok may well be the correct trade, especially since SociaVault also publishes real schemas and runs actively monitored depth on exactly those two platforms. We do not support TikTok at all, and our Instagram endpoints carry a warning in our own README.
Where this model fails. Price per unit — at volume the arithmetic stops working. And "success" needs defining just as carefully from their side; ask what a 200 with an empty body counts as before assuming the guarantee covers your worst case.
Model 4 — Change-based billing
The shape: you register a target and an interval. The service polls it. Polls are free. You are charged only when the payload actually changes.
This is ours, so read the next paragraph with that in mind.
Mechanically: a scheduled poll runs with charging disabled and is never billed, whatever it returns. After each poll the payload is fingerprinted — sorted keys at every nesting depth, with volatile fields like the fetch timestamp stripped first, because hashing those would make every poll look like a change and bill you for nothing. A charge is taken only when the fingerprint differs from the previous one. The first poll establishes a baseline and is never billed. If you cannot afford the change charge, the subscription does not break — the new fingerprint is still recorded, so you are not billed twice for the same change once you top up.
What it is good at. Monitoring, and only monitoring. If you are watching 10,000 targets hourly and 97% of them are unchanged, every other model on this page charges you for 7 million requests a month that told you nothing. This one charges you for the 3% that did. That is the arithmetic behind why polling is a tax, and it is the workload this model exists for.
Where it is simply the wrong shape — three cases, and they are common:
- One-shot research and backfills. You need 500 profiles now. There is nothing to compare against, no second poll, no change. Use batch requests.
- Data you need regardless of change. If your pipeline recomputes a rollup nightly whether or not the inputs moved, "we'll tell you when it changes" does not reduce your call volume.
- Data that changes every time you look. A view counter on an active video changes every poll. Change billing then degenerates into per-request billing with a worse forecast, because your bill depends on how volatile your targets happen to be rather than on how many requests you made. That is the one case where flat per-request pricing is more predictable.
And one honest gap. The two halves of this are individually tested — polls charge nothing, fingerprints are stable across key reordering — but the integration between them, the poll → diff → charge loop, is not yet covered end to end in our test suite. Our own spec names that gap. Do not take "billed only on change" from us or anyone else as battle-tested until someone shows you the test that fails when it breaks.
Who subsidises whom
Return to the cost table at the top. YouTube costs ~$0 to serve. Instagram costs $0.30–0.75 per 1,000. Under flat per-request pricing — which every model above uses — the YouTube user pays the Instagram user's proxy bill.
For a vendor that is fine, even elegant: cheap platforms fund expensive ones and the rate card stays simple. For a buyer it is worth noticing which side you are on. If your workload is entirely YouTube, RSS, oEmbed, or an official ad-transparency API, you are paying a blended rate for the cheapest data in the category and have more leverage in a pricing conversation than you think. If it is entirely Instagram and TikTok, you are the one being subsidised, the quoted price is better than it looks, and you should be more suspicious of anyone undercutting it dramatically.
What "free cache hits" is actually worth
Free cache hits are now table stakes — ScrapeCreators shipped them, we ship them, and the line reads identically on both sites. What it is worth to you is entirely a property of your traffic.
If your workload is head-shaped — everyone asks about the same few hundred creators — a shared cache is a genuine, large discount, and the vendor is happy because they scraped that creator once. If your workload is long-tail — you are the only person asking about your targets — your hit rate is nothing more than your own repeat rate, and "cache hits are free" is a feature you will use approximately never. Measure it before you value it: over a representative week, what fraction of your requests are for a target you already requested inside your acceptable staleness window?
One thing to ask about, since a shared cache means another tenant's request can warm yours: is per-tenant isolation available, and is it standard or an enterprise upsell? For most workloads shared caching is a straightforward win. For anything where the set of targets you are watching is itself sensitive — competitive research, an investigation — that is a question worth asking before you sign.
The four models, side by side
Including the rows we lose on, because a comparison table without them is an advertisement.
| Daily allowance | Never-expiring credits | Pay-for-success | Change-based | |
|---|---|---|---|---|
| Example | EnsembleData | ScrapeCreators | SociaVault | this API |
| Platforms | ~8 | 29 | ~2, deep | 4 |
| API operations | dozens | 179 | dozens | 15 |
| Rate at volume | ~$0.93/1k fully utilised | $0.99/1k | ~$4.83/1k | no published rate card |
| Unused capacity | expires nightly | never expires | n/a | n/a |
| Charged for failures | yes | not guaranteed | no | refunded |
| Charged for empty results | yes | yes — 15 credits documented | no | refunded |
| Hard spend ceiling | structural | no | no | daily cap, counted gross |
| Official SDKs | ✅ | ❌ | ✅ | ❌ |
| SOC 2 / compliance docs | ? | ❌ | ❌ | ❌ |
| Async + batch jobs | ❌ | ❌ | ❌ | ✅ |
| Billed per change | ❌ | ❌ | ❌ | ✅ |
| Unified cross-platform schema | ❌ | ❌ | ❌ | ✅ |
Read our column honestly. Four platforms against twenty-nine, fifteen operations against 179, no SDKs, no SOC 2, and no published rate card at all — "what do you charge per thousand" is a real question and our answer today is that we have not published one. What we have that the other three columns do not is async jobs, batching, change-based billing, and one schema across platforms. Whether that trade is good depends entirely on whether you are building a monitor or running a query.
Questions to ask about your own workload first
Answer these before you open a pricing page. They determine the answer more than the rate card does.
- What is your peak-to-average ratio across a month? Above about 3:1, daily allowances get expensive fast and prepaid credits win.
- What fraction of your requests are for something you already fetched recently? That is what free cache hits are worth to you, in a number.
- What is your expected failure rate against your hardest platform? At 5% it is a rounding error and pay-for-success is not worth a 3x unit price. At 30% against Instagram from a marginal proxy pool, it is the whole decision.
- Are you querying or monitoring? Querying wants credits or batches. Monitoring wants change billing. Most teams are doing both and should not force one model across them.
- Is anything autonomous holding your key? If yes, a structural spend ceiling stops being a nice-to-have.
- What would you be relaxed about writing off if the vendor vanished? Prepay that much and no more.
Question 4 is the one people get wrong most often, and it is the one that most changes the answer. A team that treats a monitoring workload as a query workload will pay for the 97% of polls that returned nothing, every hour, forever, and will conclude that the category is expensive.
For our part: we run per-request credits with refunds for failures and empty results, plus change-based billing on subscriptions, and every endpoint publishes its cost as an x-credit-cost field in the OpenAPI spec so the arithmetic is machine-readable rather than something you reconstruct from a table. Two models, because a monitor and a query genuinely are different products. And on the two axes above where a competitor is better — a pay-for-success design and $0.99/1k that never expires — they are better, and we would rather you heard that here than found it out three months in.
Sources and dates. Competitor pricing, plan structures, and feature comparisons are recorded in research/03-pricing-and-unit-economics.md and research/07-competitive-landscape.md, gathered 2026-08-20 from public pricing pages and vendor documentation. Our own guarantees, definitions, and the daily-cap behaviour described above are in SPEC.md §1.3, §2.5 and §6. Pricing in this category changes often — archive anything you plan to rely on.