Writing
How this data actually behaves
We buy proxies and run scrapers; we do not sell either. That is an unusual position in this category and it is the only reason these posts are worth reading — we can publish the vendor that did not respond, the endpoint of ours that is rate-limited, and the approach we tried and abandoned.
Built for machines
5 postsChange detection, schema drift, silent failures, and the billing consequences of each — the problems that appear once something else is consuming your pipeline on a schedule.
The 29-Second Ceiling: What a Gateway Timeout Does to an API's Design
→A hard synchronous timeout is not an ops detail. It decides which endpoints you can offer, and it is not the only ceiling in the stack.
10 min read · 20 Aug 2026
Your API's Next Customer Is an Agent: MCP, Skills, llms.txt, and HTTP 402
→Four machine-readable surfaces an API needs when the caller is not a person, what each one is actually for, and where the money part is heading.
11 min read · 20 Aug 2026
The Phantom Change Problem: Why JSON Key Ordering Became a Billing Guarantee
→If you bill when data changes, an unstable fingerprint bills for nothing. The dangerous failure is not the missed change — it is the phantom one.
9 min read · 20 Aug 2026
Polling Is a Tax: Change Detection for Platform Data
→If 97% of your hourly re-fetches return the same bytes, you are paying for 7 million requests a month that told you nothing. Three ways to stop.
12 min read · 20 Aug 2026
What Breaks When a Platform Changes Its JSON Shape
→The dangerous schema drift is not the field that disappears — your parser throws and you find out. It is the one that quietly changes meaning.
11 min read · 20 Aug 2026
Legal and risk
5 postsWhat the courts actually held, as opposed to the single sentence everyone quotes from each case — including the parts that went against the scraper. None of it is legal advice, and every post says so on its face.
Ad Libraries Are the Only Social Data a Regulator Made Public
→Every other public dataset on a platform is public by permission, and permission can be withdrawn. Ad transparency archives exist because the law requires them.
11 min read · 20 Aug 2026
Is Web Scraping Legal? What hiQ and Meta v. Bright Data Actually Held
→The two cases everyone cites, read properly — including the parts that went against the scraper, and the three questions people keep merging.
10 min read · 20 Aug 2026
Logged Out Is Doing All the Work: The Distinction That Decides Your Scraping Exposure
→Every load-bearing step in the Bright Data ruling depends on nobody being logged in. What that means for your contract risk, and for your architecture.
9 min read · 20 Aug 2026
Scraping and GDPR: \"Not a ToS Breach\" Is Not a Lawful Basis
→US scraping wins are about contracts and the CFAA. Neither is about personal data. What EU law actually requires, and what to demand from a data vendor.
10 min read · 20 Aug 2026
Rate Limit, Ban, Letter, Lawsuit: What Actually Happens When a Platform Notices You
→The escalation ladder from a 429 to litigation, rung by rung — what is documented, what is folklore, and the profile of who actually gets sued.
9 min read · 20 Aug 2026
The undocumented web
5 postsWhat each platform hands a logged-out request, measured rather than assumed, with the date and the network position recorded next to every result.
200 OK, Zero Bytes: How Platforms Say No Without Telling You
→The most effective bot detection does not return 403. It returns a successful response with nothing in it, and lets you conclude the data was never there.
10 min read · 20 Aug 2026
Nitter Is Almost Gone. Here's What Still Returns Public Tweets in 2026 (Tested)
→Nine Nitter instances probed, one answered. Plus the syndication, oEmbed and server-rendered surfaces that still work, with byte counts and a date.
12 min read · 20 Aug 2026
What You Can Actually Get From Each Social Platform Without an API Key
→Ten platforms probed on one day from one IP, with status codes and byte counts — including the rows that returned nothing at all.
12 min read · 20 Aug 2026
Minting YouTube PO Tokens in Node: Fixing 0-Byte Caption Responses
→YouTube caption URLs answer HTTP 200 with an empty body. Here is why, and the working Node recipe — including the three details nobody wrote down.
14 min read · 20 Aug 2026
YouTube Without an API Key: RSS, oEmbed, timedtext, and the Endpoints Google Doesn't Document
→The channel feed carries view and like counts nobody mentions, handle-to-channel-ID has four fixes, and captions return zero bytes. Measured 2026-08-20.
10 min read · 20 Aug 2026
What scraping actually costs
4 postsWritten from the buying side. We have no proxy to sell, which is why these can name the vendor whose endpoint did not respond and the tier that cannot run the workload it is sold for.
Ten Proxy Providers and Five Scraper APIs, Priced and Probed
→Prices per GB and per 1,000 requests for ten residential proxy vendors and five scraper APIs, with the two whose documented endpoints did not answer us.
11 min read · 20 Aug 2026
What a Residential Proxy Actually Is — and When You Don't Need One
→Residential IPs cost 10-30x datacenter IPs, and plenty of workloads need none at all. Here is the mechanism, and how to tell which case you are in.
11 min read · 20 Aug 2026
The Credit-Multiplier Trap: Why a 1M-Credit Scraping Plan Is 100,000 Requests
→Scraper APIs bill in credits and charge 10x for residential IPs — the exact IPs Instagram needs. Here is the arithmetic, with named vendors.
8 min read · 20 Aug 2026
Four Ways to Price a Scraping API, and the Workload Each One Is Right For
→Daily allowances, never-expiring credits, pay-for-success, and change-based billing are four different products. Which one wins depends on the shape of your month.
13 min read · 20 Aug 2026
Ad transparency data
2 postsThe one corner of this category that regulators made public on purpose, which changes both what you can build on it and how much risk comes with doing so.
Why Facebook Ad Library Scrapers Keep Breaking (and the Official API Doesn't)
→The Ad Library UI is a private contract between Meta's frontend and its backend. The Graph API is a public one with a deprecation policy. That is the whole difference.
11 min read · 20 Aug 2026
The Meta Ad Library API, Field by Field
→Every field, which ads populate it, the token path that blocks most people, and the security detail in ad_snapshot_url that will leak your access token.
12 min read · 20 Aug 2026
What you will not find here
No “how to scrape X with Y” tutorials. The largest competitor in this category published 2,146 of them and they return roughly three visits a month between them, so the format is not a gap in our coverage — it is a decision.
No post is structured around us, and none of the comparisons scores itself. Where a competitor does something better, the post says which thing and why.
Where the measurements come from
Every probe result carries the date it was taken and the network position it was taken from. That second half matters more than it sounds: a request that succeeds from a home connection tells you very little about the same request from a datacenter range, which is where production runs.