Skip to content
Developers

Rate limits and caps

Limits keep one key from slowing the others, and one crawl from costing more than you planned. Every limit answers with a code you can switch on.

Calls a minute

Limits count in one-minute windows. Every call counts toward the 60 a minute of its key. A heavy or start call also counts toward its own, lower limit. All the keys of a workspace together may make 240 calls a minute.

ClassCalls a minute per keyForCalls in this class
light60Every other call.
  • GET /crawls
  • GET /crawls/{crawl_id}
  • POST /crawls/{crawl_id}/cancel
  • GET /feeds
  • POST /feeds
  • DELETE /feeds/{feed_id}
  • GET /issue-types
  • GET /me
  • GET /sites
  • GET /sites/{site_id}
  • GET /sites/{site_id}/fixes
  • GET /sites/{site_id}/fixes/{fix_id}
  • GET /sites/{site_id}/pages/{page_id}
  • GET /webhook-endpoints
  • POST /webhook-endpoints
  • DELETE /webhook-endpoints/{endpoint_id}
  • GET /webhook-endpoints/{endpoint_id}
  • PATCH /webhook-endpoints/{endpoint_id}
  • GET /webhook-endpoints/{endpoint_id}/deliveries
  • POST /webhook-endpoints/{endpoint_id}/rotate-secret
  • GET /webhook-events
start10Calls that start work that costs credits.
  • POST /crawls
heavy20Calls that scan a lot of data or send events: lists of links, issues and pages, comparisons, exports, webhook tests and redeliveries.
  • GET /crawls/{crawl_id}/ai-readiness
  • GET /crawls/{crawl_id}/diff
  • GET /crawls/{crawl_id}/exports/{slug}.{file_format}
  • GET /crawls/{crawl_id}/issues
  • GET /crawls/{crawl_id}/links
  • GET /sites/{site_id}/fixes/{fix_id}/pages
  • GET /sites/{site_id}/pages
  • POST /webhook-endpoints/{endpoint_id}/deliveries/{delivery_id}/redeliver
  • POST /webhook-endpoints/{endpoint_id}/test

A presented key that does not resolve counts against the network address it came from: after 20 a minute that address is answered rate_limited too.

Heavy calls at the same time

A heavy call holds a slot while it runs, until its answer is fully sent. One workspace may have 2 heavy calls running at once. All workspaces together may have 4.

These caps are counted in each API process, not across the whole service. The API runs in several processes, and each one keeps its own count. Calls that reach different processes do not count against each other, but you cannot choose the process. Run at most 2 heavy calls at a time for one workspace.

  • When your workspace already has 2 heavy calls running in that process, the call answers 429 too_many_heavy_requests.
  • When that process already runs 4 heavy calls in all, the call answers 503 capacity_busy.

Such a call is answered at once and never queued. Both answers carry Retry-After with 5 seconds: wait that long, then repeat the call. The refused call still counts toward the calls a minute.

Headers

HeaderMeaning
X-RateLimit-LimitThe calls allowed in this minute by the tightest limit that applies to the call.
X-RateLimit-RemainingThe calls left in this minute under that limit.
X-RateLimit-ResetThe Unix time this minute ends and the counters start again.
Retry-AfterOn a 429 and a 503: the seconds to wait before the same call is tried again.

A call that is over a limit of calls a minute answers rate_limited. The rate_limit member of the problem document says which limit: key, heavy, start, workspace or network. See errors.

Crawl limits

These limits are about starting crawls. A crawl started through the MCP server counts in the same totals. Daily limits count from UTC midnight.

LimitValueAnswer when passed
Crawls one key may start a day10crawl_limit_reached
Crawls one workspace may start a day20crawl_limit_reached
Crawls one workspace may have running2too_many_running_crawls
Crawls running for everyone at once5capacity_busy
Minutes between two crawls of one site10crawl_too_soon
Pages a crawl may check10,000validation_error
Pages a crawl may check, with a key that has no cap set500validation_error
Credits the crawls of a key may cost a daythe key’s capdaily_credit_cap_reached

GET /crawls/{crawl_id} takes wait, the seconds to hold the call while the crawl runs, up to 25. See the daily credit cap.

Idempotency

Send Idempotency-Key with POST /crawls so a retry cannot start a second crawl. The value is 1 to 255 characters. Use something unique for each crawl you mean to start, such as a commit hash.

  • The same key and the same body within 24 hours return the crawl the first call started. The answer is 200 with Idempotent-Replayed: true, and nothing new is started or charged.
  • The same key with another body answers idempotency_key_reused.
  • The same key while the first call is still being handled answers idempotency_in_progress. Try again shortly.

Other limits

  • A query that runs longer than 15 seconds is cancelled and answers query_too_slow. Narrow the filters or lower the page size.
  • An export holds at most 250,000 rows as CSV and 150,000 as Excel. A larger report is refused with export_too_large. It is never cut.
  • A live link holds at most 50,000 rows and 2,000,000 cells. A workspace can have 20 links. Each link may be read 12 times an hour, and stops working after 365 days.
  • A workspace can have 5 webhook endpoints and 5 active keys.