Skip to the content
EnrichedNowDocs

    Type to search every page.

    Limits

    The allowance, concurrent runs, page sizes, retention and input limits.

    The numbers that bound what you can do, in one place.

    Account limits#

    LimitDefaultWhen you reach it
    Unit allowance1,000 units a calendar month (UTC)Starting a run is refused: 402 quota_exceeded.
    Runs going at the same time2Starting a run is refused: 429 too_many_runs. Wait for a run to end.

    Both are set per account and can be changed for an account.

    The console shows your account's own numbers; allowance_units is also in the answer of GET /v1/usage.

    Request limits#

    WhatLimit
    wait on POST /v1/scrapers/{id}/runs0 to 60 seconds; fractions are accepted. Default 0.
    limit on GET /v1/runs1 to 100 runs per page. Default 20.
    limit on GET /v1/runs/{id}/records1 to 1000 records per page. Default 100.
    The period of GET /v1/usageAt most 366 days.
    Sign-up and log-in attempts20 a minute. More are answered 429 rate_limited with a Retry-After header.

    A value outside its range is refused with 422 invalid_input. Calls made with an API key are not rate limited; what bounds them is the number of runs going at the same time.

    Run limits#

    WhatLimit
    How long a run may take5 minutes. A run that reaches it ends partial or failed with the reason deadline_exceeded.

    Longer lists are read page by page with cursor — see Fetch the records.

    Retention#

    WhatKept for
    A run's records7 days after the run ended
    The run itself — status, reason, counts, diagnosticsKept
    A console session30 days

    Fetch records within 7 days of the run's end and store what you need. After that the records endpoint answers with an empty list and "expired": true — see Fetch the records.

    Input limits#

    Each scraper states the limits of its own input. The limits on each scraper page are generated from the scraper's schema and are always current; the ones asked about most are here. A value outside its limit is refused with 422 invalid_input before any run is created.

    careers.company and ats.board:

    FieldLimit
    max_detail_pages0 to 200. Default 30.
    board_url, careers_urlAt most 2,000 characters.
    domainAt most 253 characters.
    company_nameAt most 200 characters.

    linkedin.jobs_search:

    FieldLimit
    max_results1 to 1000. Default 25. LinkedIn serves at most 1,000 results for one search.
    posted_within_hours1 to 720.
    keywords, locationAt most 200 characters each.

    web.fetch:

    WhatLimit
    methodGET only. No request body can be sent.
    urlAt most 2,048 characters; http or https; port 80, 443, 8080 or 8443.
    headersAt most 10, from the names on the scraper's page. Each value at most 1,024 characters. No cookie and no Authorization header.
    max_bytes1 to 10000000. Default 2000000 (2 MB): the body is cut after that, and the record's truncated is true.
    timeout_s1 to 120 seconds for the whole fetch. Default 30.
    Redirects followedAt most 5, each to a public web address.
    JavaScriptNot run. The body is what the server sent.

    What a target may be#

    A scraper reads public web pages. An input is refused when the target is:

    • an IP address rather than a host name;
    • a name that only resolves inside a network — localhost, or one ending in .local, .internal, .lan, .home, .corp, .test, .example, .invalid, .onion or .arpa;
    • not an http or https address;
    • an address that carries a login (https://user:password@…).

    We do not log in to sites and we do not read pages behind a login.

    What we cannot read yet#

    Pages that only show their content after running JavaScript. A careers run that meets one ends unsupported — or partial, when other pages of the site gave records — with the reason render_required, and an unsupported run costs nothing. web.fetch returns such a page as the server sent it, before any script has run. How we fetch explains why.