Skip to the content
EnrichedNowDocs

    Type to search every page.

    Scraper · Web

    web.fetch

    Fetch one public web page: give its address, get the status, headers and body the server sent.

    Id
    web.fetch
    Version
    1
    Billed per
    request — 1 unit for a charged run
    Runs
    within one request

    Start it with POST /v1/scrapers/web.fetch/runs. How runs work is in Running a scraper; what a run costs is in Usage and units.

    Inputs

    The fields of input. A field that is not listed here is refused.

    Inputs of web.fetch
    FieldTypeDescription
    urlrequiredstringThe page's address: http or https, on a public host name.Limit: at most 2048 characters.
    method"GET"GET is the only method.Default: "GET".
    headersobject of string valuesRequest headers to send, from: Accept, Accept-Language, User-Agent, Referer, Cache-Control, Pragma, If-Modified-Since, If-None-Match, DNT, X-Requested-With.
    follow_redirectsbooleanFollow up to 5 redirects; false returns the redirect itself.Default: true.
    timeout_snumberSeconds for the whole fetch, redirects included.Limit: 1 to 120. Default: 30.
    max_bytesintegerThe body is cut after this many bytes (decoded); truncated says when it was.Limit: 1 to 10000000. Default: 2000000.
    • GET is the only method. There is no request body, and no cookie or Authorization header can be sent: headers takes at most 10 headers, from the names listed above only.
    • No JavaScript is run. body is what the server sent to a plain request; a page that builds its content in the browser comes back as the shell it starts from.
    • A page the site itself answers with 404 or 500 is a successful fetch: the run ends ok, the record carries that status, and the run is charged like any other ok.
    • A site that refuses the request ends the run failed, with no record and no charge: blocked for a 403, a 999 or a challenge page, rate_limited for a 429, captcha for a CAPTCHA. A 403 is always read as a refusal. See [the reasons of this family](/results-and-statuses/#web).
    • The body is cut after max_bytes — 2,000,000 by default (2 MB), 10,000,000 at most — and truncated is true when it was.
    • A body that is text is given as text, decoded with the charset in charset. Anything else (an image, a PDF) is given as base64, and body_encoding says which.
    • url must be an http or https address on a public host name, on port 80, 443, 8080 or 8443, with no login in it. Up to 5 redirects are followed, and each one is held to the same rules.
    • headers on the record are the response headers that describe the page. Set-Cookie is not among them.

    Record fields

    Every record of a run has these fields. A value that is not known is null — never an empty string and never 0.

    Record fields of web.fetch
    FieldTypeNotes
    urlstringThe address that was asked for.
    final_urlstringThe address that answered, after redirects.
    statusintegerThe HTTP status of the answer (a 404 or 500 is still the page).
    headersobject of string valuesThe response headers that describe the page, names in lower case.
    content_typestring or nullThe media type, without parameters, e.g. text/html.
    charsetstring or nullThe charset the text was decoded with; null for a base64 body.
    bodystringThe body: text, or base64 when body_encoding says so.
    body_encoding"text" | "base64"base64 for a body that is not text (an image, a PDF).
    bytesintegerThe size of the returned body in bytes, before text decoding or base64.
    truncatedbooleanTrue when the server sent more than max_bytes and the body was cut.
    redirectsarray of stringThe addresses the redirects led to, in order.
    fetched_atstring (date-time)Always present.

    Example request

    Shell
    curl -X POST "https://api.enrichednow.com/v1/scrapers/web.fetch/runs?wait=60" \
      -H "Authorization: Bearer $ENRICHEDNOW_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"input":{"url":"https://example.com/"}}'

    The body on its own:

    JSON
    {
      "input": {
        "url": "https://example.com/"
      }
    }

    Example response

    The run, once it has ended. A run that has not ended within wait comes back with 202 and the status queued or running.

    JSON
    {
      "id": "run_6cce2b12aaf37a10b5e1b885",
      "scraper_id": "web.fetch",
      "scraper_version": 1,
      "status": "ok",
      "reason": null,
      "input": {
        "url": "https://example.com/"
      },
      "record_count": 1,
      "units": 1,
      "diagnostics": {
        "requests": 1,
        "bytes": 232779,
        "wall_ms": 565,
        "facts": {}
      },
      "created_at": "2026-11-16T20:27:29.009834Z",
      "started_at": "2026-11-16T20:27:29.013803Z",
      "finished_at": "2026-11-16T20:27:29.582787Z"
    }

    Its records, from GET /v1/runs/{id}/records (the values are an illustration; the fields are the ones above):

    JSON
    {
      "records": [
        {
          "url": "https://example.com/",
          "final_url": "https://example.com/",
          "status": 200,
          "headers": {
            "cache-control": "max-age=86000",
            "content-type": "text/html",
            "date": "Tue, 17 Nov 2026 09:30:01 GMT",
            "etag": "\"84238dfc8092e5d9c0dac8ef93371a07:1736799080.121134\"",
            "last-modified": "Mon, 13 Jan 2025 20:11:20 GMT"
          },
          "content_type": "text/html",
          "charset": "utf-8",
          "body": "<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n…",
          "body_encoding": "text",
          "bytes": 1256,
          "truncated": false,
          "redirects": [],
          "fetched_at": "2026-11-17T09:30:01Z"
        }
      ],
      "next_cursor": null
    }

    A run does not always end ok. Results and statuses lists every status and the reasons every scraper shares; the reasons of the Web scrapers are in their own section there.