Scraper · Web
web.fetch
Fetch one public web page: give its address, get the status, headers and body the server sent.
- Id
web.fetch- Version
- 1
- Billed per
- request — 1 unit for a charged run
- Runs
- within one request
Start it with POST /v1/scrapers/web.fetch/runs. How runs work is in Running a scraper; what a run costs is in Usage and units.
Inputs
The fields of input. A field that is not listed here is refused.
| Field | Type | Description |
|---|---|---|
urlrequired | string | The page's address: http or https, on a public host name.Limit: at most 2048 characters. |
method | "GET" | GET is the only method.Default: "GET". |
headers | object of string values | Request headers to send, from: Accept, Accept-Language, User-Agent, Referer, Cache-Control, Pragma, If-Modified-Since, If-None-Match, DNT, X-Requested-With. |
follow_redirects | boolean | Follow up to 5 redirects; false returns the redirect itself.Default: true. |
timeout_s | number | Seconds for the whole fetch, redirects included.Limit: 1 to 120. Default: 30. |
max_bytes | integer | The body is cut after this many bytes (decoded); truncated says when it was.Limit: 1 to 10000000. Default: 2000000. |
GETis the only method. There is no request body, and no cookie orAuthorizationheader can be sent:headerstakes at most 10 headers, from the names listed above only.- No JavaScript is run.
bodyis what the server sent to a plain request; a page that builds its content in the browser comes back as the shell it starts from. - A page the site itself answers with
404or500is a successful fetch: the run endsok, the record carries thatstatus, and the run is charged like any otherok. - A site that refuses the request ends the run
failed, with no record and no charge:blockedfor a403, a999or a challenge page,rate_limitedfor a429,captchafor a CAPTCHA. A403is always read as a refusal. See [the reasons of this family](/results-and-statuses/#web). - The body is cut after
max_bytes— 2,000,000 by default (2 MB), 10,000,000 at most — andtruncatedistruewhen it was. - A body that is text is given as text, decoded with the charset in
charset. Anything else (an image, a PDF) is given as base64, andbody_encodingsays which. urlmust be anhttporhttpsaddress on a public host name, on port 80, 443, 8080 or 8443, with no login in it. Up to 5 redirects are followed, and each one is held to the same rules.headerson the record are the response headers that describe the page.Set-Cookieis not among them.
Record fields
Every record of a run has these fields. A value that is not known is null — never an empty string and never 0.
| Field | Type | Notes |
|---|---|---|
url | string | The address that was asked for. |
final_url | string | The address that answered, after redirects. |
status | integer | The HTTP status of the answer (a 404 or 500 is still the page). |
headers | object of string values | The response headers that describe the page, names in lower case. |
content_type | string or null | The media type, without parameters, e.g. text/html. |
charset | string or null | The charset the text was decoded with; null for a base64 body. |
body | string | The body: text, or base64 when body_encoding says so. |
body_encoding | "text" | "base64" | base64 for a body that is not text (an image, a PDF). |
bytes | integer | The size of the returned body in bytes, before text decoding or base64. |
truncated | boolean | True when the server sent more than max_bytes and the body was cut. |
redirects | array of string | The addresses the redirects led to, in order. |
fetched_at | string (date-time) | Always present. |
Example request
curl -X POST "https://api.enrichednow.com/v1/scrapers/web.fetch/runs?wait=60" \
-H "Authorization: Bearer $ENRICHEDNOW_API_KEY" \
-H "Content-Type: application/json" \
-d '{"input":{"url":"https://example.com/"}}'The body on its own:
{
"input": {
"url": "https://example.com/"
}
}Example response
The run, once it has ended. A run that has not ended within wait comes back with 202 and the status queued or running.
{
"id": "run_6cce2b12aaf37a10b5e1b885",
"scraper_id": "web.fetch",
"scraper_version": 1,
"status": "ok",
"reason": null,
"input": {
"url": "https://example.com/"
},
"record_count": 1,
"units": 1,
"diagnostics": {
"requests": 1,
"bytes": 232779,
"wall_ms": 565,
"facts": {}
},
"created_at": "2026-11-16T20:27:29.009834Z",
"started_at": "2026-11-16T20:27:29.013803Z",
"finished_at": "2026-11-16T20:27:29.582787Z"
}Its records, from GET /v1/runs/{id}/records (the values are an illustration; the fields are the ones above):
{
"records": [
{
"url": "https://example.com/",
"final_url": "https://example.com/",
"status": 200,
"headers": {
"cache-control": "max-age=86000",
"content-type": "text/html",
"date": "Tue, 17 Nov 2026 09:30:01 GMT",
"etag": "\"84238dfc8092e5d9c0dac8ef93371a07:1736799080.121134\"",
"last-modified": "Mon, 13 Jan 2025 20:11:20 GMT"
},
"content_type": "text/html",
"charset": "utf-8",
"body": "<!doctype html>\n<html>\n<head>\n <title>Example Domain</title>\n…",
"body_encoding": "text",
"bytes": 1256,
"truncated": false,
"redirects": [],
"fetched_at": "2026-11-17T09:30:01Z"
}
],
"next_cursor": null
}A run does not always end ok. Results and statuses lists every status and the reasons every scraper shares; the reasons of the Web scrapers are in their own section there.