Running a scraper
Start a run, wait for it or poll it, page through its records, cancel it.
A run is one execution of one scraper with one input. You start it, it works in the background, and it ends with a status and a set of records. This page covers the whole life of a run.
Start a run#
curl -X POST "https://api.enrichednow.com/v1/scrapers/careers.company/runs" \
-H "Authorization: Bearer $ENRICHEDNOW_API_KEY" \
-H "Content-Type: application/json" \
-d '{"input": {"domain": "acme.com"}}'The body is always {"input": {...}}. What goes inside input depends on the scraper: each
scraper page lists its fields, and GET /v1/scrapers/{id} returns the same thing as JSON Schema.
Before the run is created, the API checks three things. Any of them stops the request, and no run exists afterwards:
| Check | If it fails |
|---|---|
| The input fits the scraper's schema | 422 invalid_input, with the field errors in details.fields |
| Your account has allowance left | 402 quota_exceeded |
| You are under your concurrent-run limit | 429 too_many_runs |
Otherwise the run is stored as queued and you get it back with 202:
{
"id": "run_eb3c3302a5eb6987c17f3b53",
"scraper_id": "careers.company",
"scraper_version": 1,
"status": "queued",
"reason": null,
"input": {
"domain": "example.com"
},
"record_count": 0,
"units": 0,
"diagnostics": null,
"created_at": "2026-11-16T20:27:31.253846Z",
"started_at": null,
"finished_at": null
}Wait for the result in the same request#
Add wait — a number of seconds from 0 to 60, default 0; fractions such as 0.5 are accepted — and the API
holds the request open until the run ends or the time is up, whichever comes first. A wait outside that range is
refused with 422 invalid_input.
curl -X POST "https://api.enrichednow.com/v1/scrapers/careers.company/runs?wait=60" \
-H "Authorization: Bearer $ENRICHEDNOW_API_KEY" \
-H "Content-Type: application/json" \
-d '{"input": {"domain": "acme.com"}}'| The run… | You get |
|---|---|
ended within wait | 200 and the finished run |
is still going when wait runs out | 202 and the run as it is now (queued or running) |
wait changes only how long the request stays open. The run is the same run either way: when wait runs out the
run carries on, and you pick it up by polling. This is a 202 after a wait that ran out — the run is running,
and has no diagnostics or finished_at yet:
{
"id": "run_a3c17df7e3858dc72370229f",
"scraper_id": "careers.company",
"scraper_version": 1,
"status": "running",
"reason": null,
"input": {
"domain": "example.com"
},
"record_count": 0,
"units": 0,
"diagnostics": null,
"created_at": "2026-11-16T20:27:30.741318Z",
"started_at": "2026-11-16T20:27:30.744947Z",
"finished_at": null
}Set your HTTP client's timeout comfortably above wait — 90 seconds for wait=60.
How long a run takes depends on the site. The careers scrapers read several pages per run; a board with a public
feed can finish in a second or two, a company site read page by page takes longer. Write your code so that a 202
is ordinary.
Poll a run#
GET /v1/runs/{id} returns the run as it is now. Ask until status is no longer queued or running.
curl "https://api.enrichednow.com/v1/runs/run_6cce2b12aaf37a10b5e1b885" \
-H "Authorization: Bearer $ENRICHEDNOW_API_KEY"import os
import time
import requests
API = "https://api.enrichednow.com"
HEADERS = {"Authorization": f"Bearer {os.environ['ENRICHEDNOW_API_KEY']}"}
ACTIVE = {"queued", "running"}
def run_and_wait(scraper_id: str, scraper_input: dict, give_up_after: float = 600) -> dict:
response = requests.post(
f"{API}/v1/scrapers/{scraper_id}/runs",
params={"wait": 60},
headers=HEADERS,
json={"input": scraper_input},
timeout=90,
)
response.raise_for_status()
run = response.json()
deadline = time.monotonic() + give_up_after
while run["status"] in ACTIVE and time.monotonic() < deadline:
time.sleep(2)
response = requests.get(f"{API}/v1/runs/{run['id']}", headers=HEADERS, timeout=30)
response.raise_for_status()
run = response.json()
return run
run = run_and_wait("careers.company", {"domain": "acme.com"})
print(run["status"], run["reason"], run["record_count"])const API = "https://api.enrichednow.com";
const headers = {
Authorization: `Bearer ${process.env.ENRICHEDNOW_API_KEY}`,
"Content-Type": "application/json",
};
const ACTIVE = new Set(["queued", "running"]);
async function call(url, init) {
const response = await fetch(url, { headers, ...init });
if (!response.ok) throw new Error(JSON.stringify(await response.json()));
return response.json();
}
async function runAndWait(scraperId, input, giveUpAfterMs = 600_000) {
let run = await call(`${API}/v1/scrapers/${scraperId}/runs?wait=60`, {
method: "POST",
body: JSON.stringify({ input }),
});
const deadline = Date.now() + giveUpAfterMs;
while (ACTIVE.has(run.status) && Date.now() < deadline) {
await new Promise((resolve) => setTimeout(resolve, 2000));
run = await call(`${API}/v1/runs/${run.id}`);
}
return run;
}
const run = await runAndWait("careers.company", { domain: "acme.com" });
console.log(run.status, run.reason, run.record_count);A pause of a couple of seconds between polls is plenty.
Read the result#
When the run has ended, three fields tell you what happened:
status—ok,partial,failed,unsupportedorcancelled.reason— a stable code whenever the status is notok;nullwhen it is.record_count— how many records the run stored.
ok with record_count 0 is a real answer: the pages were read and there were no openings.
Decide what to do from status and reason; Results and statuses lists every one.
Fetch the records#
Records exist for runs that ended ok or partial, and for a cancelled run that had found some before it
stopped. For any other run the list is empty.
curl "https://api.enrichednow.com/v1/runs/run_6cce2b12aaf37a10b5e1b885/records?limit=1000" \
-H "Authorization: Bearer $ENRICHEDNOW_API_KEY"| Parameter | Meaning |
|---|---|
limit | Records per page: 1 to 1000, default 100. |
cursor | The next_cursor of the page before. Leave it out for the first page. |
Records come in the order the run stored them. The first page of a longer run, asked for with limit=2 (the
records themselves are left out here):
{
"records": [
{
"…": "a record"
},
{
"…": "a record"
}
],
"next_cursor": "Mg"
}Keep asking with the next_cursor you were given until it is null. A cursor is an opaque string: pass it back
unchanged, do not build or edit one. The Quickstart has the loop in Python
and Node.
The shape of a record is the scraper's record_schema, shown on its scraper page.
Once the records are gone, the same request still answers 200, with an empty list and expired set. expired
appears only then:
{
"records": [],
"next_cursor": null,
"expired": true
}List your runs#
curl "https://api.enrichednow.com/v1/runs?scraper_id=careers.company&status=partial&limit=50" \
-H "Authorization: Bearer $ENRICHEDNOW_API_KEY"| Parameter | Meaning |
|---|---|
limit | Runs per page: 1 to 100, default 20. |
cursor | The next_cursor of the page before. |
scraper_id | Only runs of this scraper. |
status | Only runs with this status. A value that is not a status is refused with 422. |
The answer is {"runs": [...], "next_cursor": ...}, newest first, paged the same way as records.
Cancel a run#
curl -X POST "https://api.enrichednow.com/v1/runs/run_6cce2b12aaf37a10b5e1b885/cancel" \
-H "Authorization: Bearer $ENRICHEDNOW_API_KEY"The answer is the run, as it is at that moment:
- A run that was still
queuedendscancelledat once and never starts. - A run that is
runningis asked to stop. The answer usually still saysrunning; the run stops at its next safe point. Poll it to see how it ended. - A run that has already ended is returned unchanged.
Cancelling is cooperative: a run that was already finishing may still complete — so after a cancel, read the status the run actually ended with:
| Status after a cancel | Meaning | Charged |
|---|---|---|
cancelled | The run stopped. reason is cancelled. Records it had found before the stop are kept: record_count says how many, and they can be fetched as usual. | No — units is 0, even with records |
ok or partial | The run was already finishing and completed. | Yes, as usual |
failed or unsupported | The run was already finishing and ended this way on its own. | No |
Here a run was cancelled a moment after it started. The answer to the cancel still shows it running:
{
"id": "run_eb3c3302a5eb6987c17f3b53",
"scraper_id": "careers.company",
"scraper_version": 1,
"status": "running",
"reason": null,
"input": {
"domain": "example.com"
},
"record_count": 0,
"units": 0,
"diagnostics": null,
"created_at": "2026-11-16T20:27:31.253846Z",
"started_at": "2026-11-16T20:27:31.256121Z",
"finished_at": null
}That run was already finishing, and ended on its own a moment later:
{
"id": "run_eb3c3302a5eb6987c17f3b53",
"scraper_id": "careers.company",
"scraper_version": 1,
"status": "unsupported",
"reason": "temporarily_unavailable",
"input": {
"domain": "example.com"
},
"record_count": 0,
"units": 0,
"diagnostics": {
"requests": 0,
"bytes": 0,
"wall_ms": 370,
"facts": {
"discovery": "none",
"pages_read": 0,
"renders_refused": 0
}
},
"created_at": "2026-11-16T20:27:31.253846Z",
"started_at": "2026-11-16T20:27:31.256121Z",
"finished_at": "2026-11-16T20:27:31.628248Z"
}Scraper versions#
Every run records the scraper_version it ran with. A scraper's version is a whole number that goes up when the
scraper changes; GET /v1/scrapers/{id} shows the current one.