Skip to the content
EnrichedNowDocs

    Type to search every page.

    Running a scraper

    Start a run, wait for it or poll it, page through its records, cancel it.

    A run is one execution of one scraper with one input. You start it, it works in the background, and it ends with a status and a set of records. This page covers the whole life of a run.

    Start a run#

    Shell
    curl -X POST "https://api.enrichednow.com/v1/scrapers/careers.company/runs" \
      -H "Authorization: Bearer $ENRICHEDNOW_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"input": {"domain": "acme.com"}}'

    The body is always {"input": {...}}. What goes inside input depends on the scraper: each scraper page lists its fields, and GET /v1/scrapers/{id} returns the same thing as JSON Schema.

    Before the run is created, the API checks three things. Any of them stops the request, and no run exists afterwards:

    CheckIf it fails
    The input fits the scraper's schema422 invalid_input, with the field errors in details.fields
    Your account has allowance left402 quota_exceeded
    You are under your concurrent-run limit429 too_many_runs

    Otherwise the run is stored as queued and you get it back with 202:

    202 · JSON
    {
      "id": "run_eb3c3302a5eb6987c17f3b53",
      "scraper_id": "careers.company",
      "scraper_version": 1,
      "status": "queued",
      "reason": null,
      "input": {
        "domain": "example.com"
      },
      "record_count": 0,
      "units": 0,
      "diagnostics": null,
      "created_at": "2026-11-16T20:27:31.253846Z",
      "started_at": null,
      "finished_at": null
    }

    Wait for the result in the same request#

    Add wait — a number of seconds from 0 to 60, default 0; fractions such as 0.5 are accepted — and the API holds the request open until the run ends or the time is up, whichever comes first. A wait outside that range is refused with 422 invalid_input.

    Shell
    curl -X POST "https://api.enrichednow.com/v1/scrapers/careers.company/runs?wait=60" \
      -H "Authorization: Bearer $ENRICHEDNOW_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"input": {"domain": "acme.com"}}'
    The run…You get
    ended within wait200 and the finished run
    is still going when wait runs out202 and the run as it is now (queued or running)

    wait changes only how long the request stays open. The run is the same run either way: when wait runs out the run carries on, and you pick it up by polling. This is a 202 after a wait that ran out — the run is running, and has no diagnostics or finished_at yet:

    202 · JSON
    {
      "id": "run_a3c17df7e3858dc72370229f",
      "scraper_id": "careers.company",
      "scraper_version": 1,
      "status": "running",
      "reason": null,
      "input": {
        "domain": "example.com"
      },
      "record_count": 0,
      "units": 0,
      "diagnostics": null,
      "created_at": "2026-11-16T20:27:30.741318Z",
      "started_at": "2026-11-16T20:27:30.744947Z",
      "finished_at": null
    }

    Set your HTTP client's timeout comfortably above wait — 90 seconds for wait=60.

    How long a run takes depends on the site. The careers scrapers read several pages per run; a board with a public feed can finish in a second or two, a company site read page by page takes longer. Write your code so that a 202 is ordinary.

    Poll a run#

    GET /v1/runs/{id} returns the run as it is now. Ask until status is no longer queued or running.

    Shell
    curl "https://api.enrichednow.com/v1/runs/run_6cce2b12aaf37a10b5e1b885" \
      -H "Authorization: Bearer $ENRICHEDNOW_API_KEY"

    A pause of a couple of seconds between polls is plenty.

    Read the result#

    When the run has ended, three fields tell you what happened:

    • status — ok, partial, failed, unsupported or cancelled.
    • reason — a stable code whenever the status is not ok; null when it is.
    • record_count — how many records the run stored.

    ok with record_count 0 is a real answer: the pages were read and there were no openings.

    Decide what to do from status and reason; Results and statuses lists every one.

    Fetch the records#

    Records exist for runs that ended ok or partial, and for a cancelled run that had found some before it stopped. For any other run the list is empty.

    Shell
    curl "https://api.enrichednow.com/v1/runs/run_6cce2b12aaf37a10b5e1b885/records?limit=1000" \
      -H "Authorization: Bearer $ENRICHEDNOW_API_KEY"
    ParameterMeaning
    limitRecords per page: 1 to 1000, default 100.
    cursorThe next_cursor of the page before. Leave it out for the first page.

    Records come in the order the run stored them. The first page of a longer run, asked for with limit=2 (the records themselves are left out here):

    200 · JSON
    {
      "records": [
        {
          "…": "a record"
        },
        {
          "…": "a record"
        }
      ],
      "next_cursor": "Mg"
    }

    Keep asking with the next_cursor you were given until it is null. A cursor is an opaque string: pass it back unchanged, do not build or edit one. The Quickstart has the loop in Python and Node.

    The shape of a record is the scraper's record_schema, shown on its scraper page.

    Once the records are gone, the same request still answers 200, with an empty list and expired set. expired appears only then:

    200 · JSON
    {
      "records": [],
      "next_cursor": null,
      "expired": true
    }

    List your runs#

    Shell
    curl "https://api.enrichednow.com/v1/runs?scraper_id=careers.company&status=partial&limit=50" \
      -H "Authorization: Bearer $ENRICHEDNOW_API_KEY"
    ParameterMeaning
    limitRuns per page: 1 to 100, default 20.
    cursorThe next_cursor of the page before.
    scraper_idOnly runs of this scraper.
    statusOnly runs with this status. A value that is not a status is refused with 422.

    The answer is {"runs": [...], "next_cursor": ...}, newest first, paged the same way as records.

    Cancel a run#

    Shell
    curl -X POST "https://api.enrichednow.com/v1/runs/run_6cce2b12aaf37a10b5e1b885/cancel" \
      -H "Authorization: Bearer $ENRICHEDNOW_API_KEY"

    The answer is the run, as it is at that moment:

    • A run that was still queued ends cancelled at once and never starts.
    • A run that is running is asked to stop. The answer usually still says running; the run stops at its next safe point. Poll it to see how it ended.
    • A run that has already ended is returned unchanged.

    Cancelling is cooperative: a run that was already finishing may still complete — so after a cancel, read the status the run actually ended with:

    Status after a cancelMeaningCharged
    cancelledThe run stopped. reason is cancelled. Records it had found before the stop are kept: record_count says how many, and they can be fetched as usual.No — units is 0, even with records
    ok or partialThe run was already finishing and completed.Yes, as usual
    failed or unsupportedThe run was already finishing and ended this way on its own.No

    Here a run was cancelled a moment after it started. The answer to the cancel still shows it running:

    200 · JSON
    {
      "id": "run_eb3c3302a5eb6987c17f3b53",
      "scraper_id": "careers.company",
      "scraper_version": 1,
      "status": "running",
      "reason": null,
      "input": {
        "domain": "example.com"
      },
      "record_count": 0,
      "units": 0,
      "diagnostics": null,
      "created_at": "2026-11-16T20:27:31.253846Z",
      "started_at": "2026-11-16T20:27:31.256121Z",
      "finished_at": null
    }

    That run was already finishing, and ended on its own a moment later:

    200 · JSON
    {
      "id": "run_eb3c3302a5eb6987c17f3b53",
      "scraper_id": "careers.company",
      "scraper_version": 1,
      "status": "unsupported",
      "reason": "temporarily_unavailable",
      "input": {
        "domain": "example.com"
      },
      "record_count": 0,
      "units": 0,
      "diagnostics": {
        "requests": 0,
        "bytes": 0,
        "wall_ms": 370,
        "facts": {
          "discovery": "none",
          "pages_read": 0,
          "renders_refused": 0
        }
      },
      "created_at": "2026-11-16T20:27:31.253846Z",
      "started_at": "2026-11-16T20:27:31.256121Z",
      "finished_at": "2026-11-16T20:27:31.628248Z"
    }

    Scraper versions#

    Every run records the scraper_version it ran with. A scraper's version is a whole number that goes up when the scraper changes; GET /v1/scrapers/{id} shows the current one.