Errors
The error shape, every error code, and how to handle each.
There are two different kinds of "it did not work", and they are reported in two different places.
| Where you see it | Example | |
|---|---|---|
| The request failed | An HTTP status of 4xx or 5xx and an error body. This page. | A bad key, an input that does not fit the schema, no allowance left. |
| The scrape failed | A run with status failed or unsupported, returned with 200. | The site did not answer; the page needs JavaScript. See Results and statuses. |
The error shape#
Every error has the same body, and an HTTP status that matches its code:
{
"error": {
"code": "not_found",
"message": "There is no run 'run_000000000000000000000000'.",
"details": {}
}
}| Field | Meaning |
|---|---|
code | A stable snake_case code. Match on this. |
message | A sentence for a person. Show it or log it; do not parse it — the wording can change. |
details | Always an object. Empty, except for the three codes below that have more to say. |
What details carries:
| Code | details |
|---|---|
invalid_input | fields: a list of {"path", "message"}, one per problem. path is where the problem is, as you wrote it: input.board_url, wait, limit. |
quota_exceeded | used_units and allowance_units. |
too_many_runs | active_runs and max_concurrent_runs. |
Error codes#
| Code | HTTP | When | What to do |
|---|---|---|---|
unauthorized | 401 | The API key is missing, mistyped or revoked — or a console call was made without a session. | Check the Authorization: Bearer … header. Create a new key if the old one was revoked. |
forbidden | 403 | A console call that changes something was made without the X-CSRF-Token header. Calls made with an API key never get this. | Send the header — see Console sessions. |
not_found | 404 | There is no such scraper, run or key, or nothing at this address. A scraper that is not publicly available, a run of another account, and a method an address does not have answer this too. | Check the id and the address. Scraper ids are listed at GET /v1/scrapers. |
invalid_input | 422 | The request does not fit: the scraper's input, a query parameter, or the body itself. | Read details.fields, fix the request. Do not retry unchanged. |
quota_exceeded | 402 | Your account's unit allowance for this month is used up. No run was created. | Wait for the next calendar month, or contact us. See Usage and units. |
too_many_runs | 429 | You are at your limit of runs going at the same time. No run was created. | Wait for one of your runs to end, then send the request again. |
rate_limited | 429 | Too many sign-up or log-in attempts in a minute. API calls made with a key are not rate limited. | Wait the number of seconds in the Retry-After header. |
internal | 500 | Something broke on our side. | Try again after a pause. If it repeats, tell us what you sent. |
Invalid input#
The scraper's input is checked before anything else happens — no run is created and nothing is charged.
curl -X POST "https://api.enrichednow.com/v1/scrapers/careers.company/runs" \
-H "Authorization: Bearer $ENRICHEDNOW_API_KEY" \
-H "Content-Type: application/json" \
-d '{"input": {"domain": "acme.com", "careers_url": "https://acme.com/careers"}}'This is refused with 422 invalid_input: careers.company takes exactly one of domain and careers_url.
{
"error": {
"code": "invalid_input",
"message": "The input is not valid.",
"details": {
"fields": [
{
"path": "input",
"message": "give exactly one of domain and careers_url"
}
]
}
}
}Every problem is listed, each with its path. This request to ats.board left out board_url, sent a field the
scraper does not have (board), and a max_detail_pages of 500:
{
"error": {
"code": "invalid_input",
"message": "The input is not valid.",
"details": {
"fields": [
{
"path": "input.board_url",
"message": "Field required"
},
{
"path": "input.max_detail_pages",
"message": "Input should be less than or equal to 200"
},
{
"path": "input.board",
"message": "Extra inputs are not permitted"
}
]
}
}
}Things that make an input invalid:
- a required field is missing, or a field has the wrong type;
- a field the scraper does not have — unknown fields are refused, not ignored;
- a value outside its limits, such as
max_detail_pagesabove200; - a target that is not a public web address: an IP address, a name such as
localhostor one ending in.localor.internal, a scheme other thanhttporhttps, or an address with a login in it.
Each scraper page lists the fields and their limits.
A query parameter out of range is the same error, with the parameter's name as the path:
{
"fields": [
{
"path": "wait",
"message": "Input should be less than or equal to 60"
}
]
}No allowance left, and too many runs#
Both stop a run from being created, and both say where you stand in details.
{
"error": {
"code": "quota_exceeded",
"message": "This month's units are used up.",
"details": {
"used_units": 1,
"allowance_units": 1
}
}
}{
"error": {
"code": "too_many_runs",
"message": "The account already has as many runs in progress as it may; wait for one to end.",
"details": {
"active_runs": 2,
"max_concurrent_runs": 2
}
}
}A key that does not work#
A request without a key, and one with a key that is mistyped or revoked, are both 401 unauthorized; only the
message differs.
{
"error": {
"code": "unauthorized",
"message": "Send an API key (Authorization: Bearer <key>) or log in.",
"details": {}
}
}{
"error": {
"code": "unauthorized",
"message": "This API key is not valid.",
"details": {}
}
}Too many log-in attempts#
Signing up and logging in are limited to 20 attempts a minute, counted per client address and — for logging in —
per email as well. One attempt too many is answered 429 with a Retry-After header, in seconds:
POST /v1/auth/login
HTTP 429
Retry-After: 60{
"error": {
"code": "rate_limited",
"message": "Too many attempts; try again in a minute.",
"details": {}
}
}Which errors to retry#
| Code | Retry? |
|---|---|
internal | Yes, after a pause that grows with each attempt. Stop after a few attempts. |
rate_limited | Yes, after the seconds in Retry-After. |
too_many_runs | Yes, once one of your runs has ended. |
quota_exceeded | Not until the allowance renews. |
unauthorized, forbidden, not_found, invalid_input | No. The same request will get the same answer. |
Take care when retrying POST /v1/scrapers/{id}/runs after a timeout or a 500: the run may have been created.
Look at GET /v1/runs first, or you may start — and pay for — the same work twice.
import time
import requests
RETRY = {"internal"}
def get(url: str, **kwargs) -> dict:
"""A GET with retries. Reading is safe to repeat; starting a run is not."""
for attempt in range(4):
response = requests.get(url, timeout=30, **kwargs)
if response.ok:
return response.json()
error = response.json()["error"]
if error["code"] not in RETRY or attempt == 3:
raise RuntimeError(f"{error['code']}: {error['message']}")
time.sleep(2 ** attempt)
raise AssertionError("unreachable")