Data feed
Pulls and freshness
Check the data feed's status before you download, re-download and resume for free, and handle the 24-hour pull interval.
A pull is one download of the feed file. Your key may pull once every 24 hours. Checking the status, asking whether the file has changed, and re-downloading a file you already pulled are free.
Check the status
curl -H "Authorization: Bearer $KEY" https://api.layer3intel.com/feed/v1/status{
"files": [
{
"route": "residential_proxies",
"path": "/feed/v1/residential_proxies",
"filename": "residential_proxies.csv.gz",
"content_type": "application/gzip",
"counted": true,
"size": 1093006737,
"updated_at": "2026-11-01T14:11:03Z",
"etag": "\"i2qxe9-dmjxo52z4dlo\""
},
{
"route": "provider_map",
"path": "/feed/v1/provider_map",
"filename": "provider_map.csv",
"content_type": "text/plain; charset=utf-8",
"counted": false,
"size": 2070,
"updated_at": "2026-11-01T14:11:03Z",
"etag": "\"1li-dmjxo58zu7y9\""
}
],
"pulls": {
"min_interval_hours": 24,
"last_pull_at": "2026-11-01T17:00:36Z",
"last_pull_etag": "\"i2qxe9-dmjxo52z4dlo\"",
"next_pull_at": "2026-11-02T17:00:36Z",
"can_pull_now": false
}
}| Field | Type | Description |
|---|---|---|
files[].route | string | The route's name. |
files[].path | string | The route's path. |
files[].filename | string | The file name the download is saved under. |
files[].content_type | string | The file's Content-Type. |
files[].counted | boolean | true when downloading the file uses a pull. |
files[].size | integer | The file's size in bytes. |
files[].updated_at | time | When the file was last updated. |
files[].etag | string | An opaque tag that changes whenever the file does. |
pulls.min_interval_hours | integer | The interval between pulls, in hours. |
pulls.last_pull_at | time or null | Your latest pull. null if you have never pulled. |
pulls.last_pull_etag | string or null | The etag of the file you last pulled. null if you have never pulled. |
pulls.next_pull_at | time | When a new pull will be accepted. The current time when can_pull_now is true. |
pulls.can_pull_now | boolean | Whether a new pull would be accepted now. |
Times are RFC 3339, UTC (2026-11-01T14:11:03Z). A file that is momentarily unavailable is left out of files.
Ask whether the file has changed
Two free ways to ask the feed route itself:
HEAD /feed/v1/residential_proxiesreturnsContent-Length,Last-ModifiedandETag, and never uses a pull.- A conditional
GET: send theETagyou last downloaded asIf-None-Match, or itsLast-ModifiedasIf-Modified-Since. If the file hasn't changed you get304 Not Modified, which doesn't use a pull. If it has, you get the new file.
curl -H "Authorization: Bearer $KEY" \
-H 'If-None-Match: "i2qxe9-dmjxo52z4dlo"' \
-OJ https://api.layer3intel.com/feed/v1/residential_proxiesPull rules
- A pull is one download of
residential_proxies, counted when the download starts. - Your key may pull at most once every 24 hours (
min_interval_hours), counted from the start of your last pull. That is the only limit: there is no monthly allowance. - Downloading again is free within that interval, as long as the file hasn't changed since your pull (the same
ETag). Once the file has been updated, the new version needs a new pull after the interval. - Resume a broken download by retrying it, or with
RangeandIf-Range: <etag>. - One download at a time per key. A second, while one is running, gets
429 download_in_progress. - The provider map and the status are free and unlimited.
# Resume a broken download from where it stopped
curl -H "Authorization: Bearer $KEY" \
-H 'If-Range: "i2qxe9-dmjxo52z4dlo"' \
-C - -o residential_proxies.csv.gz \
https://api.layer3intel.com/feed/v1/residential_proxiesErrors
Errors are JSON, as on the Intel API. A refused pull is a 429 whose error carries retry_at, with Retry-After in seconds:
{
"error": {
"code": "pull_interval",
"message": "this key pulled the feed less than 24 hours ago",
"retry_at": "2026-11-02T17:00:36Z"
}
}| Status | code | When |
|---|---|---|
| 401 | unauthorized | A missing, malformed or unknown key. |
| 403 | feed_not_enabled | The key has no data feed access. See Get access. |
| 404 | not_found | A path that isn't one of the three feed routes. |
| 405 | method_not_allowed | Any method but GET and HEAD. The response carries Allow. |
| 429 | pull_interval | Less than 24 hours since your last pull, and the file has changed since. |
| 429 | download_in_progress | A download for this key is already running. Retry in a minute. |
| 503 | unavailable | Momentarily unavailable. Retry. |