LAYER3 INTEL
Data feed

Pulls and freshness

Check the data feed's status before you download, re-download and resume for free, and handle the 24-hour pull interval.

A pull is one download of the feed file. Your key may pull once every 24 hours. Checking the status, asking whether the file has changed, and re-downloading a file you already pulled are free.

Check the status

curl -H "Authorization: Bearer $KEY" https://api.layer3intel.com/feed/v1/status
{
  "files": [
    {
      "route": "residential_proxies",
      "path": "/feed/v1/residential_proxies",
      "filename": "residential_proxies.csv.gz",
      "content_type": "application/gzip",
      "counted": true,
      "size": 1093006737,
      "updated_at": "2026-11-01T14:11:03Z",
      "etag": "\"i2qxe9-dmjxo52z4dlo\""
    },
    {
      "route": "provider_map",
      "path": "/feed/v1/provider_map",
      "filename": "provider_map.csv",
      "content_type": "text/plain; charset=utf-8",
      "counted": false,
      "size": 2070,
      "updated_at": "2026-11-01T14:11:03Z",
      "etag": "\"1li-dmjxo58zu7y9\""
    }
  ],
  "pulls": {
    "min_interval_hours": 24,
    "last_pull_at": "2026-11-01T17:00:36Z",
    "last_pull_etag": "\"i2qxe9-dmjxo52z4dlo\"",
    "next_pull_at": "2026-11-02T17:00:36Z",
    "can_pull_now": false
  }
}
FieldTypeDescription
files[].routestringThe route's name.
files[].pathstringThe route's path.
files[].filenamestringThe file name the download is saved under.
files[].content_typestringThe file's Content-Type.
files[].countedbooleantrue when downloading the file uses a pull.
files[].sizeintegerThe file's size in bytes.
files[].updated_attimeWhen the file was last updated.
files[].etagstringAn opaque tag that changes whenever the file does.
pulls.min_interval_hoursintegerThe interval between pulls, in hours.
pulls.last_pull_attime or nullYour latest pull. null if you have never pulled.
pulls.last_pull_etagstring or nullThe etag of the file you last pulled. null if you have never pulled.
pulls.next_pull_attimeWhen a new pull will be accepted. The current time when can_pull_now is true.
pulls.can_pull_nowbooleanWhether a new pull would be accepted now.

Times are RFC 3339, UTC (2026-11-01T14:11:03Z). A file that is momentarily unavailable is left out of files.

Ask whether the file has changed

Two free ways to ask the feed route itself:

  • HEAD /feed/v1/residential_proxies returns Content-Length, Last-Modified and ETag, and never uses a pull.
  • A conditional GET: send the ETag you last downloaded as If-None-Match, or its Last-Modified as If-Modified-Since. If the file hasn't changed you get 304 Not Modified, which doesn't use a pull. If it has, you get the new file.
curl -H "Authorization: Bearer $KEY" \
  -H 'If-None-Match: "i2qxe9-dmjxo52z4dlo"' \
  -OJ https://api.layer3intel.com/feed/v1/residential_proxies

Pull rules

  • A pull is one download of residential_proxies, counted when the download starts.
  • Your key may pull at most once every 24 hours (min_interval_hours), counted from the start of your last pull. That is the only limit: there is no monthly allowance.
  • Downloading again is free within that interval, as long as the file hasn't changed since your pull (the same ETag). Once the file has been updated, the new version needs a new pull after the interval.
  • Resume a broken download by retrying it, or with Range and If-Range: <etag>.
  • One download at a time per key. A second, while one is running, gets 429 download_in_progress.
  • The provider map and the status are free and unlimited.
# Resume a broken download from where it stopped
curl -H "Authorization: Bearer $KEY" \
  -H 'If-Range: "i2qxe9-dmjxo52z4dlo"' \
  -C - -o residential_proxies.csv.gz \
  https://api.layer3intel.com/feed/v1/residential_proxies

Errors

Errors are JSON, as on the Intel API. A refused pull is a 429 whose error carries retry_at, with Retry-After in seconds:

{
  "error": {
    "code": "pull_interval",
    "message": "this key pulled the feed less than 24 hours ago",
    "retry_at": "2026-11-02T17:00:36Z"
  }
}
StatuscodeWhen
401unauthorizedA missing, malformed or unknown key.
403feed_not_enabledThe key has no data feed access. See Get access.
404not_foundA path that isn't one of the three feed routes.
405method_not_allowedAny method but GET and HEAD. The response carries Allow.
429pull_intervalLess than 24 hours since your last pull, and the file has changed since.
429download_in_progressA download for this key is already running. Retry in a minute.
503unavailableMomentarily unavailable. Retry.

On this page