LAYER3 INTEL
Data feed

Data feed

Download the whole residential proxy dataset as one gzipped CSV, with each exit's last sighting, days seen and providers.

The data feed is the whole residential proxy dataset as one file: every exit address, when it was last seen, how often, and the providers it was seen with. It suits attribution, research and education. For real-time scoring of a single address, use the Intel API.

curl -H "Authorization: Bearer $KEY" -OJ https://api.layer3intel.com/feed/v1/residential_proxies

Get access

Subscribe at layer3intel.com/dashboard/feed. Your account's existing API key then has feed access. Prices are on the pricing page.

The feed is separate from the lookup API: a download uses pulls, not credits, and feed access doesn't use or need /v1 credits. A key without feed access gets 403 feed_not_enabled.

Routes

Method and pathAnswersCost
GET /feed/v1/residential_proxiesThe feed: residential_proxies.csv.gz, application/gzip1 pull
GET /feed/v1/provider_mapprovider_map.csv, as text/plain; charset=utf-8Free, unlimited
GET /feed/v1/statusEach file's size and last update, and your pullsFree, unlimited

The base URL is https://api.layer3intel.com.

  • The key: Authorization: Bearer YOUR_API_KEY (recommended) or ?key=YOUR_API_KEY.
  • The file name comes in Content-Disposition. Save under it with curl -OJ or wget --content-disposition.
  • Never cached: every response is Cache-Control: private, no-store.

Downloads are limited to one pull every 24 hours. Check before you pull: see Pulls and freshness.

File format

residential_proxies.csv.gz is a gzipped CSV with a header row, one IPv4 address per row:

ip,last_seen,percent_days_seen,data_sources
203.0.113.117,1793491200,100,1;4;9
198.51.100.23,1793404800,40,2
ColumnTypeDescription
ipstringThe residential proxy exit address.
last_seenintegerThe last time the address was seen as an exit, in Unix seconds.
percent_days_seeninteger, 0–100The share of the last 30 days on which the address was seen.
data_sourcesstringA ;-separated list of provider numbers. The provider map names them.

The rows above are illustrative documentation values.

Provider map

provider_map.csv names the provider numbers in data_sources. It is a CSV with the header row provider_name,provider_number: a provider code and its number, one provider per row.

The provider codes are written differently from the brand names in the Intel API's residential_proxy.providers.

import csv, gzip

with open('provider_map.csv', newline='') as f:
    providers = {row['provider_number']: row['provider_name'] for row in csv.DictReader(f)}

with gzip.open('residential_proxies.csv.gz', 'rt', newline='') as f:
    for row in csv.DictReader(f):
        names = [providers.get(n, n) for n in row['data_sources'].split(';') if n]
        print(row['ip'], row['last_seen'], row['percent_days_seen'], names)

Requests

/feed/v1/*
MethodsGET, HEAD
Callable from a web page (CORS)No. Download from your server.
Query stringkey
gzip (Accept-Encoding: gzip)JSON (status, errors) only. The feed file is already gzipped.
RangeThe feed file and the provider map

On this page