Handling pagination & rate limits

To handle ShopAPIS pagination, follow the next_cursor token through search and listing endpoints until it is null; to handle rate limits, bound your concurrency to your plan and retry HTTP 429 responses with exponential backoff and jitter. These two patterns let you pull large result sets reliably without getting throttled, and they are the same regardless of which of the 70+ marketplaces you query.

Pagination: follow the cursor

ShopAPIS paginates with an opaque next_cursor token — request the first page, then pass the cursor back until the API stops returning one. Do not assume page numbers or fixed page sizes; the cursor is the contract. A search response looks like this:

{
  "results": [
    { "id": "B0CHWRXH8B", "title": "AirPods Pro (2nd Gen)", "price": 189.99 },
    { "id": "B0D1XD1ZV3", "title": "AirPods 4", "price": 129.00 }
  ],
  "page": { "count": 2, "next_cursor": "eyJvZmZzZXQiOjUwfQ==" }
}

Loop until next_cursor is absent or null:

import requests
 
def paginate(query, marketplace="amazon", country="US"):
    cursor, items = None, []
    while True:
        params = {"marketplace": marketplace, "country": country, "q": query}
        if cursor:
            params["cursor"] = cursor
        r = requests.get(
            "https://api.shopapis.com/v1/search",
            params=params,
            headers={"Authorization": "Bearer YOUR_API_KEY"},
            timeout=30,
        )
        r.raise_for_status()
        data = r.json()
        items.extend(data["results"])
        cursor = data["page"].get("next_cursor")
        if not cursor:
            return items

async function paginate(query, marketplace = "amazon", country = "US") {
  let cursor = null;
  const items = [];
  do {
    const params = new URLSearchParams({ marketplace, country, q: query });
    if (cursor) params.set("cursor", cursor);
    const r = await fetch(`https://api.shopapis.com/v1/search?${params}`, {
      headers: { Authorization: "Bearer YOUR_API_KEY" },
    });
    if (!r.ok) throw new Error(`HTTP ${r.status}`);
    const data = await r.json();
    items.push(...data.results);
    cursor = data.page.next_cursor ?? null;
  } while (cursor);
  return items;
}

Rate limits: back off on 429

A 429 Too Many Requests is a signal to slow down, not an error to abort on — retry with exponential backoff and randomized jitter. Jitter prevents a fleet of workers from retrying in lockstep (the “thundering herd” problem). Respect the Retry-After header when present.

import time, random, requests
 
def get_with_backoff(url, params, max_retries=6):
    headers = {"Authorization": "Bearer YOUR_API_KEY"}
    for attempt in range(max_retries):
        r = requests.get(url, params=params, headers=headers, timeout=30)
        if r.status_code == 429:
            wait = r.headers.get("Retry-After")
            delay = float(wait) if wait else (2 ** attempt) + random.random()
            time.sleep(delay)
            continue
        r.raise_for_status()
        return r.json()
    raise RuntimeError("exceeded retry budget")

Bounding concurrency

The most effective way to avoid 429 is to not send too many requests at once — cap parallelism to your plan’s concurrency limit instead of firing the whole batch. A bounded worker pool keeps throughput high and block rates near zero:

from concurrent.futures import ThreadPoolExecutor
 
def fetch_all(items, workers=8):
    with ThreadPoolExecutor(max_workers=workers) as pool:   # cap parallelism
        return list(pool.map(lambda it: get_with_backoff(
            "https://api.shopapis.com/v1/product", it), items))

Best practices

• Cursor, not page numbers — always follow next_cursor; never hardcode offsets.
• Backoff + jitter on every 429; honor Retry-After.
• Bound concurrency to your plan (pricing) rather than retrying after the fact.
• Cap your retry budget so a persistent failure does not loop forever.
• Cache where data is slow-moving — re-fetch prices hourly, but catalog fields far less often.
• Treat 404 as data, not failure — a delisted product is a valid, uncharged result.