How to batch enrich 100,000 contacts with the gender batch API

Enriching a large contact list is mostly an exercise in patience and bookkeeping. The API call itself is simple; the hard parts are pacing requests, surviving restarts, and handling partial failures without paying twice. This guide walks through enriching 100,000 contacts with POST /v1/gender/batch, with complete examples in Python and Node.

#The shape of the job

The batch endpoint accepts between 1 and 50 items per request. Each item has an id you choose, a type, and a value, plus an optional country hint:

{
  "items": [
    { "id": "c1", "type": "name", "value": "Jane", "country": "US" },
    { "id": "c2", "type": "email", "value": "li.wei@example.com" }
  ]
}

The response preserves order and reports each item separately:

{
  "data": [
    { "index": 0, "id": "c1", "charged_credits": 1, "data": { "result_status": "identified", "gender": "female", "confidence": 0.98, "confidence_kind": "observed_frequency", "sample_count": 2570, "name": "jane", "country": "US", "source": "dataset", "reason": null } },
    { "index": 1, "id": "c2", "charged_credits": 0, "error": { "code": "provider_unavailable", "status": 502 } }
  ]
}

With 50 items per request, 100,000 contacts become 2,000 requests. Finowl allows 60 requests per minute per account, and a batch counts as one request. At a steady one request per second, the whole job takes a little over 33 minutes.

#Estimating credits before you start

Each batch item costs 1 credit. Batch items default to ai_mode: "off", meaning dataset only, so they stay at 1 credit each rather than the 3 credits charged when a gender lookup requests AI. For 100,000 contacts, plan for up to 100,000 credits.

The Pro pack on the pricing page covers this job almost exactly. Your 100 free monthly credits are used first, and prepaid credits never expire, so leftovers carry into the next job. Items that fail with an error the provider confirms was not billed show charged_credits: 0 and cost nothing. If you want to model costs more precisely across different jobs, see estimating API credit usage.

Unknown results are charged like any other successful lookup. Deduplicating your input before you start is the cheapest optimization you can make: if 8,000 contacts share a first name and country with another contact, you only need one lookup per unique pair when you enrich by name.

#Preparing the input

Batch validation is all-or-nothing. If one item is invalid, the whole request is rejected with 400 invalid_input and nothing runs. That is free, but it stalls the job, so validate locally first:

• Every id must be unique within the batch and at most 64 characters. Use opaque ids, not email addresses.
• Every value must be non-empty and at most 254 characters.
• type is name or email for this job; pick one per contact based on the best data you have.
• country is an optional ISO 3166-1 alpha-2 code. Send it when you know it, because names like Andrea and Kim lean differently by country.

Write the prepared items to a JSONL file with one item per line. That file becomes the stable input for a resumable job.

#Designing for resumability

A 30-minute job will eventually be interrupted by a deploy, a laptop lid, or a network blip. Design it so a restart picks up where it left off without resending anything that already ran.

The simplest pattern uses an append-only results file:

1. Before starting, read the results file and collect every id that already has an outcome.
2. Skip those ids when building chunks.
3. After each response, append one line per item to the results file and flush it.

Because each item is recorded the moment its response arrives, a crash loses at most the in-flight request. On restart, completed ids are skipped automatically.

#Handling item errors without resending whole batches

A 200 response can contain item-level errors. Never resend a whole batch because some items failed; the successful items were already charged, and resending them charges them again.

Instead, sort failed items into two groups:

• charged_credits: 0: the lookup was not billed. Put these items into a retry queue and include them in a later batch.
• Errors with a charge, such as provider_response_unconfirmed: the lookup may have run. Record them for review rather than retrying automatically.

The same logic applies to whole-request failures. A 429 rate_limited was rejected before running, so wait for Retry-After and send the same chunk again. A 429 quota_exceeded means you are out of credits; stop and top up. A 5xx or a timeout on the whole request is uncertain, so mark that chunk for review instead of resending it blindly. Safe retries for paid APIs explains the reasoning in depth.

#Python example

This script reads contacts.jsonl, writes results.jsonl, paces requests, and resumes cleanly:

import json
import os
import time
import requests

API_URL = "https://api.finowl.co/v1/gender/batch"
HEADERS = {
    "Authorization": f"Bearer {os.environ['FINOWL_API_KEY']}",
    "Content-Type": "application/json",
}
CHUNK_SIZE = 50
MIN_INTERVAL = 1.05  # seconds between requests, just under 60 per minute


def load_done(path):
    done = set()
    if os.path.exists(path):
        with open(path) as f:
            for line in f:
                done.add(json.loads(line)["id"])
    return done


def chunks(items, size):
    for i in range(0, len(items), size):
        yield items[i : i + size]


def run(input_path="contacts.jsonl", output_path="results.jsonl"):
    done = load_done(output_path)
    with open(input_path) as f:
        pending = [json.loads(l) for l in f if json.loads(l)["id"] not in done]
    print(f"{len(pending)} items to process")

    retry = []
    with open(output_path, "a") as out:
        queue = list(chunks(pending, CHUNK_SIZE))
        while queue:
            chunk = queue.pop(0)
            started = time.monotonic()
            try:
                resp = requests.post(API_URL, headers=HEADERS, json={"items": chunk}, timeout=60)
            except requests.RequestException as exc:
                print(f"Uncertain failure, review chunk starting {chunk[0]['id']}: {exc}")
                continue

            if resp.status_code == 429:
                code = resp.json()["error"]["code"]
                if code == "quota_exceeded":
                    raise SystemExit("Out of credits. Top up and rerun.")
                time.sleep(int(resp.headers.get("Retry-After", "5")))
                queue.insert(0, chunk)
                continue
            if resp.status_code != 200:
                print(f"HTTP {resp.status_code}, review chunk starting {chunk[0]['id']}")
                continue

            for result in resp.json()["data"]:
                err = result.get("error")
                if err and result["charged_credits"] == 0:
                    retry.append(chunk[result["index"]])
                    continue
                out.write(json.dumps(result) + "\n")
            out.flush()

            if not queue and retry:
                queue = list(chunks(retry, CHUNK_SIZE))
                retry = []

            elapsed = time.monotonic() - started
            time.sleep(max(0.0, MIN_INTERVAL - elapsed))


if __name__ == "__main__":
    run()

Items with charged errors are written to the results file with their error, so they count as done and appear in your review step. Uncharged failures go back into the queue. In production you would cap retry attempts per item; a provider that is down for an hour should not keep your job spinning.

#Node example

The same flow in Node 18 or later, using the built-in fetch:

import fs from "node:fs";
import readline from "node:readline";

const API_URL = "https://api.finowl.co/v1/gender/batch";
const CHUNK_SIZE = 50;
const MIN_INTERVAL_MS = 1050;
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

async function readJsonl(path) {
  if (!fs.existsSync(path)) return [];
  const rl = readline.createInterface({ input: fs.createReadStream(path) });
  const rows = [];
  for await (const line of rl) if (line.trim()) rows.push(JSON.parse(line));
  return rows;
}

async function run(input = "contacts.jsonl", output = "results.jsonl") {
  const done = new Set((await readJsonl(output)).map((r) => r.id));
  let pending = (await readJsonl(input)).filter((i) => !done.has(i.id));
  const out = fs.createWriteStream(output, { flags: "a" });

  while (pending.length) {
    const retry = [];
    for (let i = 0; i < pending.length; ) {
      const chunk = pending.slice(i, i + CHUNK_SIZE);
      const started = Date.now();
      let resp;
      try {
        resp = await fetch(API_URL, {
          method: "POST",
          headers: {
            Authorization: `Bearer ${process.env.FINOWL_API_KEY}`,
            "Content-Type": "application/json",
          },
          body: JSON.stringify({ items: chunk }),
        });
      } catch (err) {
        console.error(`Uncertain failure, review chunk at ${chunk[0].id}`, err);
        i += CHUNK_SIZE;
        continue;
      }

      if (resp.status === 429) {
        const { error } = await resp.json();
        if (error.code === "quota_exceeded") throw new Error("Out of credits");
        await sleep(Number(resp.headers.get("Retry-After") ?? 5) * 1000);
        continue; // same chunk again
      }
      if (resp.status !== 200) {
        console.error(`HTTP ${resp.status}, review chunk at ${chunk[0].id}`);
        i += CHUNK_SIZE;
        continue;
      }

      const { data } = await resp.json();
      for (const result of data) {
        if (result.error && result.charged_credits === 0) retry.push(chunk[result.index]);
        else out.write(JSON.stringify(result) + "\n");
      }
      i += CHUNK_SIZE;
      await sleep(Math.max(0, MIN_INTERVAL_MS - (Date.now() - started)));
    }
    pending = retry;
  }
  out.end();
}

run();

#After the run

When the job finishes, check three things. First, count the review list of uncertain chunks and charged errors, and decide how to handle them by hand. Second, check the distribution of result_status and reason values; a high share of no_name_candidate usually means you sent role addresses like sales@ as emails. Third, store results in separate predicted fields rather than overwriting anything people told you themselves.

Gender predictions are statistical estimates of how a name is typically used, not how a person identifies. Use the results for aggregate analytics and low-stakes personalization, never for eligibility or pricing, and mention the enrichment in your privacy notice. The batch gender enrichment guide has more detail on the endpoint, and the rate limits documentation covers pacing.

#Frequently asked questions

#How long does it take to enrich 100,000 contacts?

At 50 items per request and 60 requests per minute, 100,000 contacts need 2,000 requests, which takes a little over 33 minutes at full pace. Leave some headroom if other services share the same account.

#How many credits will 100,000 contacts use?

Up to 100,000 credits, since each batch item costs 1 credit and batch items are dataset only by default. The Pro pack, 100,000 credits for $99, covers it, and items with charged_credits: 0 cost nothing.

#What happens if one item in a batch is invalid?

The whole batch is rejected with 400 invalid_input and nothing runs, so you are not charged. Validate ids and values locally before sending.

#Should I resend a batch that came back with some item errors?

No. Successful items in that batch were already charged. Retry only the failed items with charged_credits: 0 in a later batch, and review charged errors before doing anything else.

#Can I run several batch workers in parallel?

You can, but the 60 requests per minute limit applies to the whole account, so parallel workers do not make the job faster. A single, well-paced worker is simpler and easier to resume.