How a Nightly Inventory Sync Was Paying for Every Single Product

Most integration costs don't come from clever code. They come from a loop that nobody questioned. This is the story of a nightly warehouse sync that was quietly paying for every single product it touched, and what it took to fix it.

The setup

A retail client keeps its stock levels in a third-party inventory platform. Every night at 2 a.m., a Laravel scheduled job pulls the latest quantities and prices for the whole catalogue and mirrors them into the shop database. The platform bills per API request. For years the job worked, nobody complained, and the invoice grew a little each month as the catalogue grew.

The job looked roughly like this: fetch the list of product IDs, then loop over them and call the "get product" endpoint once per ID. With about 12,000 SKUs, that is 12,000 requests every night, plus retries, plus the occasional rate-limit pause. Roughly 360,000 billable calls a month for data that changes in small pieces.

Measure first

Before reading any documentation, we measured what the job actually did. We logged request counts, response times and payload sizes for one week. Three numbers told the story:

• Requests per run: a little over 12,000, almost all of them to the single-product endpoint.
• Average response time: about 180 ms, which means the run spent more than 35 minutes just waiting on the network.
• Useful data per request: a few hundred bytes, wrapped in headers and authentication overhead.

Measuring first matters because it stops you from optimising the wrong thing. Had we assumed the database writes were slow, we would have spent a week tuning queries that accounted for under 5% of the runtime.

Then read the docs

With the numbers in hand, we sat down with the API reference and read it end to end, not just the endpoint we were already using. Three findings stood out.

1. There is a batch endpoint

The platform exposes a bulk lookup that accepts up to 50 product IDs in one call and returns them all in a single response. Our loop had been ignoring it since the original integration was written, probably because the first version only ever synced a few dozen products.

The arithmetic is simple: 12,000 products at 50 per call is 240 requests instead of 12,000. That is a 98% reduction in calls.

2. The page size was set too low

When listing products, our code asked for 100 per page. The documentation allows 250. Moving to the maximum cuts the listing step from 120 requests to 48. It is a small win on its own, but it is a one-line change.

3. A parameter that isn't in the spec

Our request included a query parameter, include_archived=false, that does not appear anywhere in the documented parameters. It might be an old undocumented feature, it might be silently ignored, or it might have been added by a developer who guessed. We cannot tell from the docs alone, so we marked it as unverified. We do not remove it and we do not rely on it until we test it against a sandbox account.

What the new code looks like

In Laravel, the change is small. Instead of looping, we chunk the IDs and send each chunk in one request:

collect($productIds)
    ->chunk(50)
    ->each(function ($chunk) {
        $response = Http::withToken(config('services.inventory.token'))
            ->post(config('services.inventory.url') . '/products/bulk', [
                'ids' => $chunk->values()->all(),
            ])
            ->throw();

        foreach ($response->json('data') as $product) {
            $this->syncProduct($product);
        }
    });

The batch size and page size live in configuration rather than in the loop, so they can be changed without a deploy if the vendor revises its limits.

Risks we wrote down

Before changing anything, we put the plan in a short document and got it agreed. A cheaper job is not worth it if it fails silently. The risks we listed:

• Partial failures: if one ID in a batch is invalid, does the whole call fail or only that item? We need to check the response format and handle both cases.
• Larger payloads: a 50-item response is bigger and may hit memory or timeout limits. We will test with the real catalogue.
• Rate limits: fewer requests should help, but the batch endpoint may have its own, stricter limit.
• Billing rules: some vendors count each item in a batch as a separate billable unit. If this one does, the saving shrinks dramatically. This must be confirmed with the vendor, not assumed.

What we could not verify

It is worth being explicit about this part. We have not yet confirmed how the batch endpoint is billed, how it behaves with invalid IDs, or whether include_archived does anything. The 98% figure is a reduction in calls, and it only becomes a reduction in cost once the billing question is answered.

Rollout checklist

1. Run the new sync against a sandbox account and diff the results with the old sync.
2. Confirm billing behaviour for batch calls with the vendor in writing.
3. Deploy behind a config flag, so the old per-product loop is one switch away.
4. Run both for a week, comparing request counts and data differences.
5. Remove the old loop once the numbers match.

The lesson

Measure first, read the docs second, and say clearly which claims you have not tested yet. The cheapest optimisation in this job was not a faster query or a bigger server. It was reading the spec and noticing that the API had already offered a better way to ask for the same data.