Plan API pagination in FileMaker before importing records

The first API response contains 100 customers. The account has 640. A successful request has retrieved one page, not necessarily the whole collection. Before writing an import loop, understand how the provider tells you there is another page.

This is a design walkthrough using fictional JSON responses. It is not a working integration for a named service. Build the actual request from your provider’s documentation and test retrieval before allowing the script to write records.

1. Identify the provider’s paging contract.

Providers may use page numbers, offsets, cursors or a next-page URL. Do not combine rules from different APIs. Write down the request parameter, maximum page size, response location of the continuation value, and exact signal that means “finished”.

Fictional first page

{
  "items": [{"id":"C001","name":"Mira"}],
  "next_cursor": "page-B"
}

Fictional final page

{
  "items": [{"id":"C002","name":"Leon"}],
  "next_cursor": null
}

In our invented contract, a non-empty text cursor requests the next page, and JSON null ends retrieval. An absent cursor key is an invalid response, not an instruction to silently finish. Your provider may define those cases differently.

2. Validate a page before using its cursor.

Keep the request status and response body together. A network error, an HTTP error and a valid response with no items are different outcomes. Confirm the response shape before interpreting the continuation value.

For this fixture, items must be an array. next_cursor must be either non-empty text or JSON null. In FileMaker Pro 19.5 or later, JSONGetElementType can distinguish those types. Do not rely only on an empty-looking extracted value.

Use the JSON response guide to practice reading the two sample bodies locally. For older FileMaker versions, choose and test a compatible validation strategy rather than dropping type checks without a replacement.

3. Make completion different from a safety stop.

Design outline — not FileMaker script syntax

Start with no cursor and an empty set of seen cursors.
Request one page using the provider’s documented parameters.
Check transport, HTTP status and JSON structure.
Validate and stage the page’s items.
If next_cursor is null: mark retrieval complete and stop.
If next_cursor is empty or already seen: stop with an error.
Remember the cursor and request the next page.
If the run limit is reached: stop as incomplete, not successful.

Choose a maximum page count and elapsed-time limit appropriate to the provider and your job. Those limits protect the workflow from an endless loop, but reaching one is not proof that all records were retrieved. Record why the run stopped.

Treat cursors as opaque values: do not increment, decode or rewrite them unless the provider requires it. Encode query parameters correctly. If the API returns a URL, validate its scheme and approved host before sending credentials to it.

4. Plan for an interrupted run.

An import should not create a second customer every time you retry a page. Match incoming items by a stable provider ID and define what happens when that ID already exists. Do not match customers by their names alone.

Keep progress only after the page has been safely processed. Otherwise, a saved cursor can skip records whose writes failed. A staging table can separate “retrieved” from “applied”, which makes failures easier to inspect.

If the provider changes data during pagination, records may move between pages. Follow its documented ordering or snapshot behavior and use IDs to detect repeats. Do not treat a total count from the first response as an immutable promise.

5. Rehearse failures with fixtures.

The two sample pages: stage C001 and C002, then finish on null.
First page returns an empty items array with a continuation cursor: follow the contract, not an assumed item-count rule.
A repeated cursor: stop and report an incomplete run.
An HTTP failure on page two: preserve the completed work and report failure.
The same item ID appears twice: apply the documented duplicate policy.
The safety limit is reached: report incomplete rather than “all imported”.

Use the cURL builder for an individual FileMaker request and the troubleshooting guide for response diagnostics. Neither tool supplies the provider’s paging rules. Prove one page, then two pages, then a controlled retry before increasing the volume.

PUT IT INTO PRACTICE

Continue with Filesoft.

Open the cURL builder

Prefer a guided learning path? Explore the free FileMaker courses.

Continue reading

Claris references: Insert from URL · JSONGetElementType · Supported cURL options. Examples are learning aids; verify them in your own FileMaker working copy.