Search Individuals by ID Range

Pull individuals in ID order, in batches, using the individual_id filter on the individual search endpoint.

Use this guide when you need to walk through every individual in a church in a predictable order, or pull a specific block of individual IDs. Typical cases are a first full sync, a resumable backfill, or splitting a large pull across several workers.

The endpoint is POST /search/individuals/results. There is no query parameter or configuration key for an ID range. You set the range with an individual_id filter in the request body.

📘

Set exclude_total=true on every request

This is the biggest performance gain available on this endpoint. By default the API counts every matching row on every request, which is slow on large populations. Adding exclude_total=true to the URL skips that count. All examples in this guide use it.

With the count skipped, X-Total and X-Total-Pages are not returned, so do not use them to decide when to stop. Stop when a page returns no rows.

Parameters versus filters

A request has two places where you send settings, and they do different jobs.

WhereWhat it isWhat it controls
URL query stringParameters: page, per_page, exclude_totalPaging only. Which page you get, how many rows, and whether the total is counted.
Request body: configurationA JSON objectResult shape and population. Sort order, which profile types are included (active, inactive, deceased), pending and limited access profiles, and saved searches.
Request body: filtersA JSON objectWhich individuals match. This is where the individual_id range goes.

The ID range is a filter, so it lives in the body under filters. It is not a parameter, and it is not a configuration key. Paging values such as per_page are parameters, so they stay in the URL. Putting exclude_total or per_page in the body has no effect.

POST /search/individuals/results?page=1&per_page=100&exclude_total=true     <- parameters (URL)

{
  "configuration": { ... },                              <- body: result shape and population
  "filters": { ... }                                     <- body: who matches
}

The individual_id filter

individual_id is a number criterion, so it accepts the standard number operators.

OperatorValueResult
greater_than_or_equalnumberIDs at or above the value.
greater_thannumberIDs above the value.
less_than_or_equalnumberIDs at or below the value.
less_thannumberIDs below the value.
between{ "lbound": n, "ubound": n }IDs inside the range, inclusive.
in[n, n, ...]Only the listed IDs.

Example: everyone from ID 1001 and up

{
  "configuration": {
    "return_search_results": true,
    "filter_profile_type": ["active", "inactive", "deceased"],
    "order_by": [{ "column": "id", "direction": "asc" }]
  },
  "filters": {
    "type": "constraint",
    "id": "individual_id",
    "operator": "greater_than",
    "value": 1000,
    "invert": false
  }
}

Send it as POST /search/individuals/results?page=1&per_page=100&exclude_total=true. The JSON above is the body. page, per_page, and exclude_total stay in the URL.

Example: a bounded range

Send it as POST /search/individuals/results?page=1&per_page=100&exclude_total=true.

{
  "configuration": {
    "return_search_results": true,
    "filter_profile_type": ["active", "inactive", "deceased"],
    "order_by": [{ "column": "id", "direction": "asc" }]
  },
  "filters": {
    "type": "constraint",
    "id": "individual_id",
    "operator": "between",
    "value": { "lbound": 1000, "ubound": 2000 },
    "invert": false
  }
}

Query string parameters

These go in the URL, not the body.

ParameterTypeDefaultDescription
pageinteger11-based page number.
per_pageinteger25Page size. Must be 25, 50, 75, or 100.
exclude_totalbooleanfalseSkips the total row count. Set to true on every request. Much faster on large result sets, but the X-Total-Pages and X-Totalheaders are omitted.

Response headers: X-Page, X-Per-Page, X-Offset, X-Total, X-Total-Pages.

📘

Set exclude_total=true

Adding exclude_total=true to the URL skips that count. >
With the count skipped, X-Total and X-Total-Pages are not returned, so do not use them to decide when to stop. Stop when a page returns no rows.

Things to know

  • Sort on id, not individual_id. The filter id is individual_id, but the sort column is id. Using individual_id in order_by returns 412 with Invalid order_by column.
  • Always set order_by when paging. Without a sort, records can repeat or be skipped between pages.
  • Set filter_profile_type for a full population. By default only active profiles are returned, even inside your ID range. Add inactive and deceased if you need them.
  • IDs are not contiguous. A range of 1000 to 2000 will not return 1001 people. Gaps are normal.
  • Pending and limited access profiles are included by default. Set include_pending or include_lau to false to exclude them.

Walking the whole population

To process everyone in order without relying on deep page numbers:

  1. Send the greater_than request with value: 0, order_by on id ascending, per_page=100, and exclude_total=true.
  2. Process the page and note the highest id returned.
  3. Send the same request with value set to that highest id.
  4. Stop when a page returns no rows.

This keeps each request cheap and lets you resume from the last ID after a failure.


Did this page help you?