Pagination, Filtering & Field Selection

Every list endpoint in the Siren API shares the same set of cross-cutting query patterns for pagination, filtering, sorting, and field selection. These patterns are implemented through a middleware pipeline that processes the request before it reaches the controller, and an interceptor that wraps the response afterward. This page documents how each pattern works, what parameters are available, and how they combine.

Understanding these patterns once means understanding them everywhere — the same parameters behave identically whether you are listing programs, collaborators, transactions, conversions, or any other resource.

These patterns are implemented by FilterMiddleware, PaginationMiddleware, FieldResolverMiddleware, and ListResponseWrapperInterceptor. Each list controller wires these into its middleware stack.

---

Pagination

List endpoints use offset-based pagination controlled by two query parameters:

Parameter	Type	Default	Max	Description
number	integer	10	50	Number of items per page (page size).
offset	integer	0	—	Zero-based index of the first item to return.

Defaults and limits

If number is omitted, the API returns 10 items. If number exceeds 50, the API silently caps it at 50 — you will never receive more than 50 items in a single response regardless of the value you pass. If offset is omitted, it defaults to 0 (start from the beginning).

The offset parameter must be zero or a positive integer. Negative values are rejected with a 400 validation error.

Page calculation

The response envelope includes a page field calculated from offset and page size:

page = floor(offset / number) + 1

For example, offset=0, number=10 yields page=1. offset=20, number=10 yields page=3.

Sorting

Two additional parameters control result ordering:

Parameter	Type	Default	Description
orderBy	string	id	Column to sort by.
order	string	varies	Sort direction: ASC or DESC.

The default sort direction varies by resource. Resources that represent configuration (programs, collaborators, program groups) default to ASC (oldest first). Resources that represent activity or time-series data (transactions, conversions, engagements, distributions, obligations, fulfillments, payouts) default to DESC (newest first).

Default ASC (oldest first)	Default DESC (newest first)
Programs	Transactions
Collaborators	Conversions
Program Groups	Engagements
	Distributions
	Distributors
	Obligations
	Fulfillments
	Payouts

Valid orderBy values depend on the resource. Most endpoints accept any string column name; some endpoints (e.g., transactions) validate against a whitelist of allowed columns. If an invalid column is provided, the API returns a 400 error.

---

Filtering

List endpoints support two filtering mechanisms: exact-match filters and text search. Both are processed by FilterMiddleware before the controller runs.

Exact-match filters

Each endpoint declares which columns support filtering. Pass the column name as a query parameter with the desired value:

GET /programs?status=active
GET /collaborators?email=alice@example.com
GET /engagements?collaboratorId=42&programId=7

Multiple filter parameters are combined with AND logic — all conditions must match.

To match any of several values, pass comma-separated values (which generates an IN clause):

GET /programs?status=active,inactive

This returns programs whose status is either active or inactive. The API splits the value on commas and generates an IN (...) SQL clause.

Common filterable columns by resource

Resource	Filterable Columns
Programs	status, incentiveType, incentiveResolverType, units
Collaborators	status, fullName, nickname, email
Engagements	status, collaboratorId, programId, opportunityId
Transactions	status
Conversions	status, distributorId
Distributions	status, distributorId
Distributors	status
Obligations	status
Fulfillments	status
Payouts	status

Each resource reference page lists its own filterable columns.

Text search (?s=)

Some endpoints support free-text search via the ?s= parameter. When provided, the API generates LIKE '%term%' clauses across all searchable columns for that resource. Searchable columns are combined with OR logic — a match in any column returns the record.

GET /programs?s=referral
GET /collaborators?s=john

For programs, ?s= searches across name and description. For collaborators, it searches fullName, nickname, and email — and additionally searches collaborator alias codes (coupon codes, referral slugs, etc.) via a secondary lookup.

Not all resources support ?s=. Resources with no text-searchable columns (transactions, engagements, distributions, obligations, fulfillments, payouts) do not respond to the ?s= parameter.

Resource	Searchable Columns
Programs	name, description
Collaborators	fullName, nickname, email + alias codes
Others	(no text search)

Combining filters and search

Filters and search combine with AND logic at the top level. The API builds a compound query structure:

• All exact-match filters form an AND group (every filter must match).
• All search clauses form an OR group (any searchable column can match).
• The AND group and the OR group are combined together — a record must satisfy all exact filters AND match at least one search column.

GET /collaborators?status=active&s=john

This returns collaborators who are active AND whose name, nickname, email, or alias code contains “john”.

Junction table filters

Some endpoints support filtering by related resource IDs through junction tables. These are not simple column filters — they resolve the relationship first and then filter the primary resource by matching IDs.

GET /programs?programGroupId=3        # Programs in group 3
GET /collaborators?programId=7        # Collaborators enrolled in program 7
GET /collaborators?distributorId=12   # Collaborators assigned to distributor 12

When no matching records exist in the junction table, the endpoint returns an empty result set (not an error).

---

Field Selection

Every list and detail endpoint supports field selection via the ?fields= query parameter. This controls which fields appear in each item of the response. Fields are resolved dynamically through a field resolver registry — each resource registers its own set of available fields.

The fields parameter

Pass a comma-separated list of field names:

GET /programs?fields=id,name,status
GET /collaborators?fields=id,fullName,email,status

The middleware parses the CSV string, validates each field against the registry of available resolvers, and silently drops any unrecognized field names. The controller then resolves only the requested fields for each record.

Required vs. optional

On some endpoints (programs, collaborators, program groups), fields is a required parameter — omitting it returns a 400 validation error. On other endpoints (transactions, engagements, conversions, distributions, fulfillments, payouts), fields is optional and the endpoint falls back to a set of default fields when omitted.

When fields is optional, the endpoint defines a DEFAULT_FIELDS constant. For example, the transactions endpoint defaults to ['id', 'status', 'dateCreated'] and the engagements endpoint defaults to ['id', 'opportunityId', 'programId', 'collaboratorId', 'score', 'status', 'dateCreated', 'dateModified'].

How field resolution works

Field resolution is a two-step process:

1. Registration: When a list endpoint handles a request, it broadcasts an event (e.g., ProgramResolverRegistryInitiated). Listeners respond by registering resolver functions — closures that accept a data model and return the field value. Core fields map directly to model properties. Extended fields perform additional queries or computations (e.g., collaboratorCount runs an aggregate query, collaborators loads related records).

2. Resolution: The controller iterates over each record and each requested field, calling the registered resolver function to produce the value. The result is an array of associative arrays, one per record, with only the requested fields.

This means that requesting expensive fields (like collaborators which loads a full list of related records) has a real performance cost. Request only the fields you need.

Core vs. extended fields

Each resource has two categories of fields:

• Core fields map directly to columns on the resource’s database table. They are cheap to resolve — just reading a property from the already-loaded model. Examples: id, name, status, dateCreated.

• Extended fields require additional queries or computation. They are registered by separate listeners and may involve joins, subqueries, or cross-datastore lookups. Examples: collaboratorCount, collaborators, totalValue, distributorName.

The resource reference pages list all available fields and note which are extended.

Detail endpoints (get by ID)

Single-resource endpoints (GET /programs/{id}, GET /collaborators/{id}, etc.) also support ?fields=. The behavior is identical — pass a comma-separated list, and the response includes only those fields. When omitted, detail endpoints return their full set of default fields.

GET /programs/42?fields=id,name,engagementTypes,transactionCompilers

Detail endpoints do not use PaginationMiddleware or ListResponseWrapperInterceptor — they return a flat JSON object, not a paginated envelope.

---

Response Format

List response envelope

All paginated list endpoints wrap their response in a standard JSON envelope with five fields:

{
  "items": [
    { "id": 1, "name": "Partner Commissions", "status": "active" },
    { "id": 2, "name": "Referral Rewards", "status": "active" }
  ],
  "total": 47,
  "page": 1,
  "perPage": 10,
  "totalPages": 5
}

Field	Type	Description
items	array	Array of objects, each containing only the requested fields.
total	integer	Estimated total count of records matching the current filters.
page	integer	Current page number (1-based, derived from offset).
perPage	integer	Page size used for this request.
totalPages	integer	Total pages available: ceil(total / perPage).

This wrapping is applied by ListResponseWrapperInterceptor after the controller has set the response body. The interceptor reads number and offset from the request and x-siren-estimated-count from the response header to compute the pagination metadata.

The x-siren-estimated-count header

Every list response includes a custom response header:

x-siren-estimated-count: 47

This header contains the total number of records matching the current filters (before pagination). The envelope’s total field is sourced from this header. The header is also included in the Access-Control-Expose-Headers response header, making it accessible to browser-based JavaScript clients via response.headers.get('x-siren-estimated-count').

The count is called “estimated” because it is computed at query time and may shift between requests if records are being created or deleted concurrently.

Empty results

When no records match the query, the API returns HTTP 200 with an empty items array:

{
  "items": [],
  "total": 0,
  "page": 1,
  "perPage": 10,
  "totalPages": 1
}

The API never returns 404 for an empty list. A 404 is only returned when a specific resource ID does not exist (on detail endpoints).

Endpoints without the envelope

A small number of list endpoints return raw JSON arrays without the pagination wrapper. These are typically configuration or metadata endpoints like GET /event-types and GET /programs/currency-types. These endpoints return simple lists where pagination is unnecessary.

---

Examples

Basic pagination

Fetch the first page of 10 programs:

curl "https://api.example.com/siren/v1/programs?fields=id,name,status" \
  -H "Authorization: Bearer $TOKEN"

Fetch page 3 (items 21-30):

curl "https://api.example.com/siren/v1/programs?fields=id,name,status&number=10&offset=20" \
  -H "Authorization: Bearer $TOKEN"

Fetch 25 items per page (maximum 50):

curl "https://api.example.com/siren/v1/programs?fields=id,name,status&number=25" \
  -H "Authorization: Bearer $TOKEN"

Filtering by status

curl "https://api.example.com/siren/v1/programs?fields=id,name,status&status=active" \
  -H "Authorization: Bearer $TOKEN"

Multi-value filter

Fetch programs that are either active or inactive (excludes deleted):

curl "https://api.example.com/siren/v1/programs?fields=id,name,status&status=active,inactive" \
  -H "Authorization: Bearer $TOKEN"

Text search

Search for collaborators whose name, email, or alias code contains “smith”:

curl "https://api.example.com/siren/v1/collaborators?fields=id,fullName,email,status&s=smith" \
  -H "Authorization: Bearer $TOKEN"

Combining filters and search

Active collaborators matching “smith”:

curl "https://api.example.com/siren/v1/collaborators?fields=id,fullName,email&status=active&s=smith" \
  -H "Authorization: Bearer $TOKEN"

Sorting

Collaborators sorted by name descending:

curl "https://api.example.com/siren/v1/collaborators?fields=id,fullName,status&orderBy=fullName&order=DESC" \
  -H "Authorization: Bearer $TOKEN"

Transactions sorted by date created ascending (overriding the default DESC):

curl "https://api.example.com/siren/v1/transactions?fields=id,status,dateCreated&orderBy=dateCreated&order=ASC" \
  -H "Authorization: Bearer $TOKEN"

Field selection on a detail endpoint

Get a single program with only specific fields:

curl "https://api.example.com/siren/v1/programs/42?fields=id,name,collaboratorCount,engagementsCount" \
  -H "Authorization: Bearer $TOKEN"

Response (flat object, no envelope):

{
  "id": 42,
  "name": "Partner Commissions",
  "collaboratorCount": 156,
  "engagementsCount": 1203
}

Junction table filtering

Collaborators enrolled in program 7:

curl "https://api.example.com/siren/v1/collaborators?fields=id,fullName,status&programId=7" \
  -H "Authorization: Bearer $TOKEN"

Programs in program group 3:

curl "https://api.example.com/siren/v1/programs?fields=id,name,status&programGroupId=3" \
  -H "Authorization: Bearer $TOKEN"

Iterating all pages

#!/bin/bash
PAGE_SIZE=25
OFFSET=0
TOTAL_PAGES=1

while [ "$OFFSET" -lt "$((TOTAL_PAGES * PAGE_SIZE))" ]; do
  RESPONSE=$(curl -s \
    "https://api.example.com/siren/v1/collaborators?fields=id,fullName,status&number=$PAGE_SIZE&offset=$OFFSET" \
    -H "Authorization: Bearer $TOKEN")

  TOTAL_PAGES=$(echo "$RESPONSE" | jq '.totalPages')
  ITEMS=$(echo "$RESPONSE" | jq '.items')

  # Process items...
  echo "$ITEMS"

  OFFSET=$((OFFSET + PAGE_SIZE))
done

Reading the estimated count header

curl -s -D - \
  "https://api.example.com/siren/v1/programs?fields=id,name&status=active" \
  -H "Authorization: Bearer $TOKEN" \
  | grep -i x-siren-estimated-count

# x-siren-estimated-count: 47

---

Quick Reference

Parameter	Type	Default	Applies To	Description
fields	string	varies by endpoint	List + Detail	Comma-separated field names to include in response.
number	integer	10	List only	Page size (max 50).
offset	integer	0	List only	Zero-based starting position.
orderBy	string	id	List only	Column to sort by.
order	string	ASC or DESC	List only	Sort direction (default varies by resource).
s	string	—	List only	Free-text search across searchable columns.
(column)	string	—	List only	Exact-match filter (comma-separated for multi-value).

---

See Also

• See the individual resource reference pages for available fields and filterable columns per endpoint.