Pagination divides a large result into manageable portions, but its mechanism affects integration consistency. Offset is easy to understand as pages, while a cursor carries a position from the previous result. When data changes during traversal, the two can behave very differently.
How do offset and cursor pagination work?
Offset asks the server to skip a number of records and return the next limit. Requests are simple and users can jump to a page. A cursor carries a token or keyset representing the last position, then the server continues after it. The cursor need not reveal its keys and should be treated as opaque by clients.
Both need deterministic order. Sorting only by a timestamp that can repeat creates an ambiguous boundary; add a unique tiebreaker. Document fields and ordering in the data contract.
What happens when data changes?
With offset, an insertion near the beginning can shift records so one appears twice or is skipped. Deletion also changes positions. A cursor using a stable keyset continues after the last value, so earlier shifts do not affect its position. Updating the sort column can still move a record across the boundary.
If the business needs one consistent view, the API must provide a snapshot, watermark, or another mechanism. A cursor alone does not freeze the dataset.
How do performance and user experience compare?
| Aspect | Offset | Cursor |
|---|---|---|
| Jump to page | Easy | Usually sequential |
| Continuously changing data | Prone to shifting | More stable with keyset |
| Large offset query | Can grow expensive | Can use position index |
| Manual debugging | Simple number | Opaque token |
| Recovery | Store offset | Store last safe cursor |
What should a cursor contract explain?
Document lifetime, relationship with filters and sort, behavior for an invalid cursor, maximum page size, and whether reuse is allowed. A client should never modify an opaque token. Do not place secrets or personal data in a merely encoded cursor; encoding is not encryption.
For a long process, save a checkpoint only after the page is completely processed. If the middle fails, decide whether the page repeats and whether destination effects are idempotent. Connect this with the boundaries of retry.
How do filters and watermarks help?
Change polling often requests updated_at greater than a watermark. Combine timestamp with a unique ID so equal time values are not lost. A small overlap can cover delayed commits but requires deduplication. Save a watermark from source data, not the consumer clock, which may differ.
If records can be changed to an older timestamp or deleted without tombstones, that filter is insufficient. Add an official change mechanism or reconciliation. Read webhook versus polling for the broader discovery decision.
Which scenarios should be tested?
- Insert before and after the current position during paging.
- Delete a record from a page already read.
- Give two records the same sort value.
- Use an expired, damaged, or filter-mismatched cursor.
- Restart after one page succeeds.
- Return an empty page before true completion.
NitroNet or another platform may consume paginated APIs, but this article claims no particular product behavior. Use a safe large dataset and reconciliation to verify the final count, not only successful requests.
Create at least ten thousand records with repeated timestamps and unique IDs. While the client reads pages, insert, update, and delete at different positions. Compare the final record set with a reference snapshot so duplicates and omissions become visible.
- Start with sorting that lacks a tiebreaker to show why page boundaries become ambiguous.
- Add an ID tiebreaker, repeat the same mutations, and compare results.
- Insert a record before the active offset to observe shifting and duplicate reads.
- Change the sort value of an unread record so it moves across the cursor.
- Use an old cursor with a new filter and require defined rejection or behavior.
- Restart the consumer after a saved checkpoint and prove completed pages are not lost.
- Limit extreme page size so one request cannot consume excessive memory or destination time.
- Count unique IDs, total records, and a business checksum after traversal.
No pagination mechanism removes the need for a contract and reconciliation. Offset can fit user interfaces and stable datasets; cursor helps sequential traversal but needs clear semantics. When snapshot consistency is required, provide a dedicated mechanism instead of assuming a cursor freezes data.