fix: Keep pagination iterators advancing past fully-filtered pages (#964)
The offset- and cursor-based pagination iterators stopped as soon as a page's `items` list came back empty. But under dataset filters (`clean`, `skip_empty`, `skip_hidden`), the API can *scan* a full chunk (up to 1000 items) while *returning* none — `items=[]` with `x-apify-pagination-count: 1000`. The iterators broke on that page and silently never yielded any item behind it, even though the offset bookkeeping (`max(count, len(items))`) had already advanced past the scanned rows. Now all four iterators (offset + cursor, sync + async) terminate on the number of items *scanned* rather than *returned*: they keep advancing the offset/cursor across fully-filtered pages and stop only when a page scans nothing. The `limit` and `cursor is None` checks are unchanged. Behavior is identical for every response without a divergent `count` (collection endpoints, `ListOfRequests`). Added a regression test for each of the four iterators. Related to https://github.com/apify/apify-core/issues/27324. *✍️ Drafted by Claude Code*
V
Vlada Dusek committed
2e7f75ca951f2b7b59c319badd6e024d47fe67f1
Parent: 9bdca6a
Committed by GitHub <noreply@github.com>
on 7/27/2026, 12:41:42 PM