SIGN IN SIGN UP

feat(dataset): add `collect_all_keys` option and keep CSV export columns aligned (#2079)

- Fixes CSV export writing values under the wrong column. Rows were written positionally with
  `csv.writer`, so any item whose key set or key order differed from the first item's had its values
  shifted or placed under the wrong header. Rows are now written by key with `csv.DictWriter`.
- Columns are taken from the first non-empty item. Keys that appear only in later items are dropped
  instead of being appended out of position, and an item missing a column now gets an empty cell
  instead of a short row. Exports of datasets with mixed keys therefore change content.
- Dropped keys are reported once per export in a warning that names them. Crawlee for JS drops them
  silently, so this is a Python-only addition.
- Adds a `collect_all_keys` parameter to `Dataset.export_to` and `BasicCrawler.export_data`. When
  True, the columns are the union of all item keys in first-seen order, so nothing is dropped. It
  reads the whole dataset before writing the header, unlike the default. Named and behaving like
  `collectAllKeys` in Crawlee for JS.
- Adds `restval`, the value written for columns an item has no key for. It was rejected before,
  because `csv.writer` has no such argument.
- Documents `ExportDataCsvKwargs` as mirroring `csv.DictWriter` and `ExportDataJsonKwargs` as
  mirroring `json.dump`, so Crawlee-specific options stay out of them.
- Documents the column behavior in the dataset export example.
- Alignment with Crawlee for JS: the by-key row mapping, deriving columns from the first item, and
`collectAllKeys` all match `Dataset.exportTo` in `packages/core/src/storages/dataset.ts`.

*✍️ Drafted by Claude Code*
A
Anas Khan committed
cd1e1f632cab692a54ba8db63e9f64ce3130a6d6
Parent: 3468eee
Committed by GitHub <noreply@github.com> on 7/28/2026, 12:16:31 PM