Skip to content

gh-154744: Improve detection of skipinitialspace in csv.Sniffer#154745

Open
serhiy-storchaka wants to merge 1 commit into
python:mainfrom
serhiy-storchaka:sniff-skipinitialspace
Open

gh-154744: Improve detection of skipinitialspace in csv.Sniffer#154745
serhiy-storchaka wants to merge 1 commit into
python:mainfrom
serhiy-storchaka:sniff-skipinitialspace

Conversation

@serhiy-storchaka

@serhiy-storchaka serhiy-storchaka commented Jul 26, 2026

Copy link
Copy Markdown
Member

skipinitialspace is now detected by parsing the sample both with and without it and comparing the two readings, instead of testing whether every field which follows a delimiter starts with a space.

  • If the readings differ in the number of fields, the one which splits the sample into rows with the more consistent widths wins. This is what recognizes the space before a quoted field: a quote is only a quote at the very start of a field, so not skipping the space splits the quoted field.
  • Otherwise a field differs between the readings only if a space was actually skipped at its start. This tells the padding apart from the spaces inside quoted or escaped fields, which are never skipped and therefore say nothing about the format.
  • The first field of a row is judged separately, because it is commonly not padded (a, b, c).

Measured on the 560 files of the CSVsniffer corpora, skipinitialspace changes the number of fields in 7 of them, and in 3 of those the correct reading collapses the rows to a single width.

Detect the padding by parsing the sample both with and without
skipinitialspace and comparing the two readings, instead of testing
whether every field following a delimiter starts with a space.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant