Skip to content

Use ASCII-only character checks in remaining format validation - #531

Open
adityakale2002 wants to merge 1 commit into
arthurdejong:masterfrom
adityakale2002:fix/ascii-only-format-checks
Open

adityakale2002 wants to merge 1 commit into
arthurdejong:masterfrom
adityakale2002:fix/ascii-only-format-checks

Conversation

@adityakale2002

Copy link
Copy Markdown

Third and last part of the non-ASCII character cleanup, after #527 (pt.cc, iso6346) and #528 (checksum modules, ie.pps). Like those, it is independent of them and based on master.

\d, str.isalpha() and str.isalnum() also match characters outside 0-9 / A-Z (e.g. '٣', ARABIC-INDIC DIGIT THREE, or 'é'), so these validators accept numbers containing them:

  • \d in a regex: br.cnpj, fi.hetu, is_.kennitala, ro.onrc, si.maticna
  • isalpha() / isalnum(): ad.nrt and es.cae (control letter), eu.banknote (two-character prefix)
>>> from stdnum.fi import hetu
>>> hetu.is_valid('131052-308T')
True
>>> hetu.is_valid('131052-30٨T')   # ARABIC-INDIC DIGIT EIGHT in place of 8
True
>>> from stdnum.es import cae
>>> cae.is_valid('ES00008V1488é')   # control letter is an e-acute
True

This switches them to explicit ASCII classes ([0-9]) and 'A' <= c <= 'Z' comparisons. Fullwidth digits are untouched, since util.clean() maps those to ASCII on purpose.

Found by replacing single characters of every known-valid number in the doctests with non-ASCII ones and checking which validators still accepted the result. I left fr.accise, eu.excise and gs1_128 alone: they do not validate the affected character at all.

Tests: one doctest per module in the existing tests/test_*.doctest files. All fail before the change and pass after; the full suite and flake8 pass.

Written with the help of Claude Code, and I reviewed the change and ran the tests.

🤖 Generated with Claude Code

Regular expressions using \d and the str.isalpha() and str.isalnum()
methods match characters outside of 0-9 and A-Z (e.g. ARABIC-INDIC DIGIT
THREE or LATIN SMALL LETTER E WITH ACUTE), so numbers containing them were
accepted as valid by br.cnpj, fi.hetu, is_.kennitala, ro.onrc, si.maticna,
ad.nrt, es.cae and eu.banknote.

Use explicit ASCII character classes and comparisons instead.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant