You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
match_type uses re.match with an unanchored INTEGER alternative, so misspelled types (int24, intbanana) classify as native integers #1530
match_type uses re.match rather than re.fullmatch, and the INTEGER pattern
anchors $ on only one of its alternatives. Misspelled types are therefore classified as
native integers and emitted into the DDL verbatim, so the user gets a MySQL syntax error
instead of Unsupported attribute type.
try:
returnnext(categoryforcategory, patterninTYPE_PATTERN.items() ifpattern.match(attribute_type))
exceptStopIteration:
raiseDataJointError("Unsupported attribute type {type}".format(type=attribute_type))
pattern.match anchors only at the start. Most patterns compensate with a trailing $,
but in INTEGER (declare.py:66) the $ binds to the final alternative only:
Because | has the lowest precedence, this reads as (…int|integer)(…)?(…)?(…)?orserial$. The integer branch has no terminal anchor,
so int followed by anything at all matches. The same latent issue applies to any pattern
whose $ is inside an alternation.
Impact
A typo in a definition produces a confusing failure at the database layer rather than a
clear DataJoint error. int24 is the case most likely to be hit in practice: it looks
like a plausible core type, migrate.py actually emits it (see the companion issue on uint*/int24 markers), and it is accepted here — then MySQL rejects int24 as a
column type with a syntax error that does not mention the attribute name.
Proposed fix
Use fullmatch, and drop the now-redundant $ anchors:
fullmatch is the safer form here because it cannot be defeated by an unanchored
alternative in any individual pattern. Every TYPE_PATTERN entry needs checking for
whitespace tolerance if the $ anchors are removed in the same pass — CODEC=r"<.+>$"
and the parameterized string/decimal patterns are the ones to look at.
Note that heading.py:520,537,545 and codecs.py also call .match() against TYPE_PATTERN; those call sites should move to fullmatch together with this one so the
declaration path and the load path agree on what a type is.
Worth a test that walks a list of near-miss spellings (int24, intbanana, float64x, varchar, enum()) and asserts each raises.
Correction to the "Proposed fix" section above, based on implementing it in #1531.
I wrote that the TYPE_PATTERN call sites in heading.py and codecs.py should move
to fullmatch together with match_type. They should not. Those match against
types reported by the server, which are not normalized, and PostgreSQL returns
spellings that only classify on their leading word:
double precision -> FLOAT via prefix "double"
timestamp without time zone -> TEMPORAL via prefix "timestamp"
fullmatch there breaks the PostgreSQL backend. adapters/postgres.py:949 parse_column_info normalizes character varying, character and numeric, but not
these.
So #1531 changes match_type only — the declaration path, where the input is the
user's own type spelling and a full match is what we want — and leaves the load path on
prefix matching, with a comment at the call site explaining why the two differ.
The real fix for the load path is for each adapter to normalize the types it reports to
a canonical set, at which point both paths can be strict. That is a larger change and
wants its own issue rather than riding along here.
Activity
dimitri-yatsenko commentedon Aug 10, 2026
Correction to the "Proposed fix" section above, based on implementing it in #1531.
I wrote that the
TYPE_PATTERNcall sites inheading.pyandcodecs.pyshould moveto
fullmatchtogether withmatch_type. They should not. Those match againsttypes reported by the server, which are not normalized, and PostgreSQL returns
spellings that only classify on their leading word:
fullmatchthere breaks the PostgreSQL backend.adapters/postgres.py:949parse_column_infonormalizescharacter varying,characterandnumeric, but notthese.
So #1531 changes
match_typeonly — the declaration path, where the input is theuser's own type spelling and a full match is what we want — and leaves the load path on
prefix matching, with a comment at the call site explaining why the two differ.
The real fix for the load path is for each adapter to normalize the types it reports to
a canonical set, at which point both paths can be strict. That is a larger change and
wants its own issue rather than riding along here.
fix: validate attribute types at declaration and insert (#1527, #1528, …