Skip to content

Commit 268b47e

Browse files
derek73claude
andcommitted
feat(vocab): add "abd", and let a dual-membership word join
"abd Allah Smith" read given 'abd', middle 'Allah' where "abdul Rahman Smith" reads given 'abdul Rahman'. The spellings that write the article as its own word -- Abd Allah, Abd al-Rahman -- match none of abdul/abdel/abdal, and the Arabic-script عبد has covered the same word since #269, so only the Latin side was short. Adding the word alone does not fix it, which is why this is not a one-line change. `abd` is also the postnominal ABD ("All But Dissertation") in SUFFIX_ACRONYMS, and the join's reserve counted every piece except titles and suffixes -- so the bound word's own piece did not count toward the three it needed, and the rule declined silently on the common three-word shape. The claimed piece now counts, which is what the reserve means: it asks whether enough OTHER words are left to spare, and this piece is not one of them to spare. Measured, the vocabulary-only alternatives each cost something this does not: moving `abd` to SUFFIX_ACRONYMS_AMBIGUOUS loses "Jane Smith, ABD", and dropping it from SUFFIX_ACRONYMS loses the postnominal outright. Keeping both readings costs nothing here -- position tells them apart, and all three credential spellings still parse. `abd` is the only shipped word in both sets (asserted by measurement, not assumption), so nothing else moves; all 751 differential corpus names are byte-identical and the harness reports 0 unexplained. Same shape as #397, where dual membership shrinks the rootname count P3's carve-out tests -- recorded at decisions.md#P5, with the general question left open on that issue. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 55a7822 commit 268b47e

10 files changed

Lines changed: 160 additions & 5 deletions

File tree

‎docs/design/decisions.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -310,10 +310,15 @@ Excluded (SUFFIX_ACRONYMS / SUFFIX_WORDS — the esq dual membership, deliberate
310310

311311
- 2026-06-30 (first-name-prefix-join design; v1-era, carried into the v2 port) — the join is vocabulary-driven and deliberately tiny.
312312

313+
- 2026-08-18 — `abd` ADDED, and the reserve count corrected to let it work. It is the transliteration for the spellings that write the article as its own word ("Abd Allah", "Abd al-Rahman"), which abdul/abdel/abdal do not match; the Arabic-script عبد has covered the same word since #269, so only the Latin side was short. Not a C-ii collision, though it looks like one: C-ii settles a PRECEDENCE by frequency where neither reading is a name, and here one reading is a name and neither had to be given up. It is C-i's positional qualifier (2026-08-17) applied to a bound-given word — the naming use is LEADING, the credential use trailing, which is the `de` row of that table ('a TRAILING surname — no clash') rather than the `vd` row. `abd` is also the postnominal ABD ("All But Dissertation") in SUFFIX_ACRONYMS, and all three credential spellings still parse ("Jane Smith ABD", "Jane Smith, ABD", "Jane Smith A.B.D.").
314+
What made it look impossible: adding the word alone changed nothing on the common three-word shape. The reserve count in the join gate excluded suffix-vocabulary pieces, and `abd` IS one, so its own piece did not count toward the three it needed — the rule declined silently. Counting the claimed piece as a name piece fixes it, and is right on its own terms: the reserve asks whether enough OTHER words are left to spare. Measured, the vocabulary-only alternatives both cost something the count fix does not — moving `abd` into SUFFIX_ACRONYMS_AMBIGUOUS loses "Jane Smith, ABD", and dropping it from SUFFIX_ACRONYMS loses the postnominal entirely.
315+
Two limits, both measured and both accepted. The three-token "abd al rahman" still does not join — `al` is a particle and chains forward, which is the half of the original exclusion that survives. And position decides at the two ENDS only: in the given slot of a family-comma name the credential still wins, so "Smith, Abd" reports suffix 'Abd' with no given name where "Smith, Abdul" reports the given name, and no ambiguity is emitted for it.
316+
Same shape as #397 (Catalan `i`, where dual membership shrinks the rootname count P3's carve-out tests), and the second instance the SAME DAY, #397 having been filed two and a half hours earlier. The general question — whether a word's membership in one set should suppress it from a count gating a rule keyed on another — is open there.
317+
313318
Excluded (BOUND_GIVEN_NAMES):
314319

315320
- mohamad — a standalone given name in its own right; binding it would eat the middle name.
316-
- abd — collides with the academic post-nominal "ABD", and the real form is the deferred multi-token "abd al rahman".
321+
- abd — ADDED 2026-08-18, see the entry above; the exclusion stood on two reasons and only one of them survived. The collision with the post-nominal "ABD" turned out to cost nothing (position decides at both ends). The other reason stands as a known limit: the three-token spelling "abd al rahman" is still not joined, `al` being a particle that chains forward, so only the two-token "abd X" and the hyphenated "abd al-Rahman X" join.
317322

318323
Excluded (DEFAULT_NICKNAME_DELIMITERS):
319324

‎docs/design/mechanisms.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,7 @@ Contract statement. A workaround keyed to a third-party library's measured defec
142142
### Field notes — the traps themselves
143143

144144
- Enumerate the rules that BUILD a structure; do not recall them. #395's unit walk was written three times in one PR — P2's chain missing, then the conjunction and bound-given branches absorbing one token where the particle branch absorbed a unit, then the suffix stop — and each miss came from listing the joining rules from memory instead of reading them out of rules.md. Both later misses reproduced the very defect the first fix had just removed, mirrored.
145+
- Re-exec a module in-process and its enums become NEW classes. Every `is` comparison against the ORIGINAL module's member then answers wrong and silently: `_group.py`'s `bound_join is not BoundJoin.DISABLED` compared across two `BoundJoin` classes and was always true, so a family comma's own segment joined when it must not, and a mutation sweep reported 3,150 phantom movers. Rebind the identity-compared names from the real module after the exec (`mod.__dict__["BoundJoin"] = real.BoundJoin`), and sanity-check one input the mutation must NOT move before believing any count.
145146
- Assert which tree you imported, on BOTH sides of a comparison. `python -c` puts CWD on sys.path; a script's own directory holds no nameparser in tools/differential/, so a stray PYTHONPATH outranks the editable install — measured: 89 diffs became 0, exit 0, both tell halves passing, because both sides had become the shadow.
146147
- Release notes describe intent; TAGS are ground truth. Before trusting a changelog cross-reference about what shipped where, run `git diff <tagA> <tagB> -- <file>` — the "1.2.1 pickle shim" claim recurred twice from a changelog line describing a thing that never shipped.
147148
- Never pipe a gate's output. Under zsh, `compare.py | tail` makes `$?` tail's status. Redirect to a file and read the file.

‎docs/design/rules.md‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -226,10 +226,20 @@ P4. Rationale: a particle links forward from inside a name; at the
226226
P5. Rationale: some given-name words are incomplete alone — "abdul"
227227
is a bound form that the next word completes.
228228
A recognized bound given-name word joins the word after it into
229-
one given name.
229+
one given name. It needs a name word to spare, so two name words
230+
alone do not join — the second is the family name — except after
231+
a family comma, where the family is already fixed. Where the word
232+
is BOTH bound-given and suffix vocabulary, position decides and
233+
both readings survive: leading, it is the bound word; trailing,
234+
it is the suffix (S2). In the given slot after a family comma the
235+
suffix reading wins.
230236
"abdul salam ahmed salem" → given="abdul salam"
237+
"abd Allah Smith" → given="abd Allah"
238+
"Salam, abd Allah" → given="abd Allah"
239+
"abd Allah" → given="abd"
240+
"Smith, Abd" → suffix="Abd"
231241
"mohamad ali smith" → given="mohamad" · boundary
232-
history: decisions.md#P5 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_post_rules.py
242+
history: decisions.md#P5 · interacts: S2 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_post_rules.py
233243

234244
P6. Rationale: a particle ending the name has nothing to link
235245
forward to, so it is not doing a particle's work there. A

‎docs/release_log.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ Release Log
3232

3333
- Fix a name opening with a particle that is *never* a given name being split at the particle under a family-first name order -- ``Policy(name_order=FAMILY_FIRST)`` and ``Policy(name_order=FAMILY_FIRST_GIVEN_LAST)`` alike, and identically: ``"de Mesnil"`` read as family ``de``, given ``Mesnil``, and ``"de la Vega"`` as family ``de``, given ``la Vega``. Each is now the whole surname, as it has always been in the default order. The rule enforcing it asked for the particle by the ``GIVEN`` role, which under a family-first order belongs to the token *after* the particle, so the test read the wrong word and declined. It now also asks by position -- the piece that opens the name -- so both shapes of the same rule are caught: where such a particle stands alone as a piece, either opening the name or in the given position, the name is left with no given name at all, the given and the middles folding into the family. Standing *alone* is the whole of it, and the rule claims nothing wider: ``"Juan de la Vega"`` under ``FAMILY_FIRST`` still reports given ``de la Vega``, because there the particle chained onto the words after it rather than standing alone, and a bare ``"de"`` with nothing to fold into is still reported as the given name. The decision behind the fix: a word that can never be a given name leaves ``name_order`` nothing to decide, so declaring family-first is not a reason to make ``de`` a surname on its own. A leading particle that *may* be a given name is genuinely order-dependent and is untouched -- ``"van Gogh"`` still reads as family ``van``, given ``Gogh`` under both family-first orders. This is also what gives ``Lexicon.particles_ambiguous`` an effect outside the default order: taking a word out of it now changes the parsed fields under a family-first order, where before it moved only the ambiguity report. Seven of the 751 differential corpus names move, the same seven under each family-first order; default-order output is byte-identical over all 751, at the 1.4.0, 2.0.0 and 2.1.0 differential baselines alike (closes #359)
3434

35+
- Add ``abd`` to :data:`~nameparser.config.bound_given_names.BOUND_GIVEN_NAMES`, so the spellings that write the article as its own word join like the others do: ``"abd Allah Smith"`` was given ``abd``, middle ``Allah`` and is now given ``abd Allah``. ``abdul``, ``abdel`` and ``abdal`` were already there, and the Arabic-script ``عبد`` has covered the same word since 2.0, so only the Latin spelling was short. The word is also the postnominal ABD ("All But Dissertation") and stays in ``SUFFIX_ACRONYMS``: position tells the two readings apart, so ``"Jane Smith ABD"``, ``"Jane Smith, ABD"`` and ``"Jane Smith A.B.D."`` all still read the credential as a suffix. Making that work needed one fix beyond the wordlist. The join reserves enough words to leave a family name behind, and the reserve counted every piece except titles and suffixes -- so a word in BOTH vocabularies did not count as a name word even when the rule had already claimed it as one, and the join declined silently on the common three-word shape. The claimed piece now counts, which is what the reserve means: it asks whether enough OTHER words are left to spare. No shipped word other than ``abd`` is in both sets, and for any word that is not, the changed expression reduces to the old one -- so nothing else can move, by arithmetic rather than by sampling. The differential corpus agrees at all 751 names, but it holds no name carrying a dual-membership word, so that agreement tests the corpus rather than the change (#400)
36+
3537
- Change how far a leading never-given particle takes the surname when a family-first ``name_order`` is declared. ``Policy(name_order=FAMILY_FIRST)`` read ``"de Mesnil Juan"`` as family ``de Mesnil Juan`` -- the whole name -- and now reads family ``de Mesnil``, given ``Juan``. Declaring a family-first order asserts that what follows the family is not more surname, and where the surname run ends is exactly that question, so the declaration settles it. The default order is unchanged, deliberately: with no order declared nothing marks where the surname ends, and a particle followed by several words really can be all surname -- ``von Bergen Wessels`` is one such name. Nothing in ``"de Mesnil Juan"`` distinguishes it from that reading except a declared order or a comma. A caller who means family ``de la Vega`` plus given ``Juan`` in the default order writes the comma, which already parses that way. The run takes one name WORD rather than one token: a conjunction-joined run and a bound given-name pair each count once, so the stop cannot land inside one -- ``"de la Vega y Santos Juan"`` reads family ``de la Vega y Santos``, and ``"ibn Awf abdul Rahman"`` reads given ``abdul Rahman``. Where two or more words are left over the two family-first orders differ from each other for the first time: ``"de la Cruz Juan Carlos"`` reads given ``Juan``, middle ``Carlos`` under ``FAMILY_FIRST`` and middle ``Juan``, given ``Carlos`` under ``FAMILY_FIRST_GIVEN_LAST``. An ambiguous leading particle is untouched in every order -- ``"van Gogh Jan Pieter"`` still reads family ``van`` under both family-first orders -- and so is a family comma, where the comma has already fixed the surname (``"Smith, de Mesnil"`` keeps family ``Smith de Mesnil``). One of the 751 differential corpus names moves, ``"de Mesnil Garcia"`` to family ``de Mesnil``, given ``Garcia``, under each family-first order; default-order output is byte-identical over all 751. This reverses the answer #364 was closed on, and the reasoning is recorded at ``docs/design/decisions.md#P1`` (closes #395)
3638

3739
- Change the ``detail`` text of a ``PARTICLE_OR_GIVEN`` ambiguity to name the role the leading particle was actually given. It said "read as a given name" under every ``name_order``, which is false under ``Policy(name_order=FAMILY_FIRST)`` -- there ``"Van Johnson"`` reads as family ``Van``, given ``Johnson``, and the report described the reading not taken. It now ends "read as a family name" in that case, reading the role off the assigned token the way ``SUFFIX_OR_NAME`` already did -- that kind names both parts (``read as a family name rather than a post-nominal``), while this one names only the part it took. The ``kind`` is unchanged and stays ``PARTICLE_OR_GIVEN``: the fork really is particle-or-given, and only the human-readable text moved. Default-order output is identical (#355)

‎nameparser/_pipeline/_group.py‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -342,8 +342,18 @@ def merge(lo: int, hi: int, add: Set[str] = frozenset(),
342342
and len(pieces[first_name_k]) == 1
343343
and "vocab:bound-given"
344344
in tokens[pieces[first_name_k][0]].tags):
345+
# first_name_k counts as a name piece even when it is
346+
# ALSO suffix vocabulary. The reserve asks whether enough
347+
# OTHER words are left to spare, and this piece is the one
348+
# the rule has already claimed as a name -- excluding it
349+
# made a dual-membership word silently un-joinable --
350+
# found while adding 'abd' ("All But Dissertation" as well
351+
# as عبد), which this had to be fixed for, though it is
352+
# not why the word was excluded. Same shape as the count
353+
# #397 describes.
345354
non_suffix = sum(1 for k in range(len(pieces))
346-
if not title(k) and not suffix(k))
355+
if not title(k)
356+
and (k == first_name_k or not suffix(k)))
347357
if non_suffix >= bound_join:
348358
merge(first_name_k, first_name_k + 2)
349359
return pieces, ptags

‎nameparser/config/bound_given_names.py‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,10 @@
66
#: rule on the FIRST non-title piece, so it is not about roles -- it
77
#: fires whatever name_order later assigns. It reserves a piece for what
88
#: follows: three pieces that are neither title nor suffix in a main
9-
#: segment, which is why two-word "abdul salam" stays given "abdul" plus
9+
#: segment -- counting the bound word's OWN piece even where that word is
10+
#: also suffix vocabulary, since it is the piece the rule has claimed
11+
#: rather than one left to spare -- which is why two-word "abdul salam"
12+
#: stays given "abdul" plus
1013
#: family "salam"; only two after a family comma, where the family name
1114
#: is already fixed ("salam, abdul rahman" → given "abdul rahman").
1215
#: Mirrors :py:data:`~nameparser.config.particles.PARTICLES`, which
@@ -15,6 +18,22 @@
1518
'abdul',
1619
'abdel',
1720
'abdal',
21+
# The bare transliteration, which abdul/abdel/abdal do not match:
22+
# "abd Allah Smith" -> given "abd Allah", and "Abd al-Rahman
23+
# Smith" likewise, al-Rahman being ONE token. The three-token
24+
# spelling "abd al rahman smith" is still not joined -- `al` is a
25+
# particle and chains forward -- and stays deferred, as it was
26+
# when this word was excluded. Same word as عبد below, which has
27+
# covered the Arabic-script side since #269 (shipped in 2.0).
28+
#
29+
# Collides with the postnominal ABD ("All But Dissertation"),
30+
# which stays in SUFFIX_ACRONYMS. Position decides at the two ends
31+
# -- a leading `abd` reads as a name, a trailing one as the
32+
# credential -- but not everywhere: in the given slot of a
33+
# family-comma name the credential still wins, so "Smith, Abd"
34+
# reports suffix 'Abd' and no given name, where "Smith, Abdul"
35+
# reports the given name. Recorded at decisions.md#P5.
36+
'abd',
1837
'abu',
1938
'abou',
2039
'umm',

‎nameparser/config/suffixes.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -204,6 +204,12 @@
204204
'aas',
205205
'aba',
206206
'abc',
207+
# "All But Dissertation". Also the Latin transliteration of the
208+
# Arabic bound given-name word in BOUND_GIVEN_NAMES (whose
209+
# Arabic-script counterpart عبد is in that set only) -- the one
210+
# word in both sets, kept in both deliberately. Position decides
211+
# at the two ends but not in a family comma's given slot; see
212+
# decisions.md#P5.
207213
'abd',
208214
'abpp',
209215
'abr',

‎tests/test_bound_given_names.py‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,3 +147,60 @@ def test_opt_out_via_clear(self) -> None:
147147
self.m(hn.middle, "salam ahmed", hn)
148148
self.m(hn.last, "salem", hn)
149149

150+
# --- 'abd', which is also the postnominal ABD ---
151+
def test_abd_joins_the_word_after_it(self) -> None:
152+
"""The spelling that writes the article separately."""
153+
hn = HumanName("abd Allah Smith")
154+
self.m(hn.first, "abd Allah", hn)
155+
self.m(hn.last, "Smith", hn)
156+
157+
def test_abd_joins_after_a_family_comma(self) -> None:
158+
hn = HumanName("Smith, abd Allah")
159+
self.m(hn.first, "abd Allah", hn)
160+
self.m(hn.last, "Smith", hn)
161+
162+
def test_abd_pairwise_like_abdul(self) -> None:
163+
"""Joins ONCE: 'Rahman' pairs, 'Ahmed' stays a middle name."""
164+
hn = HumanName("abd Rahman Ahmed Salem")
165+
self.m(hn.first, "abd Rahman", hn)
166+
self.m(hn.middle, "Ahmed", hn)
167+
self.m(hn.last, "Salem", hn)
168+
169+
def test_abd_joins_with_a_suffix_present(self) -> None:
170+
"""The line this fix changes is a SUFFIX count, so the shapes
171+
that carry a real suffix piece are the ones it governs."""
172+
for text, suffix in (("abd Allah Smith jr", "jr"),
173+
("abd Allah Smith PhD", "PhD"),
174+
("abd Allah Smith, PhD", "PhD"),
175+
("abd Allah Smith III", "III")):
176+
hn = HumanName(text)
177+
self.m(hn.first, "abd Allah", hn)
178+
self.m(hn.middle, "", hn)
179+
self.m(hn.last, "Smith", hn)
180+
self.m(hn.suffix, suffix, hn)
181+
182+
def test_both_readings_of_abd_in_one_name(self) -> None:
183+
"""Position tells them apart -- the claim the collision rests
184+
on, in a single string: bound given name in front, postnominal
185+
behind."""
186+
hn = HumanName("abd Allah Smith ABD")
187+
self.m(hn.first, "abd Allah", hn)
188+
self.m(hn.last, "Smith", hn)
189+
self.m(hn.suffix, "ABD", hn)
190+
191+
def test_abd_joins_behind_a_title(self) -> None:
192+
hn = HumanName("Dr. abd Allah Smith")
193+
self.m(hn.title, "Dr.", hn)
194+
self.m(hn.first, "abd Allah", hn)
195+
self.m(hn.last, "Smith", hn)
196+
197+
def test_the_abd_postnominal_still_reads_as_a_suffix(self) -> None:
198+
"""The collision: ABD is also All But Dissertation. Position
199+
tells the two apart, so neither reading had to be given up."""
200+
for text in ("Jane Smith ABD", "Jane Smith, ABD",
201+
"Jane Smith A.B.D."):
202+
hn = HumanName(text)
203+
self.m(hn.first, "Jane", hn)
204+
self.m(hn.last, "Smith", hn)
205+
self.assertTrue(hn.suffix.upper().replace(".", "") == "ABD",
206+
f"{text}: suffix={hn.suffix!r}")

0 commit comments

Comments
 (0)