feat(S2): read an all-caps credential after a comma by default (#564) - #577
Merged
Merged
Conversation
Policy.unlisted_caps_suffixes becomes a CapsSuffixes StrEnum --
OFF / AFTER_COMMA / EVERYWHERE -- defaulting to AFTER_COMMA. The field
was new in 2.4 and unreleased; a bool now raises a TypeError naming
both replacements.
The all-caps SURNAME convention ('Jean DUPONT', 'DUPONT, Jean') writes
the capitals at the end of a name or before a comma, never after a
comma behind a full name, so the default reads that one position:
'John Smith, XYZ' gives suffix 'XYZ', as do the corpus names 'Ahmad
Jayadi, CHA', 'John Smith, RAI' and 'The Rt Hon Kenneth Clarke QC MP,
HMG', all read as the given name at 2.3.0. 'John Smith, LEED AP' now
reads suffix too, closing the #291 deviation marker C1 carried. A lone
two-letter word is never admitted at the comma (the undotted form of
#563's paired initials): 'García Márquez, MJ' keeps given 'MJ'.
EVERYWHERE adds the trailing slots, the old True; OFF is 2.3's reading
and the way to keep a capitalized given name behind a two-word surname
('García Márquez, GABRIEL', the accepted cost).
The comma test is on by default, so it checks isupper() and the
two-letter length in C before calling the shared predicate: ordinary
comma names pay no frame. rules.md S2/C1/C2, decisions.md#S2,
mechanisms.md, customize/usage/modules docs, the release log, and the
ledgers at all five baselines follow; the gate exits 0 with radar
counts unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Mixed runs (Derek): an unlisted all-caps word is a by-shape member of C1's run beside listed credentials, so 'John Smith, PhD XYZ' and 'John Smith, XYZ Jr.' read as credential runs where adding a listed credential had flipped the part back to a name. A two-letter caps word in a run is read as #563 reads a pair ('García Márquez, MJ PhD' keeps given 'MJ'; 'MJ JK' flips). - Case repair: classify tags a caps word in the part a suffix comma opened under the default too, so capitalized(force=True) keeps 'XYZ' in 'John Smith, XYZ' as EVERYWHERE does. - Frames: the comma test needs two words before the comma first, so 'Smith, JOHN' pays nothing (it paid +25); classify skips a listed member before the caps predicate. - rules.md#C1 and the release log state the two-letter rule as the code does; new examples pin it and the mixed run. Stale "off by default" prose fixed in tests, the case table, mechanisms.md (which also named a caller that does not exist) and a guard comment; decisions.md#C1 points at the S2 entry; the old recipe is translated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- An all-caps word joins C1's run only where the NAME before the comma has a lowercase letter: the first fix let a mixed-case credential supply the contrast, so 'LLOYD WEBBER, ANDREW PhD' lost its given name. 'García Márquez, JUAN Jr.' (a mixed-case name) stays in the accepted cost, now named with a case row. - rules.md#C1 points two capitals at #563's own sentences instead of restating them; the restatement contradicted them. Measured by the review: MJ and M.J. flip identically across 4036 inputs. - rules.md#R4 names the caps shape; a forced-repair test guards the classify tagging (an R4 example could not: R5 leaves mixed case unrepaired), with its negative control run. - Every caller of the caps predicate asks, in C, what it would decline anyway, so a listed credential pays no frame ('John Smith, CPA' and 'John Smith, Ph. D.' are back to master's counts). - decisions.md#S2, the release log and mechanisms.md corrected to match, including the third-comma-part render and the cost figures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ords
The second round's contrast test read every token before the comma, so
a maiden clause ('LLOYD WEBBER née Smith, ANDREW PhD') or a title ('Mr
LLOYD WEBBER, ANDREW') supplied the lowercase and an all-caps record
lost its given name again; it also cost a frame per character on an
all-caps record. name_contrast now reads the name's own words before
the comma (own_words, shared with case_class's walk rather than a
second spelling of it), titles and particles aside, in one C-level
comparison, and both caps branches ask it.
The all-caps run's later words get the same C-level prechecks as its
first, so a listed credential never reaches the caps predicate.
rules.md#C1 limits "exactly as paired initials" to mixed-case names and
states the contrast as the name's own words; decisions.md#S2 records
both wrong drafts, the 'De La Cruz García, MARÍA' accepted case, and
the cost (all-caps record +24 constant, no longer per character).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…words
The third round excluded titles and particles by list and missed a
generation ('LLOYD WEBBER Jr., ANDREW', the common SURNAME Jr., GIVEN
record), a connective ('GARCÍA y LÓPEZ, ANDREW') and a title recognized
by shape ('Insp. LLOYD WEBBER, ANDREW'): each lost its given name. The
test is inverted now: the contrast comes only from the name's own words
before the comma that no wordlist claims and no period marks, through
_vocab.in_any_wordlist -- the caps predicate's own "unlisted", lifted
out so both share it. Case rows pin the generation, connective and
later-comma-part shapes.
rules.md#C1 states MJ = M.J. as what it is, the comma's decision where
the name carries the contrast; decisions.md#S2 replaces the relabelled
4036 figure with a measurement on this tree (0 of 1295 with the
contrast, 325 of 925 without, by design) and its recompute.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Four review rounds showed that "a lowercase letter somewhere, less some
exclusions" cannot separate an all-caps record from a mixed-case name:
each draft leaked a class the next found, last glued particles and
unlisted lowercase words ("GISCARD d'ESTAING, VALÉRY", 'HAFEZ al-ASSAD,
BASHAR', 'LLOYD McDONALD, RONALD', 'LLOYD ap RHYS, DAFYDD'), all losing
the given name.
Derek's criterion: the name carries the contrast only through one of
its own words before the comma written in Title case -- S2's "written
the way a name is written" -- with no period and not claimed by the
vocabulary as non-name text. A lowercase-only word never counts, so no
list has to know 'ap' or 'thi'. That question gets its own predicate,
_vocab.claimed_as_non_name, which leaves the surname and bound-given
lists out: a draft sharing in_any_wordlist made a caller's surname list
switch the reading off. Accepted with it: an all-lowercase name ('john
smith, XYZ') reads the listing form, as do Mc/Mac-only names; an
unlisted Title-case title ('Doña') still carries the contrast.
rules.md#C1, the Policy docstring, customize.rst, the release log and
decisions.md#S2 say so; case rows pin the glued particle, the unlisted
lowercase word and the lowercase name, and a unit test the surname
list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`istitle()` accepts a single capital, so an undotted initial carried
the name's contrast and a capitalized-surname record with lowercase
particles lost its given name ('de GAULLE C, CHARLES', 'de la O GARCÍA,
MARÍA'). Title case now needs a lowercase letter behind the capital.
decisions.md#S2 records the whole class istitle() rejects when every
name word is of it (Mc/Mac and other interior capitals, the Dutch IJ,
lowercase-led elisions, a vocabulary claim beside one) rather than
Mc/Mac alone; two case notes and the cost comment state the Title-case
criterion; the Policy docstring and customize.rst add the vocabulary
exclusion.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #577 +/- ##
==========================================
+ Coverage 98.89% 98.90% +0.01%
==========================================
Files 45 45
Lines 4070 4118 +48
==========================================
+ Hits 4025 4073 +48
Misses 45 45 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…se (Derek)
Title case (`istitle()`) read too narrowly: interior capitals and
elisions ('DiCaprio', 'IJzerman', 'al-Rashid', "d'Estaing", 'McDonald')
are mixed-case names and kept a given 'XYZ'. Derek's criterion: a word
of the name carries the contrast if it holds a capital directly
followed by a lowercase letter, a leading Mc/Mac skipped where a capital
follows it (_vocab.written_as_a_name). A lowercase prefix glued to a
surname written in capitals ("d'ESTAING", 'al-ASSAD', 'McDONALD') has
no such pair, so those records keep their given names; a lone capital
and a lowercase-only word have none either.
Weighed and declined, with measurements in decisions.md#S2: "contains
a capital" (any lowercase word in an all-caps record then makes every
word a contrast) and "not all capitals but contains a capital" (takes
d'ESTAING and al-ASSAD). The accepted costs shrink to all-lowercase
names and unlisted mixed-case titles. A parametrized test pins the
predicate; case rows pin McDONALD and DiCaprio.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…letter (Derek)
The capital-then-lowercase test leaked a whole class: any capitalized
prefix glued to a surname written in capitals supplies the pair
('LLOYD FitzGERALD, RONALD', 'PAOLO DeVITO, MARCO', 'JEAN LaFLEUR,
PIERRE', 'DICK VanDYKE, JOHN', 'PAUL DuBOIS, JEAN', 'JEAN St-PIERRE,
MARC', 'LLOYD SMITH-McDONALD, RONALD'), as does ß ('MÜLLER WEIß, HANS');
each lost its given name, the Mc/Mac skip being one member of the class.
It also missed titlecase digraphs ('Džokić Ljubić, XYZ') and cost a frame
per letter.
Derek's criterion: a word carries the contrast if it holds a capital
and its last letter is lowercase, a trailing ß aside. A surname written
in capitals ends in one whatever is glued in front, so no prefix list
is needed; 'McDonald', 'Džokić' and decomposed accents pass. Two C-level
checks, so the per-letter cost is gone (a record with lowercase
particles is a constant +22 over master at any length). Measured against
the previous commit: exactly those eight prefixes and Džokić moved, no
corpus or case-table name, OFF still equal to master.
rules.md#C1, the Policy docstring, customize.rst and the release log
drop the false universal; decisions.md#S2 records the chosen option's
own leaks and the accepted hyphenated case; case rows and a broader
parametrized test pin it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The previous commit tested the last character, so a name typed with
decomposed accents ending in an accented letter lost the contrast
('José André, XYZ', 'Lê Thị Hà, XYZ', 'René Noé, XYZ' read given where
their composed spellings read suffix), as did "Jones'", 'Smith2' and a
glued 'Smith)'. written_as_a_name now composes the word (NFC), passes
over trailing non-letters, and passes over a lowercase letter with no
single capital form (ß, ĸ), which generalizes the ß rule. One call, no
generator.
Measured against the previous commit: the four decomposed-accent names
and 'JOHN Smith)' regain the contrast and the ĸ record keeps its given
name; nothing else moves, and OFF equals master on 4292 inputs.
Also from the review: rules.md#C1 states the last-letter reading and
pins 'MÜLLER WEIß, HANS'; decisions.md#S2 records the true blast radius
of the previous commit (seven prefixes plus ß and Džokić, and
'JOHN O'NEILL's' / 'JEAN-pierre DUPONT' / 'MARY-kate OLSEN', now
accepted costs) with a recompute recipe; the cost prose says what it
is, about four frames per word before the comma and constant in a
word's letters; the fix(#564) ledger comments drop the wording
decisions.md calls wrong.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… recipe The last-letter scan stopped at a caseless letter, so a mixed-case name whose words end in one lost the contrast: 'Asmāʾ Wafāʾ, XYZ' and 'JOHN Jonesʼ, XYZ' (U+02BC) kept given 'XYZ' where 'JOHN Jones', XYZ' read suffix. written_as_a_name now passes over every letter without case, as it passes over non-letters, so the rule is one sentence: the last cased letter, read after composing the word, setting aside a lowercase letter whose capital is not a single letter. rules.md#C1 states exactly that. decisions.md#S2's recipe compared against "the parent commit" while its numbers were measured against a16927c, and its grid was unnamed. It now names both comparators by SHA and lists every input, and was run as written: against a16927c exactly the intended classes move, and OFF equals master on all 4286 inputs. Correction to the previous commit's message: "nothing else moves" also missed 'JOHN SMIfi' (a ligature), which moved there and is unchanged here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov flagged the trailing `return False` as the PR's one uncovered
line. It was dead: any word failing `text == text.lower()` holds an
uppercase or titlecase letter (checked over every code point, and NFC
keeps one), and the scan returns on reaching it. The capital test now
sits on the scan's return instead, so a word with no letter carrying
case evidence ('李', '2', 'ß', '') falls through and is pinned by new
rows. Same answers as before on 1,421,107 inputs (the corpora and
case-table words, every code point, 300,000 random mixes); no frame
added, the gate unchanged at all five baselines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of d7f4dac/2479eabb: the recipe's 4286 came from a regex ("([^"\n]{2,60})") the prose did not state; reading its words literally gave 4273 (tokenize) or 4284 (an escape-aware regex), same conclusions. The prose now names the regex and records the exact-literal count beside it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #564.
Policy.unlisted_caps_suffixesbecomes aCapsSuffixesStrEnum,OFF/AFTER_COMMA/EVERYWHERE, defaulting toAFTER_COMMA. The field was new in 2.4 and unreleased, so no deprecation is owed. Passing the oldTrue/Falseraises aTypeErrorthat namesCapsSuffixes.EVERYWHERE/CapsSuffixes.OFF.Why the comma position is safe by default: the all-caps SURNAME convention (
Jean DUPONT,DUPONT, Jean) writes the capitals at the end of a name or before a comma, never after a comma behind a full name. The off default was argued for the convention's positions and was switching this one off along with them.John Smith, XYZXYZ, familyJohn SmithXYZ, reportedAhmad Jayadi, CHA/John Smith, RAICHA/RAIThe Rt Hon Kenneth Clarke QC MP, HMGHMGQC MP, HMGJohn Smith, LEED APLEEDLEED AP(closes thedeviates: #291marker)John Smith, PhD XYZPhDPhD XYZGarcía Márquez, MJMJMJ(unchanged; undotted paired initials)Smith, XYZXYZXYZ(unchanged; one name word)GISCARD d'ESTAING, VALÉRY/LLOYD FitzGERALD, RONALD/LLOYD WEBBER, ANDREW PhDJohn Smith XYZ(no comma)XYZXYZ(trailing slot needsEVERYWHERE)Decisions (Derek's calls marked in decisions.md#S2)
bool | "comma".OFFexists so a capitalized given name behind a two-word surname can still be kept (García Márquez, GABRIEL→ suffixGABRIELby default, the accepted cost).García Márquez, G.J.has no given name #563 decidesM.J.. Measured: 0 of 1295 swapped inputs differ.John Smith, PhD MEngreads firstPhD, lastJohn Smith#544 run beside listed credentials. Otherwise adding a credential (PhD XYZ,XYZ Jr.) would flip the reading back to a name.ß), not claimed by the vocabulary as a title, particle, connective, credential or generation.Smith,DiCaprio,IJzerman,al-Rashid,d'Estaing,McDonaldandDžokićcarry it. A surname written in capitals ends in a capital whatever is glued in front of it (d'ESTAING,al-ASSAD,McDONALD,FitzGERALD,DeVITO,St-PIERRE), so such a record keeps its given name, with no prefix list. Derek reached this throughistitle(), "contains a capital", "not all capitals but contains a capital" and "capital then lowercase"; each one's leaks are recorded in decisions.md#S2. The accepted costs: an all-lowercase name, an unlisted mixed-case title (Doña), and in an otherwise all-caps record a glued lowercase ending or a mixed-case hyphenated part (JOHN O'NEILL's,JOHN SMITH-Jones,JEAN-pierre DUPONT).De La Cruz, Edlose its given name? #575, without whichVan Buren, MARTINandDe La Cruz, MARIAwould have lost their given names.Implementation
John Smith, PhD MEngreads firstPhD, lastJohn Smith#544 run's caps member share onename_contrast(), which reads the own-words walkcase_classalready does.XYZ(rules.md#R4). A forced-repair test guards this, with its negative control run.in_any_wordlist(the caps shape's "unlisted") andclaimed_as_non_name(which words can carry the contrast; a caller's surname list counts as name text).Smith, John,Smith, JOHN,John Smith, PhD,John Smith, MA,John Smith, CPA,Ph. D.,MD PhD, and the benchmark reference).John Smith, XYZgoes from 251 to 266. An all-caps record with a credential pays a constant +26. Past that, the contrast test is linear in the words before the comma (about four frames a word) and constant in a word's letters.Docs and ledgers
rules.md S2/C1/C2/R4, decisions.md#S2 (plus a C1 pointer), mechanisms.md,
customize.rst,usage.rst,modules.rst(addsCapsSuffixes), the release log, and all five ledgers. Afix(#564)rule classifies the movers. #342's comma pair (RAI/CHA) reads suffix again by its capitals, so that rule'sfieldslosegivenandJohn Smith, RAI's watched shape is re-recorded.Verification
OFFagainst master's default over the corpora, everycases.pytext and generated grids of up to 388k names, with 0 differences.istitle()pitfall, and caseless letters (ʾ,ʼ,李) stopping the last-letter scan. All are fixed and pinned by case rows. The last two commits (that caseless-letter fix, and making the scan's fall-through reachable after Codecov flagged it as dead code, with the same answers on 1.4M inputs) were reviewed together: no defects; the decisions.md#S2 recipe now states the extraction regex its input count depends on.🤖 Generated with Claude Code