Skip to content

feat: spec-change.py links reads markers in submodules and answers error when it cannot decide (#548) - #572

Merged
kiki830621 merged 47 commits into
mainfrom
idd/548-spec-change-py-links-submodule-missing
Nov 16, 2026
Merged

kiki830621 merged 47 commits into
mainfrom
idd/548-spec-change-py-links-submodule-missing

Conversation

@kiki830621

@kiki830621 kiki830621 commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Refs #548

Summary

spec-change.py links 原本只在主 repo 找 Example 的標記,測試放在 submodule 的專案一律回 missing。這個 PR 讓它也讀 submodule,並在無法判斷時回 error。

  • links 自己列出 head 的 gitlink,逐一在記錄的 commit 上搜尋(含巢狀)。一個 Example 有三種結果:找到標記是有連結;沒找到且每個 gitlink 都搜尋過是 missing;沒找到且有 gitlink 搜尋不了是 error。
  • 只在符合六個條件的 submodule 目錄執行 git;執行時關掉 lazy fetch、拿掉六個繼承的變數(GIT_DIR、GIT_WORK_TREE、GIT_INDEX_FILE、GIT_OBJECT_DIRECTORY、GIT_ALTERNATE_OBJECT_DIRECTORIES、GIT_COMMON_DIR)。目錄的比對看的是作業系統回報的裝置與 inode,不是路徑名稱。
  • 解析後落在規格目錄裡的 submodule 不搜尋,不論它的路徑叫什麼。
  • 沒有 submodule 的 repository,答案不變。
  • Requirement「Added and modified requirements carry an executable check」的 Example「markers in submodules」從十列加到六十五列;九個 Spectra change 都已封存。

驗證

維護者的決定

…"markers in submodules" (#548)

One check per row of the Example table: a marker in a submodule at the recorded commit, in a nested submodule, under a path with a space, only in the recorded commit, only in a later commit, only in a .md file, nowhere, in a submodule that is not checked out, in the main repository beside such a submodule, and a recorded commit that the object store lacks. On the script as it stands, rows 1 to 4, 8 and 10 fail.
…ror when it cannot decide (#548)

links lists the gitlinks of the head revision itself, searches each checked-out submodule at the commit the tree records (nested ones too), and names a file there by the submodule's path. An Example whose marker is found nowhere while a submodule could not be searched makes the command answer error and name the Example, the path and the reason; a marker found elsewhere still counts. A repository without gitlinks gives the same output as before.
…d reasons, a nested submodule not checked out, a submodule under the spec directory (#548)

Five more checks for the Example "markers in submodules" (rows 11 to 15), and a message before the suite ends on a failed fixture step. On the script as it stands, rows 11 and 12 fail: with submodule.recurse set, git grep recurses by itself and a recorded commit that is absent ends it with status 128.
With submodule.recurse set in the git configuration, git grep on a tree walked the submodules itself: it skipped one that is not checked out without saying so and ended with status 128 on a recorded commit it could not read, so an Example whose marker is in the main repository was answered with error. Every grep now carries --no-recurse-submodules; links lists and searches the submodules itself, as before.
…f the Example "markers in submodules" (#548)

Twenty-six checks for the repairs after the first verify round: a path that is
not valid UTF-8, a tree or a file that cannot be read, a directory of tracked
files shaped like a git directory, paths that leave the repository, a path that
ends in a space, a submodule under the spec directory, inherited git variables,
partial clones, the order and the length of the reason. Rows 21, 22 and 31
check that a command named in a planted config does not run; row 22 also logs
every git call. The fixtures of rows 4 and 5 now stop when a step fails.

On the script of afd0021 this file gives 587 passing and 61 failing checks.
…t only in a checked-out submodule (#548)

The first verify round found that the submodule search could run a command
named in files of a commit, and that a failure inside a submodule could
override a marker already found in the main repository.

- A marker found in the main repository settles the Example: no gitlink is
  listed and no git command runs in a submodule.
- git runs in a submodule's directory only when the recorded path stays
  inside the repository, the directory is not a symbolic link, holds its own
  .git entry, is the top of its own working tree and holds the recorded commit.
- There git runs with safe.bareRepository=explicit, without lazy fetching and
  without six inherited GIT_* variables.
- A failure inside a submodule counts as "could not be searched"; the reason
  is one of five fixed texts and carries no text from git.
- Paths are handled as bytes; gitlinks under the spec directory are skipped;
  the reason names at most five submodules, in the order of their paths.

A repository without gitlinks, and one whose markers are all in the main
repository, give the same output as before this branch.
…error stream and checks the git version — rows 42 to 49 of the Example "markers in submodules" (#548)

Rows 42 to 48, the block "top directory names", the row for a path under the spec directory that is not
valid UTF-8, the confirmation that rows 11 and 12 really set submodule.recurse, and the check of row 22
that no gitlink is listed once the main repository has the marker.

Added after the mutation run of the acceptance: row 49 (an object that is present but damaged), the run
of row 47 with a git version that cannot be read, and the row of "what the link check reports" for a tree
of the main repository that cannot be read (the reason is what git wrote, cut to 200 characters).
…g its error stream, and links, archived and preflight read the top directory alike (#548)

- repo_top() removes only the line end git adds; links uses it again, so the three subcommands agree
  about a directory whose name ends in a space.
- A submodule is searched even when its object store lacks objects of the recorded commit; a file that
  counts settles the Example. Without one, the submodule counts as not searched when objects are
  lacking, when the search exits with a code other than 0 or 1, or when its tree cannot be listed.
  What git writes to its error stream no longer decides this.
- The submodules inside a submodule whose search failed are still searched when its tree can be listed.
- git older than 2.45, or git whose version cannot be read, does not run in the directory of a
  submodule: GIT_NO_LAZY_FETCH needs 2.45.
- A path under the spec directory that is not valid UTF-8 gives a fixed reason.
- The main repository's failed search is reported as before this branch.
…put of a search that ended early — rows 50 to 54 of the Example "markers in submodules" (#548)

Rows 50 to 54, the row for a path that is not valid UTF-8 outside specs/, three rows for a top directory
whose name ends in a line feed, and a check of which 200 characters of git's message the main
repository's failed search keeps (a git that fails with a known text of 300 characters).

The suite itself: it stops with one sentence when git is older than 2.45, it clears the git variables it
could inherit from the caller, and a fixture step that fails inside a command substitution now stops
the whole suite.
…at a submodule's search printed before it ended early (#548)

- submodule_lacks_objects(): the output of rev-list is split at line feeds only (a file name can hold a
  carriage return), and the object ids are given to cat-file --batch-check, through the wrapper that
  turns lazy fetching off. An object file that is present but empty or unreadable is reported missing
  there, while rev-list lists it as present and the search skips the file with exit code 0 or 1.
- When the search of a submodule ends with a code other than 0 or 1, what it printed is not read: how
  much it printed depends on its worker threads, and the same repository answered ok on some runs and
  error on others.
- spec-wrapper.md: git submodule update --init --recursive also contacts the addresses in the
  .gitmodules of each recorded submodule commit, and checks the content out.
The default branch gained #547 and release 4.1.0 after this branch started. Three files needed a hand:

- spec-change.py: both sides changed what repo_top() returns. The result is one_line(os.fsdecode(p.stdout)):
  the helper of 4.1.0 with the decoding this branch already had, so that a name that is not valid UTF-8 is
  read and not refused. The same expression in marker_files() calls the helper too. No return value changes.
- spec.md: both sides added requirements at the end of the file. Both blocks are kept as they were; the two
  requirements that say the same thing are handled by a change of their own.
- CHANGELOG.md: git had put the entry of this issue inside the released 4.1.0 section. It moves to a new
  [Unreleased] section, and the 4.1.0 section is the default branch's text.

All 75 suites pass on the merged tree.
…ple released in 4.1.0 (#548)

The requirement this branch wrote for the top directory is removed by the change
idd-548-one-top-directory-requirement, because release 4.1.0 states the same rule. The checks are unchanged;
the marker and the title of the block now name the released Example, whose table takes their six rows.
…hipped (#548)

Reading the top directory as git prints it is part of 4.1.0. The entry now says what this branch adds on top
of it: six rows of the Example.
The requirement this branch wrote for reading the top directory is removed: the requirement released in
4.1.0 states the same rule for all five subcommands. The six rows of its Example become rows of the released
Example.
…ry — row 55 of the Example "markers in submodules" (#548)

The change idd-548-fix5 and the check of its new row. The row fails here: the script still answers ok and runs git in the planted directory.
…sitory's own git directory (#548)

Condition 2 says the directory lies inside the working tree of the main repository. The git directory is under the top directory and is no part of the working tree: such a directory now counts as leaving the repository, and git is not run in it. The git directory is located by asking git once, when the first submodule directory is examined, and directories are compared by identity, not by name.
…ates; the suite stops when its temporary directory cannot be made (#548)

Row 29 now adds seven submodules whose recorded commit holds the marker, confirms that, and deinitialises them; its assertions are unchanged. The checks of rows 7 to 12 of the Example "a top level whose name ends in whitespace" are labelled with that title. No assertion changes.
…nd which answer changes with submodule.recurse on (#548)
…nd nothing else — rows 56 to 59 of the Example "markers in submodules" (#548)

Row 56 fails on this commit (the script is not changed yet): in a linked worktree whose top directory holds the
main repository, a clone placed in the repository's git directory is searched. Rows 57 to 59 state present
behaviour and pass: the git directory of a submodule is not excluded; a top directory inside the git directory;
a recorded path that resolves to the top directory. The change idd-548-fix6 comes with its documents.
…linked worktree that holds the main repository is covered (#548)

At the top directory git is asked for two directories, rev-parse --absolute-git-dir and --git-common-dir. A
submodule directory, or a directory between it and the top directory, that is one of the two is not searched and
git is not run there. In an ordinary repository the two are one directory; in a linked worktree the first is the
private directory of the worktree and the second the repository's git directory, which was not compared before.
The walk still ends below the top directory. Row 56 passes; 825 checks pass.
…on as the script checks it (#548)

The sentence on which directories are searched no longer speaks of the working tree: it names the two directories
git prints at the top directory and says that no other directory is excluded.
…rows 57 to 59 show the directory searched (#548)

Added before archiving idd-548-fix6. On the input of row 56, with a stand-in for git that prints nothing when
asked for the common directory, links must end in error without running git in the planted directory: three of
the new checks fail on this commit, because the empty answer is joined to the top directory, which is never
compared. Rows 57, 58 and 59 expect ok; each now also shows, in a log of the git calls, that git searched the
directory its row describes, and row 57 has a control without the planted clone.
…wo questions (#548)

An empty answer to rev-parse --absolute-git-dir or --git-common-dir was joined to the top directory, and the top
directory is never compared: the directory under the repository's git directory was then searched. An empty answer
now ends the run, as an answer that cannot be read does. Real git does not print an empty answer here; a stand-in
for git does. 840 checks pass.
Two whole runs before archiving (75 suites and no failure at 4af6539; at 6f91370 one suite that this change does
not touch failed once and passed when run alone), the two tests-first pairs, four mutations, the layout of row 1
of round 6 under four scripts, the relative answer of git from other working directories, and the call counts.
…in another letter case — rows 60 to 63 of the Example "markers in submodules" (#548)

Rows 60 and 61 fail on this commit (the script is not changed yet), on a volume that does not distinguish letter
case: a submodule reached through a link whose target names the top directory in another letter case answers
error (path leaves the repository), and one whose recorded path differs from the directory on disk in letter case
only answers error (not checked out). Row 62 (a link to a directory outside the top directory, named in another
letter case) passes. Row 63 (the input of row 55 through such a link) gives the right answer for the wrong reason:
the check that git was asked for the two directories fails. On a volume that distinguishes letter case the four
rows are not run and the suite says so. The change idd-548-fix7 comes with its documents.
…ath names (#548)

Whether a submodule's directory lies under the top directory was decided by comparing resolved path strings, and
so was whether git names that directory as the top of its own working tree. A resolved path keeps the spelling it
was given, so a second spelling of one directory (another letter case on a volume that does not distinguish letter
case; on macOS, the data-volume path) answered error. Now one walk goes from the resolved directory up through its
parents and compares each with the top directory by device and inode; the directories it passes are the ones
compared with the two git directories, and the comparison for condition 5 is made the same way. Git is still asked
for the two directories only after the top directory was met. Rows 60 to 63 pass (868 checks, none failing).
…atever its path is named — rows 64 and 65 of the Example "markers in submodules" (#548)

Both rows fail on this commit (the script does not compare with the spec directory yet): a submodule whose
recorded path names the spec directory in another letter case (row 64, on a volume that does not distinguish
letter case) and one reached through a symbolic link that leads into the spec directory (row 65, on any volume)
answer ok and git runs in their directory; row 15 says a submodule mounted under the spec directory answers
missing. The second shape has answered ok since the first change of this branch; the first answered error only
because condition 5 compared names, and would have answered ok after the repair of that condition alone. The
documents of the change idd-548-fix7 and its delta gain this step.
…ec directory, whatever its path is named (#548)

A submodule whose recorded path is under the spec directory was already passed over by name. One whose
directory lies there under another name was searched: through a symbolic link that leads into the spec directory
(answer ok since the first change of this branch), or, after conditions 2 and 5 came to compare directories,
through a path that names the spec directory in another letter case. After conditions 1 and 2 hold, the
directories passed on the way up to the top directory are now compared with the spec directory by device and
inode; when it is among them the submodule is not searched and is not one that could not be searched, as when
its path is under the spec directory by name. Rows 64 and 65 pass (878 checks, none failing).
…what git says about it (#548)

The fixture asked `git rev-parse --is-inside-git-dir` whether its top directory lies inside the git directory.
For that layout git 2.55.0 answers true and git 2.54.0 (the one macOS ships) answers false, so with the older git
first on PATH the suite stopped at this fixture. The row states where the top directory lies; the fixture now
checks that through the file system. With either git first on PATH the suite has 878 passing checks and none
failing. No check of the script's behaviour changes.
Two whole runs before archiving (75 suites and no failure at e01674e and at f6fe368), the check counts in four
settings and with two versions of git, eight mutations, the layouts that had only been reasoned about, the
inputs that lie in the spec directory under three scripts, the search for answers that went from error to ok,
the call counts, and the correction of the fixture of row 58.
…skips an empty object or ends the search on it (#548)

The fixture of row 50 stopped the whole suite when the search over an empty
object file ended with an exit code above 1. git 2.54.0 and 2.55.0 skip the
file and end with 0 or 1; git 2.56.0 ends the search itself with 128, and
the CI of the pull request (git 2.56.1) stopped at that line, so no check
after row 49 had run there.

The fixture now records the exit code, prints one line when it is above 1,
and runs the checks of the answer in both cases: the answer is the same
(error, its recorded commit could not be read). The other premise of the
row, that git lists the empty object as present, still stops the suite when
it fails. spec-change.py is unchanged.

Measured after the change, spec-change suite: 878 pass, 0 fail with git
2.54.0, 2.55.0 and 2.56.0; only 2.56.0 prints the line. Whole runner: 75
suites, 0 failed.

With the documents of the change idd-548-fix8 (proposal, design, tasks and
the delta for the first column of row 50 of the Example).
The delta is applied to the live spec: the first column of row 50 of the
Example "markers in submodules" no longer says that the search over an
empty object always ends with exit code 0 or 1. Nothing else in the live
spec changes (one line removed, one added).
@kiki830621
kiki830621 marked this pull request as ready for review November 16, 2026 08:23
@kiki830621
kiki830621 merged commit 28e0de2 into main Nov 16, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant