ENH: add bids-simbids - #165
Draft
asmacdo wants to merge 3 commits into
Draft
Conversation
The 2022 translation of this script from bash rendered its
`mkdir -p "$TMPDIR"` (with TMPDIR=$topdir/tmp) as `topdir.mkdir()`.
So the mkdir no-ops on the repo root, while the directory actually
exported as TMPDIR/SINGULARITY_TMPDIR is never created.
Every `singularity pull` then fails with
FATAL: ... failed to create build parent dir:
stat <repo>/tmp: no such file or directory
Since that exits 255 -- the code the retry loop treats as a transient
network error -- it retries 4x with backoff first, so it presents as a
flaky download rather than a missing directory.
Unnoticed so far because tmp/ is gitignored and long-lived checkouts
still carry the directory created by the pre-2022 bash version. It
reproduces on any fresh clone or worktree.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
simbids generates BIDS dataset and derivative skeletons for CI testing, and is used by babs as a fast stand-in BIDS App. Adding it here lets those projects fetch it like any other container instead of building it from Docker Hub themselves. It implements the BIDS App CLI, so it belongs in the bids family, but it is not in the bids-website apps.yml that we scrape above -- so it needs an explicit entry rather than auto-discovery. family="bids" is passed explicitly since it would otherwise default to the "pennlinc" docker namespace and register as pennlinc-simbids. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The process lived entirely in scripts/create_singularities and in whoever last ran it. Documents teaching the script about an image (groups vs explicit calls, the family= default that is usually wrong, how a version is chosen), running it, the two commits that result, and how to verify. Also states who can do what: test_repo_state.bats requires annexed content to be available from the web or datasets.datalad.org, so an image built locally cannot arrive by PR. Contributors PR the script change; a maintainer runs the builder. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
asmacdo
marked this pull request as ready for review
July 31, 2026 15:15
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
Member
Author
|
@yarikoptic assuming you merge this, could you run the builder afterwards? |
Member
Author
|
We dont want a special case for this, and @yarikoptic wants the bugfix and docs fixed separately. |
yarikoptic
marked this pull request as draft
July 31, 2026 20:48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
bids-simbidsso BABS and mechababs (or other tools!) can fetch it like any other container instead of building it from Docker Hub.simbids simulates BIDS datasets and derivatives for CI testing.
BABS and mechababs use it as a fast dummy BIDS App in their test suites, so today each of them builds the image itself.
The change
bids-appsare discovered from the bids-website catalog, and simbids is not in it, since it simulates BIDS data rather than analyzing it.So it gets an explicit call to build it, exactly like
bids/validator.We could instead add simbids to that catalog.
The proposal is to do it this way now; if we decide it should be listed, I will open a PR over there and drop this entry.
Docs
DOC: add CONTRIBUTING.md— how to add a container: what to run, the two commits it produces, why a contributor cannot land an image, and how to verify. README gains a pointer to it.Bugfix
BF: create the tmp/ dir we point TMPDIR at—topdir.mkdir()should be(topdir / "tmp").mkdir(), so the directory used asTMPDIRwas never created.Hit while testing this PR: it reproduces on any fresh clone or worktree, and long-lived checkouts do not see it because they still carry the directory the pre-2022 bash version made.
Happy to split it into its own PR.
Manual verification
Ran
scripts/create_singularities pennlinc/simbidson this branch. It produced exactly the expected pair:which registered
datalad.containers.bids-simbids.image = images/bids/bids-simbids--0.0.3.sif.The resulting dataset then drove a complete BABS campaign end to end: init, submit, a job that ran, merge, and a real derivative.