stile v0.1 import fails: 'treecorr.corr2' removed in newer treecorr

[cailmdaley]

Symptom

4 shapepipe modules fail to import:

• shapepipe.modules.mccd_package.mccd_plot_utilities
• shapepipe.modules.mccd_plots_runner
• shapepipe.modules.random_cat_package.random_cat
• shapepipe.modules.random_cat_runner

Traceback ends:

File ".../stile/treecorr_utils.py", line 8
ModuleNotFoundError: No module named 'treecorr.corr2'

Root cause

The Dockerfile pins treecorr==5.1.1 and installs stile @ git+https://github.com/tobias-liaudat/Stile@v0.1. treecorr 5.x flattened its namespace and removed the treecorr.corr2 submodule; stile v0.1 still imports it.

Options

1. Bump stile to a newer version (if one exists that supports treecorr 5.x). Check Stile upstream.
2. Pin treecorr<5 in the Dockerfile. Regresses other modules that rely on treecorr 5 features (if any).
3. Fork stile and patch the import.

Caught by

tests/unit/test_imports.py for the 4 modules above (xfailed in #708).

[cailmdaley]

Investigation: no newer Stile version fixes this

I looked for a Stile release that's compatible with treecorr>=5, since the obvious first move is "just bump the version." There isn't one.

• tobias-liaudat/Stile — only the v0.1 tag (the one we pin); master last touched 2020-10-30 and still imports from treecorr.corr2 import corr2_valid_params.
• msimet/Stile (the original) — last commit 2018; unmaintained.
• All 5 other forks I could find (Jeanne-jw, dylancromer, WentaoLuo, EiffL, enourbakhsh) — same broken import.

So bumping @v0.1 to any other ref on any existing fork just moves the failure around.

What actually broke

treecorr 5.x flattened the namespace. The symbol itself still exists; it just moved:

# treecorr 4.x
from treecorr.corr2 import corr2_valid_params   # works

# treecorr 5.x
from treecorr import corr2_valid_params         # works
from treecorr.corr2 import corr2_valid_params   # ModuleNotFoundError

I verified from treecorr import corr2_valid_params works against the 5.1.2 we ship, and import stile succeeds once that one line in stile/treecorr_utils.py is patched.

Options (none are clean)

#	Option	Pros	Cons
1	Pin treecorr<5 in the Dockerfile	One-line, no patching	Regresses whichever modules benefit from treecorr 5 features; we'd owe ourselves a diff to see if anything current actually cares
2	Post-install sed in the Dockerfile to patch the installed Stile file	Self-contained; one-line fix lives in the image build	sed on a site-packages file is ugly, and silently breaks if Stile's layout ever changes
3	Fork Stile under CosmoStat, apply the one-line fix, repoint the pin	Proper artifact; reviewable; other users can pin to it	One more abandoned repo to own; small but real infra burden
4	Leave stile's 4 dependents broken (as xfail in #708)	Zero work now	mccd_plots_runner, random_cat_runner and their packages stay broken in-container

Recommendation

I lean toward (2) as a short-term fix (matches the urgency of the other bugs we're shaking out) with (3) as the right long-term answer if/when anyone else hits this. (1) is only clean if no one is using treecorr-5-specific features; I haven't surveyed that.

But I genuinely don't have strong feelings — happy to do whichever you prefer. If you want (1) I can check quickly whether anything on develop actually needs treecorr 5.x.

[martinkilbinger]

My suggestion is to remove leave it broken at the moment or apply a minial fix, and later remove the dependency on stile, in particular the tlaudiat fork. Should be easy, it's only used for the rho stats, and we have alternative ways to compute these, e.g. with https://github.com/CosmoStat/shear_psf_leakage.
I modified this issue for this: #657