Skip to content

Fix correctness, robustness and memory issues in the MERSCOPE reader - #428

Open
timtreis wants to merge 1 commit into
mainfrom
audit/merscope-reader
Open

timtreis wants to merge 1 commit into
mainfrom
audit/merscope-reader

Conversation

@timtreis

Copy link
Copy Markdown
Member

Audited merscope() against real Vizgen data (Figshare VPT output: 137k cells / 70.5M transcripts / 960-gene panel; the Vizgen showcase mosaic images; GSE241870 for the counted-cell-without-polygon case). Each fix has a regression test (fails on main, passes here).

  • Transcript genes silently dropped across dask partitions. gene was passed to PointsModel.parse as object dtype, leaving dask categories "unknown"; spatialdata then infers them from the first partition only, turning genes absent from partition 0 into NaN. Now a known categorical before parse; sort=True drops a redundant eager index scan.
  • Table stored dense int64. Now csr_matrix like the other scverse readers (~1,057 MB → ~197 MB on the Figshare panel); also removes the deprecated AnnData(dtype=...) arg.
  • Crash without images/. The micron→pixel transform was read unconditionally; table/transcripts-only reads now fall back to an identity transform with a warning.
  • Non-deterministic image channel order (list(set(...)), PYTHONHASHSEED-dependent) → sorted().
  • VPT-dict mode crashed with string paths ('str' object has no attribute 'exists') → values wrapped in Path.
  • Plain Polygon boundaries crashed MultiPolygon(x.geoms) → geometries normalised to MultiPolygon.

No public API changes. Adds tests/test_merscope.py (self-contained synthetic fixtures). ruff / ruff-format / mypy clean.

@timtreis
timtreis force-pushed the audit/merscope-reader branch from e4a8ddf to 7e5c789 Compare October 15, 2026 22:11
@codecov-commenter

codecov-commenter commented Oct 15, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.72727% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.03%. Comparing base (261578f) to head (6a2e15f).

Files with missing lines Patch % Lines
src/spatialdata_io/readers/merscope.py 72.72% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #428      +/-   ##
==========================================
+ Coverage   65.79%   68.03%   +2.23%     
==========================================
  Files          26       26              
  Lines        3263     3272       +9     
==========================================
+ Hits         2147     2226      +79     
+ Misses       1116     1046      -70     
Files with missing lines Coverage Δ
src/spatialdata_io/readers/merscope.py 80.95% <72.72%> (+51.96%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- transcripts: make `gene` a known categorical before PointsModel.parse so
  categories are not inferred from the first dask partition (which can drop
  genes absent from partition 0); pass sort=True and drop the redundant
  post-parse re-cast
- table: store X as csr_matrix like the other scverse readers (MERSCOPE
  matrices are ~10% dense; ~1GB -> ~200MB on a 960-gene panel)
- read the micron-to-pixel transform only when present, else fall back to an
  identity transform with a warning, so table/transcripts-only reads work
- handle a missing images/ directory in merscope() with a warning
- sort channel names for a deterministic image channel order
- accept string paths in vpt_outputs dict mode (wrap in Path)
- normalise boundary geometries to MultiPolygon so plain Polygons don't crash

Add merscope reader-fix tests to tests/test_merscope.py.
@timtreis
timtreis force-pushed the audit/merscope-reader branch from 7e5c789 to 6a2e15f Compare October 16, 2026 06:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants