Skip to content

Relational query reorders table obs by region in 0.8 #1162

Description

@timtreis

Since #1131, filtering a table that annotates multiple regions reorders its obs rows by region instead of preserving the original order. This shifts table rows relative to element geometries and broke an ordering assertion in a spatialdata-plot test.

Cause: _filter_table_by_elements was changed from a boolean mask (order-preserving) to join_spatialelement_table(how="left"), which does obs.groupby(region) and thus regroups the rows.

MRE

import numpy as np
import pandas as pd
from anndata import AnnData
from geopandas import GeoDataFrame
from shapely.geometry import Point
import spatialdata as sd
from spatialdata.models import ShapesModel, TableModel


def circles(n):
    gdf = GeoDataFrame({"geometry": [Point(i, i) for i in range(n)], "radius": [1.0] * n})
    return ShapesModel.parse(gdf)


# table annotates two regions, obs rows INTERLEAVED: a,b,a,b,a,b
obs = pd.DataFrame(
    {
        "region": pd.Categorical(["a", "b", "a", "b", "a", "b"]),
        "instance_id": [0, 0, 1, 1, 2, 2],
        "label": ["a0", "b0", "a1", "b1", "a2", "b2"],
    }
)
table = TableModel.parse(
    AnnData(X=np.zeros((6, 1)), obs=obs, var=pd.DataFrame(index=["g0"])),
    region=["a", "b"],
    region_key="region",
    instance_key="instance_id",
)
sdata = sd.SpatialData(shapes={"a": circles(3), "b": circles(3)}, tables={"table": table})

before = list(sdata["table"].obs["label"])
after = list(sdata.filter_by_coordinate_system("global")["table"].obs["label"])
assert after == before, f"obs reordered {before} -> {after}"

Result

┌─────────┬───────────────────────────────────────┐
│ version │ obs after filter_by_coordinate_system │
├─────────┼───────────────────────────────────────┤
│ 0.7.2   │ a0, b0, a1, b1, a2, b2 (preserved) ✅ │
├─────────┼───────────────────────────────────────┤
│ 0.8     │ a0, a1, a2, b0, b1, b2 (regrouped) ❌ │
└─────────┴───────────────────────────────────────┘

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions