Problem or motivation
Vortex's Python API exposes concrete, standalone store objects for S3/Azure/GCS/HTTP/Local/Memory (e.g. vortex.store.S3Store), which can be constructed once and passed directly to vortex.io.read_url / vortex.io.write via the store= argument.
Tencent Cloud COS is currently only reachable through URL-based resolution (cos://), which depends on the opendal feature and requires configuration via environment variables (e.g. TENCENTCLOUD_SECRET_ID / COS_ENDPOINT). There is no way to build a concrete COS store object and hand it to read_url/write the way users can with the other stores — an inconsistency that also makes credential handling less ergonomic and less type-checked.
Additionally, the OpenDAL dependency is pinned at 0.56.0; bumping to 0.57.0 unblocks newer service behavior (e.g. COS versioning is now enabled by default).
Proposed solution
- Add a new pyclass, `vortex.store.CosStore`, feature-gated on `opendal`. It builds an OpenDAL `services::Cos` operator bridged to an `object_store::ObjectStore` via `object_store_opendal::OpendalStore`, wrapped as a `PyObjectStore` so it can be passed directly to `read_url`/`write` via `store=`, exactly like the built-in S3/Azure/GCS stores.
- Extend `read_url`/`write` with a small extractor (`AnyVortexStore`) that recognizes the new OpenDAL-backed class and normalizes it to `Arc<dyn ObjectStore>`.
- `From<Arc<dyn ObjectStore>> for PyObjectStore` is required but blocked by the orphan rule, so vendor the `pyo3-object_store` source under `[patch.crates-io]` and add the needed `from_store` constructor there.
- Bump `opendal` and `object_store_opendal` to `0.57.0` (in `vortex-object-store-opendal`).
Example:
```python
from vortex.store import CosStore
from vortex.io import read_url
store = CosStore(
bucket="my-bucket",
endpoint="https://cos.ap-guangzhou.myqcloud.com",
secret_id="AKID...",
secret_key="...",
)
array = read_url("cos://my-bucket/data.vortex", store=store)
Notes:
CosStore is only available when Vortex is built with the opendal feature; the existing cos:// environment-variable resolution is unchanged.
- This is intentionally scoped to COS. Other OpenDAL-backed providers (e.g. OSS) can be added later by following the same pattern.
Additional context
No response
Problem or motivation
Vortex's Python API exposes concrete, standalone store objects for S3/Azure/GCS/HTTP/Local/Memory (e.g.
vortex.store.S3Store), which can be constructed once and passed directly tovortex.io.read_url/vortex.io.writevia thestore=argument.Tencent Cloud COS is currently only reachable through URL-based resolution (
cos://), which depends on theopendalfeature and requires configuration via environment variables (e.g.TENCENTCLOUD_SECRET_ID/COS_ENDPOINT). There is no way to build a concrete COS store object and hand it toread_url/writethe way users can with the other stores — an inconsistency that also makes credential handling less ergonomic and less type-checked.Additionally, the OpenDAL dependency is pinned at 0.56.0; bumping to 0.57.0 unblocks newer service behavior (e.g. COS versioning is now enabled by default).
Proposed solution
Notes:
CosStoreis only available when Vortex is built with theopendalfeature; the existingcos://environment-variable resolution is unchanged.Additional context
No response