Skip to content

[python] Avoid concurrent import failures during split deserialization - #9250

Open
XiaoHongbo-Hope wants to merge 3 commits into
apache:masterfrom
XiaoHongbo-Hope:codex/fix-ray-concurrent-import
Open

[python] Avoid concurrent import failures during split deserialization#9250
XiaoHongbo-Hope wants to merge 3 commits into
apache:masterfrom
XiaoHongbo-Hope:codex/fix-ray-concurrent-import

Conversation

@XiaoHongbo-Hope

@XiaoHongbo-Hope XiaoHongbo-Hope commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Problem

A production Ray job using the REST/DLF catalog failed while workers first deserialized query_auth_split and DataFileMeta objects concurrently. The reported error was:

ImportError: cannot import name 'SimpleStats' from partially initialized module
'pypaimon.manifest.schema.simple_stats' (most likely due to a circular import)

All 18 workers in the reported job failed. Pre-importing PyPaimon through worker_process_setup_hook avoided the race. A FULL global-index fallback can amplify the problem by scheduling many splits, but the import race is not specific to FULL mode.

Root cause

Importing any low-level pypaimon submodule executes the package root first. Its eager public exports load the catalog, filesystem, schema, and read dependency graph while the requested split module is still being initialized. Concurrent deserialization can therefore observe partially initialized modules or hit a Python module-lock deadlock.

Fix

Resolve the existing root-level public exports lazily on Python 3.7 and newer, while retaining the eager compatibility path for Python 3.6. Public imports such as from pypaimon import CatalogFactory remain unchanged.

Add a fresh-process regression test that concurrently imports and deserializes a real QueryAuthSplit containing DataFileMeta instances across 24 threads.

Validation

  • Concurrent import regression repeated 30 times
  • Query-auth, catalog environment, and concurrent import tests: 61 passed
  • flake8 and git diff --check passed

@XiaoHongbo-Hope
XiaoHongbo-Hope marked this pull request as ready for review August 16, 2026 11:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant