Repository navigation
Ensure the store key's constraint again when another manager made it meanwhile, and write again a stamp it refuses (#298) - #308
Merged
Conversation
…meanwhile, and write again a stamp it refuses (#298) Port of liberation-data/drivine4j#473 (drivine4j PR #477), following drivine PR #489. Managers resolving the store identity at once on a store without the DrivineStore(key) constraint race to make it. On FalkorDB and Postgres the losers' statements were refused for what the winner made meanwhile ("Constraint already exists.", a 23505 on pg_class_relname_nsp_index), so they went on without the constraint, warning that identity was best-effort, and a stamp the winner's constraint then refused was raised out of resolve(). - _ensure_unique_key ensures the constraint again, at most 3 times, when its making was refused as made meanwhile. A Postgres failure counts only as a 23505 on a pg_ catalog constraint, judged by SQLSTATE and constraint name, since psycopg's message carries "already exists" for any violation. Deadlocks are left to the schema executor's contention retry, which already runs Neo4j's DeadlockDetected again. - _assign_if_absent merges again on a uniqueness violation even when the constraint wasn't confirmed, on a manager with schema management only. A write conflict is still merged again only under a confirmed constraint. Tests: unit tests with a fake Neo4j store through the real ConstraintManager and SchemaExecutor, async and sync; and 16 managers racing per round on every store (5 rounds; 3 on Postgres, 2 on Oracle), with tasks and with threads. Fixes #298.
…ead Oracle's code from the driver's error
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #298. Port of liberation-data/drivine4j#473 (drivine4j PR liberation-data/drivine4j#477), following liberation-data/drivine#489.
What happened
When several managers resolve the store identity at once on a store with no
DrivineStore(key)constraint, they all try to make it. A loser whose statement was refused went on without the constraint (_ensure_unique_keyreturnedFalse). Then_assign_if_absentraised any failure of the MERGE when unguarded. So a stamp refused by the winner's constraint, now in force, failed the boot.On main, the new concurrency test (16 managers per round) fails on two stores, with tasks and with threads:
Could not ensure the DrivineStore key constraint: Constraint already exists. Identity is best-effort.duplicate key value violates unique constraint "pg_class_relname_nsp_index"best-effort warnings in one run. In the other, a loser's stamp was refused by the winner's constraint, andDrivineError: duplicate key value violates unique constraint "drivine_uq_drivinestorekey_…"was raised out ofresolve().Neo4j, Memgraph and Oracle pass on main. The schema executor already runs a schema statement again on a transient Bolt error, through
when_not_contended(drivine/schema/schema_executor.py), and that includes Neo4j'sDeadlockDetected. On Oracle the race gives only ORA-04020, whichddl_contendedalready retries.What this changes
_ensure_unique_key: the constraint is ensured again, at most 3 times, when its making was refused for something another manager made meanwhile. That covers FalkorDB'sConstraint already existsandalready indexed, Postgres's 23505 on apg_catalog constraint, and Oracle's ORA-00955. The secondensurefinds the winner's constraint, so the stamp is written under it. Deadlocks are not retried here, because the schema executor has already retried them.diag.constraint_namealone. psycopg's message carries the DETAIL,Key (...)=(...) already exists., of any violation, so matching the text would count a violation in a user's table as made meanwhile._assign_if_absent: a uniqueness violation (unique constraint|already exists, not any "constraint") is merged again even when the constraint wasn't confirmed, because the violation proves a constraint is in force. This applies only to a manager with schema management. A write conflict is still merged again only under a confirmed constraint. Merges stay bounded at 5.Neo4jConnection.releasealready logs a closing failure at debug, so it needs no change here.Tests
Unit (
tests/unit/test_store_identity_race.py), async and sync: a fake Neo4j store, run through the realConstraintManagerandSchemaExecutor. Cases:Constraint already exists.,already indexed, a Postgres 23505 onpg_class_relname_nsp_index): ensured again, no best-effort warning;orders_pkey, with psycopg's DETAIL): not ensured again;SyntaxError, or a not-null violation, on the MERGE: raised at once;On main, 10 of the 22 fail: the violation, made-meanwhile and 3-attempt cases.
Integration: 16 managers resolve at once on a store with the constraint dropped and the stamps wiped. That is 5 rounds on Neo4j, Memgraph and FalkorDB, 3 on Postgres and 2 on Oracle. Each round must agree on one id, leave one stamp, and log no best-effort warning. The test runs with tasks (
tests/integration/test_schema.py,async_only) and with threads (tests/sync/test_store_identity_race.py). It fails on main on FalkorDB and Postgres, and passes on every store with this change.The full gate passed with
ORACLE_TESTS=true: ruff check, ruff format --check, pyright,unasync.py --check, and pytest (5674 passed, 298 skipped).