Skip to content

Concurrent local-mode starts race while populating the SEA binary cache #351

Description

@abhinavkr26104

Description

SEA binary cache initialization is not concurrency-safe. _copy_to_cache() derives the temporary filename solely from the binary content hash, so every thread/process installing the same version writes and replaces the same temporary path.

After one caller moves the shared temp file into place, the others fail while writing or replacing it. On Windows this reliably surfaces as PermissionError; on POSIX callers can see FileNotFoundError or race with replacement.

Reproduction

On current main:

import tempfile
import threading
from pathlib import Path

import stagehand._custom.sea_binary as sea_binary

with tempfile.TemporaryDirectory() as directory:
    root = Path(directory)
    source = root / source
    source.write_bytes(bx * 5_000_000)

    original_cache_dir = sea_binary._cache_dir
    sea_binary._cache_dir = lambda: root / cache
    errors = []
    barrier = threading.Barrier(16)

    def resolve():
        try:
            barrier.wait()
            sea_binary._copy_to_cache(src=source, filename=stagehand, version=test)
        except Exception as exc:
            errors.append(repr(exc))

    threads = [threading.Thread(target=resolve) for _ in range(16)]
    for thread in threads:
        thread.start()
    for thread in threads:
        thread.join()

    sea_binary._cache_dir = original_cache_dir
    print(len(errors), errors[:3])

On Windows this produced 15 errors from 16 callers, primarily:

PermissionError(13, 'The process cannot access the file because it is being used by another process')

Code reference

src/stagehand/_custom/sea_binary.py:64-78

In particular, line 74 creates the same .{filename}.{sha256}.tmp path for all callers, and line 76 moves that shared path.

Expected behavior

Concurrent cold starts for the same Stagehand version should all resolve to the completed cached binary without error. A per-caller temporary file plus atomic publish, or an inter-process lock with an existence recheck, would avoid the collision.

Actual behavior

Only one caller succeeds; competing callers fail during the shared temp-file write/replace.

Why this matters

Parallel workers and multiple application processes commonly initialize SDK clients at the same time. A cold cache can make otherwise valid local-mode startup nondeterministically fail.

Prior-art check

I searched open/closed issues and PRs for SEA cache concurrency, binary cache races, and extraction locking and found no existing report.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions