Skip to content

fix(deps): exclude nltk 3.10.1 from the extras that pull rouge-score - #6576

Open
wuliang229 wants to merge 2 commits into
mainfrom
fix/exclude-broken-nltk-import-hook
Open

fix(deps): exclude nltk 3.10.1 from the extras that pull rouge-score#6576
wuliang229 wants to merge 2 commits into
mainfrom
fix/exclude-broken-nltk-import-hook

Conversation

@wuliang229

Copy link
Copy Markdown
Collaborator

Summary

Excludes nltk 3.10.1 from the two extras that pull rouge-score (eval and test), and adds a guard test to tests/unittests/test_release_dependencies.py.

This fixes all 28 unit-test failures currently on main (and the equivalent 13 on v1).

Root cause

nltk 3.10.1 (released 2026-08-01) added nltk/inisec.py, an import hook. It reaches ADK transitively via rouge-score. The hook does two independently harmful things:

1. find_spec refuses imports whose origin resolves under the CWD. For the standard in-project virtualenv layout — uv's default, in-project Poetry, plain python -m venv .venvsite-packages is under the CWD, so NLTK blocks its own dependency:

ImportError: Blocked import of regex from current working directory for security reasons.

This produces the evaluation/* collection errors, test_list_metrics_info, and the cli_eval failures.

2. _install() calls os.environ.setdefault("PYTHONSAFEPATH", "1"), mutating the importing process's environment. Every subprocess later spawned with os.environ.copy() inherits -P semantics it never opted into. This produces the subprocess import-isolation failures (test_import_loading, test_litellm_import, test_auth_config, test_adk_web_server_import_isolation, test_managed_agent), which fail with a bare ModuleNotFoundError: No module named 'google.adk' in the child.

Three properties made (2) hard to attribute: it fires even when the nltk import itself fails with (1); nothing in the child's traceback mentions nltk; and it only manifests on Python 3.11+, because PYTHONSAFEPATH is a no-op on 3.10.

That version split is the tell — on main, Python 3.10 fails 15 tests (cause 1 only) while 3.11–3.14 fail 28 (causes 1 and 2).

Why a pin rather than a CI-only workaround

rouge-score depends on nltk with no upper bound, so the exclusion must be declared by each extra that pulls it. A [tool.uv] constraint would fix CI but would not protect users — anyone installing google-adk[eval] into an in-project venv currently cannot import the eval stack, and picks up the PYTHONSAFEPATH mutation in their own application. Base pip install google-adk is unaffected; rouge-score is only in eval and test.

!= rather than an upper bound so a fixed release is picked up automatically.

Upstream

Tracked in nltk/nltk#3730. nltk/nltk#3731 (harden the hook) was closed in favour of nltk/nltk#3732 (remove inisec.py entirely), which is the maintainers' stated preference and clears both problems. This pin can be dropped once that ships.

Worth noting: #3731 would have kept _install(), so it would have fixed cause 1 and left cause 2 in place. The os.environ side effect is not yet documented upstream.

Test plan

  • pytest tests/unittests/test_release_dependencies.py — 7 passed; the two new cases fail if the pyproject.toml change is reverted (verified).
  • pyproject-fmt 2.24.0 reports no change.
  • uv pip compile on rouge-score + the new constraint resolves to nltk==3.10.0.

nltk 3.10.1 added nltk/inisec.py, an import hook that breaks CI and any
in-project virtualenv. It reaches ADK transitively through rouge-score,
which the eval and test extras pull in.

The hook does two independently harmful things:

1. find_spec refuses any import whose origin resolves under the CWD. For
   the standard in-project venv layout (uv, in-project Poetry, plain
   `python -m venv .venv`) site-packages *is* under the CWD, so nltk
   blocks its own dependency and `import regex` raises.
2. _install() calls os.environ.setdefault("PYTHONSAFEPATH", "1"), which
   mutates the importing process's environment. Every subprocess later
   spawned with os.environ.copy() inherits -P semantics it never opted
   into. This fires even when the nltk import itself fails, is invisible
   in the child's traceback, and only manifests on Python 3.11+ because
   PYTHONSAFEPATH is a no-op on 3.10.

Together these account for all 28 unit-test failures currently on main:
the evaluation/CLI collection errors from (1), and the subprocess
import-isolation failures from (2), which is why the latter reproduce on
3.11-3.14 but not on 3.10.

Upstream tracked in nltk/nltk#3730. The maintainers' preferred fix,
nltk/nltk#3732, removes inisec.py entirely and therefore clears both
problems; `!=` rather than an upper bound so the fixed release is picked
up automatically once it ships.
@adk-bot adk-bot added the eval [Component] This issue is related to evaluation label Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eval [Component] This issue is related to evaluation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants