Add native Python integration

[philmillman]

Adds a varlock Python package (packages/varlock-python) that resolves the env schema from inside a running process, so no wrapped launch is needed.

import varlock
env = varlock.load()

This is a follow-up to the Python codegen support, and the next step toward parity with the deeper JS/TS integrations. It came out of a user request about Jupyter: VS Code and JupyterLab spawn the kernel themselves, so there is no command line for varlock run to wrap, which leaves shelling out to varlock load --format json and parsing it by hand as the only option today.

How it works

Same architecture as the deep JS integration, not a reimplementation. It resolves nothing itself: it shells out to varlock load --format json-full --compact, parses the graph, injects into os.environ, and exposes the values. Every plugin, cache, and validation behavior comes from the installed CLI.

Module	Mirrors
_cli.py	execSyncVarlock
_binary.py	findVarlockBin, plus the standalone installer's directories
_runtime.py	initVarlockEnv state and process.env injection
_env.py	the ENV proxy
_redaction.py	resetRedactionMap / redactSensitiveConfig / scanForLeaks
_patch.py	patchGlobalConsole
auto_load.py	varlock/auto-load

When the process was started by varlock run, the values are already resolved, so load() adopts the blob instead of resolving again. The same code works both ways.

ENV is a read-only mapping that also allows attribute access. Unknown keys raise rather than returning None. Zero runtime dependencies, Python 3.9+.

Redaction

load() masks values marked @sensitive in output, following the schema's @redactLogs setting. Output escapes a Python process in three separate places, so all three are covered:

• logging, via the record factory, which every logger and handler goes through no matter when it was created. A filter would only cover the one logger or handler it was attached to.
• print(), via builtins.print, which resolves its stream at call time and so survives a library or kernel replacing sys.stdout after redaction was installed. The stream objects are patched too, for direct write() calls.
• Notebook cell output, via IPython's display formatter. Cell results never go through stdout, and this is the main way a secret ends up saved in an .ipynb.

Plus the primitives: redact() walks strings, lists, and dicts; reveal() marks a value as deliberately shown; scan_for_leaks() raises VarlockLeakError naming the key and respects @sensitive={preventLeaks=false}.

Matching the JS runtime, only values that resolved to strings are registered (masking every occurrence of a sensitive number would wreck unrelated output), and elements of composite values register individually so leaking one item of a list is caught.

Typing

No new codegen mode. The Env TypedDict that @generatePythonEnv already emits types the native loader as-is:

env = cast(Env, varlock.load())
env["NOT_A_KEY"]   # error: TypedDict "Env" has no key "NOT_A_KEY"

Verified with mypy, which catches both unknown keys and wrong-typed assignments. Separately, ENV.<TAB> and ENV["<TAB>"] complete schema keys at runtime in Jupyter and IPython (the subscript form needs _ipython_key_completions_, since IPython only infers keys for real dict instances). The cast is a no-op at runtime, so the object is still the live env. This is documented rather than automated: a generated module that imports varlock would give up the current one's selling point of having no dependencies, to save a single line.

Deliberate differences from the JS runtime

• Raises instead of exiting. load() raises VarlockLoadError with the CLI's formatted stderr attached. Killing a kernel is the wrong response to a schema typo. import varlock.auto_load opts into fail-fast for scripts and servers.
• Unset optional keys are absent, not present as None, matching the contract the generated module already documents. ENV["OPTIONAL"] reports "exists in your schema, but has no value in this environment", distinct from the unknown-key error.
• Env var serialization follows the CLI, not Python. True injects as "true", composites use the blob's envStr, and unset keys are not injected at all (what varlock run does, since Node drops undefined env values).
• A failed reload() keeps the previous values. The uninjected environment is handed to the CLI as a copy rather than applied in place first, so a bad schema edit mid-notebook does not wipe a working env.
• Merely importing the package never patches global state. Under varlock run, import sets up the redaction map so redact() works, but only an explicit load() installs the patches.

Binary discovery reads VARLOCK_BIN, then PATH, then the standalone install directories, then walks up for node_modules/.bin.

Testing

• 128 unit tests, run on 3.9 and 3.13 in a new python-package CI job.
• 3 smoke tests against the built CLI: resolving in-process, adopting the varlock run blob, and error reporting. The in-process one also asserts redaction of print, logging, reveal(), and scan_for_leaks.
• Manually exercised end to end against an installed CLI, including a live reload() picking up a schema edit, unload() restoring os.environ, and the notebook path driven through a real IPython shell (cell output, print, and the ENV repr all masked).

Docs

New Jupyter page covering setup, reloading, which schema gets loaded, redaction and its limits, auth prompts, and the varlock run -- jupyter lab alternative. The Python page now leads with the two paths and gains a native package section covering the API, typing, and redaction; the codegen content moves under a "Generated env module" heading. @redactLogs no longer says it is JavaScript-only.

Reviewer notes

• The docs say pip install varlock, but nothing is published to PyPI yet. Both varlock and varlock-cli are available as names. Either publish an initial version before this merges, or hold the docs pages back. This is the one thing blocking merge as-is.
• Windows .cmd shims no longer go through shell=True. list2cmdline only quotes arguments containing whitespace, but cmd.exe parses &/|/</>/^/parens out of unquoted text before argv splitting, so a metacharacter in a --path value or in the install path could start a second command (the Node CVE-2024-27980 class). Now built as an explicit cmd.exe /d /s /c line with every token quoted, passed with shell=False. The quoting is tested against the stdlib and round-tripped through an MSVCRT parser, but the execution path itself is untested for want of a Windows machine, so a Windows reviewer would help.
• Wrapping builtins.print is invasive. It is reversible via uninstall_redaction(), and it is the only way to keep print redaction working across a stream swap, but worth a second opinion.
• The package is managed with uv, not bun. The package.json is a private shim so bun workspaces and turbo can see the directory; it is never published to npm.
• Remaining follow-up: platform wheels behind a varlock[cli] extra, so pip install alone is enough in a fresh notebook environment.

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	varlock-website	1a8989a	Commit Preview URL Branch Preview URL	Jul 28 2026, 09:00 PM

[philmillman]

@pullfrog review this

[pullfrog]

Important

Redaction lifecycle failures can leave global patches active or break logging after uninstall. The package publication also needs to be sequenced before the new install docs ship.

Reviewed changes: This PR adds a native Python package that resolves Varlock schemas through the CLI, exposes a live typed environment mapping, and redacts sensitive output for scripts and notebooks.

• Add the varlock Python package: Implements binary discovery, CLI execution, serialized graph parsing, environment injection, reload, unload, and public errors.
• Add sensitive-output protection: Introduces redaction, leak scanning, deliberate reveal support, and global patches for logging, print, streams, and IPython display output.
• Add Python package coverage: Adds unit tests, a Python CI job, and smoke tests against both direct CLI resolution and varlock run adoption.
• Document Python and Jupyter usage: Adds native package setup, API, typing, redaction, reload, and notebook guidance.

⚠️ Install instructions precede package publication

The new user paths all begin with pip install varlock, but this package is not published and the PR adds no PyPI release workflow. If the docs deploy before a manual publication, every native Python and Jupyter setup attempt stops at installation.

Technical details

# Install instructions precede package publication

## Affected sites
- `packages/varlock-python/README.md:6` — presents the package as installable from PyPI.
- `packages/varlock-website/src/content/docs/integrations/jupyter.mdx:20` — requires the unpublished package during setup.
- `packages/varlock-website/src/content/docs/integrations/python.mdx:14` — requires the unpublished package during setup.

## Required outcome
- Ensure `pip install varlock` succeeds before these docs are deployed, or hold the install guidance until publication is complete.
- Establish how later Python package versions are published so package fixes can be released after merge.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using azure/gpt-5.6-sol ｜ 𝕏}

[pullfrog]

Pushed dc6630c1 to PR #952 with symmetric redaction disablement and ownership-safe teardown for logging, print, streams, and IPython formatters. All 95 Python tests and repository lint pass; all three review threads were answered and resolved.

Task list (5/5 completed)

• Check out PR 952 and load review feedback
• Verify and address every review comment
• Run targeted tests and review the resulting diff
• Commit and push the fixes to the PR branch
• Reply to and resolve addressed review threads

  ^{｜ View workflow run ｜ via Pullfrog ｜ Using azure/gpt-5.6-sol ｜ 𝕏}