Extract transport-agnostic proxy core as varlock/proxy-core - #964
Extract transport-agnostic proxy core as varlock/proxy-core#964theoephraim wants to merge 2 commits into
Conversation
The credential proxy's policy evaluation, substitution guards, placeholder substitution, response scrubbing, auth-token parsing, and the shared request pipeline (egress gate -> policy -> cleartext guard -> uninjected-placeholder guard -> substitution guards -> approval gate -> substitute) now live in a private workspace package with no node transport dependencies, bundled into varlock via tsup noExternal. runtime-proxy.ts becomes the node adapter: servers, MITM, CA handling, upstream identity verification, and streaming stay put, built on the shared core. This is the first slice of hosting the proxy on fetch-shaped gateways (Cloudflare Workers, Vercel), which will reuse the same core.
|
The changes in this PR will be included in the next version bump.
|
📦 Bundle size
dist/ only; native binaries are versioned separately and not counted here. |
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
The published TypeScript declarations retain an unavailable private-package import, so this needs one packaging fix before merge.
Reviewed changes in 160160a8: the complete extraction of the proxy's transport-neutral behavior and its Node adapter integration.
- Core package: Moves policy, proxy types, substitution guards, authentication, response-header handling, and activity types into private workspace package
@varlock/proxy-core. - Request pipeline: Introduces pre-body and body-dependent phases while retaining the egress, policy, cleartext, approval, substitution, audit, and tunnel-teardown sequence.
- Streaming response handling: Encapsulates cross-chunk secret scrubbing in
StreamingScrubberwhile the Node adapter retains byte decoding and response transport behavior. - Build integration: Bundles the private package into
varlockJavaScript and redirects existing source and test imports to its subpath exports.
azure/gpt-5.6-sol ｜ 𝕏
commit: |
Per review: the private-package approach leaked unresolvable @varlock/proxy-core imports into the published d.ts files (tsup's dts resolve cannot inline source-only workspace packages; the same pre-existing issue exists for @env-spec/utils/type-utils). Instead of publishing another package, the core now lives at src/proxy/core inside varlock and ships as a public subpath export, varlock/proxy-core, so gateway adapter packages can import it with working types. The subpath is built by a second sequential tsup invocation with platform 'neutral', which doubles as build-time enforcement that no node builtin sneaks into the core. (A second entry in the main config array does not work: tsup builds array configs in parallel and the main dts pass deletes the other config's d.ts output.)
theoephraim
changed the title
There was a problem hiding this comment.
ℹ️ No critical issues; one stale package reference remains inline.
Reviewed changes since the prior Pullfrog review, focusing on the replacement of the private workspace package with a published Varlock subpath.
- Published core: Moved the extracted source under
packages/varlock/src/proxy/coreand exposed it asvarlock/proxy-core. - Neutral build: Added a sequential neutral-platform tsup pass that emits self-contained JavaScript and declarations for the new subpath.
- Declaration fix: Removed all private
@varlock/proxy-coredependencies and restored resolvable generated declarations for Varlock consumers. - Release metadata: Updated the package bump and changeset to describe the new public API.
azure/gpt-5.6-sol ｜ 𝕏
| // Shared request pipeline for both transports (MITM tunnel + absolute-form http). | ||
| // The decision order — egress gate → per-call policy (block) → cleartext guard → | ||
| // uninjected-placeholder guard → substitution guards → approval gate → | ||
| // scrub+inject — lives in @varlock/proxy-core's two-phase pipeline; this |
There was a problem hiding this comment.
@varlock/proxy-core no longer exists after this commit, so this comment and the matching comment in audit.ts:9 point maintainers to the removed package. Please rename both references to varlock/proxy-core, or describe the shared core without a package name.
1 participant
What
Extracts the credential proxy's transport-independent logic out of
runtime-proxy.ts(1580 lines, interleaved with node http/tls) intosrc/proxy/core/, shipped as a new public subpath export:varlock/proxy-core.What moved (verbatim where possible):
types.tsandpolicy.ts(already pure) pluspolicy.test.tssubstitution.ts: substitution guards, placeholder<->real replacement, uninjected-placeholder detectionscrub.ts: leak detection, scrubbed-key detection, and a newStreamingScrubberthat owns the chunk-carry/hold-back logic for streamed responses (node wraps it in aTransform; a fetch-based gateway can wrap it in a WebTransformStream)headers.ts: header transforms and response-redaction predicates, re-typed against a neutralHeadersRecordauth.ts:Proxy-Authorizationparsing and the data-plane gate, with portable (Buffer/node:crypto-free) base64 and constant-time compareactivity.ts: theProxyActivity/ProxyAuditDecisiontypes (audit.tsre-exports them)pipeline.ts: the shared request decision order as a two-phase pure pipeline (pre-body: egress gate -> policy -> item scoping -> cleartext guard; with-body: uninjected-placeholder guard -> substitution guards -> approval gate -> substitution), returning blocked/forward outcomes with the exact same messages, audit activities, and teardown behavior as beforeruntime-proxy.tsis now the node adapter: listeners, CONNECT/MITM, CA handling, verified-upstream-identity forwarding, and response streaming, driving the shared pipeline. The two-phase split preserves the existing behavior of not buffering request bodies for requests denied pre-body.Packaging
The first iteration made this a private workspace package bundled via
noExternal, but that leaked unresolvable@varlock/proxy-coreimports into the published declarations (review finding). tsup's dts resolve cannot inline source-only workspace packages (verified; the published package has the same long-standing issue with@env-spec/utils/type-utils), so instead of publishing another package the core lives insidevarlockand ships as thevarlock/proxy-coresubpath. Future gateway adapter packages (@varlock/cloudflare-gatewayetc.) import it from there with working types.The subpath is built by a second sequential tsup invocation (
tsup.proxy-core.config.ts) withplatform: 'neutral', which doubles as build-time enforcement that no node builtin sneaks into the core. It is a separate invocation because tsup builds array configs in parallel and the main dts pass deletes the other config's d.ts output.Why
First slice of running the proxy on fetch-shaped gateway hosts (Cloudflare Workers first, Vercel later): those adapters need the policy/substitution/scrub pipeline without node transport types. Also makes the security-critical decision order independently testable.
No behavior change intended; the full proxy suite (including the TLS/MITM integration tests and the Bun Invariant #1 check) passes unchanged, and
distnow contains no imports of private workspace packages from the proxy core.