Skip to content

Add Deno and macOS Keychain smoke coverage - #847

Open
bjesuiter wants to merge 32 commits into
dmno-dev:mainfrom
bjesuiter:deno-compatibility
Open

Add Deno and macOS Keychain smoke coverage#847
bjesuiter wants to merge 32 commits into
dmno-dev:mainfrom
bjesuiter:deno-compatibility

Conversation

@bjesuiter

@bjesuiter bjesuiter commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR improves Deno compatibility and makes the macOS Keychain access model explicit and non-destructive.

Deno

  • allow deno processes to connect to the Varlock IPC helper
  • generate shell completion scripts that invoke Varlock correctly through deno run -A
  • add an opt-in Deno compatibility path to the shared smoke-test runner

macOS Keychain

  • make all production secret reads go directly through Security.framework; there is no /usr/bin/security read fallback
  • change varlock keychain fix-access to read each referenced item through VarlockEnclave and guide the user to choose Always Allow
  • support fix-access for one item or every explicit keychain(...) reference in an env file
  • add varlock keychain cloneToOwned as an explicit, non-destructive way to create a Varlock-owned copy
  • add varlock keychain delete for generic-password items
  • update the Keychain documentation and daemon IPC types accordingly

Keychain access behavior

For a secret that already exists in Keychain and is owned or managed outside Varlock, the normal path is to read it in place. macOS may prompt for access; choosing Always Allow adds VarlockEnclave to that item's ACL so repeated reads no longer prompt. Choosing Allow Once only authorizes the current read.

cloneToOwned is optional. It reads the source through Security.framework, creates a separate Varlock-owned destination item, optionally writes its keychain(...) env reference, and leaves the original item unchanged. The PR does not perform automatic ownership transfer or delete/recreate existing external items.

Smoke coverage

The gated macOS suite covers fix-access, cloneToOwned, set, and import, including subsequent reads and daemon-state resets between tests. /usr/bin/security is used only by test setup and cleanup to create or remove fixtures, never as a production secret-read path.

  • Deno compatibility tests: VARLOCK_RUN_DENO_COMPAT=1
  • Keychain tests: VARLOCK_RUN_KEYCHAIN_SMOKE=1

Autoreview found regressions in the KeychainManager ownership rewrite for fix-access.

The ownership path read and deleted items from an explicit keychain, then recreated them in the default keychain. It also rejected valid service-only refs, could delete matching internet-password items while rewriting a generic password, and used a lossy security CLI fallback that could strip trailing newlines from secrets.

Batch fix-access also unlocked all keychains before processing any item, so one stale or unavailable keychain could abort the whole batch instead of failing only that ref.

Fix this by preserving keychainName on generic-password writes, resolving account-less refs only when unambiguous, reading exact secret data through SecItemCopyMatching for ownership, deleting only the targeted generic-password item, falling back to legacy ACL handling for unsupported cases, and moving batch unlock into per-item handling.

Validation: bun run lint:fix; bun run --filter varlock build:binary; autoreview --mode local --engine codex reported no accepted/actionable findings.
Add Deno execution support to the shared smoke-test runner so the existing smoke tests can be reused under deno run -A instead of maintaining a separate deno-only test file.

Also teach varlock completion generation how to emit shell completion scripts when invoked through Deno, and route keychain-set through the shared runner so it participates in the Deno smoke path.
When takeOwnership is scoped to a named keychain, ownership reads and deletes must not fall back to the default keychain search list if that name cannot be resolved. A typo in --keychain could otherwise read or delete a matching item from another keychain before failing during recreation.\n\nResolve the requested keychain explicitly for both the ownership read and delete paths, and throw keychainNotFound when resolution fails.
takeOwnership has to delete and recreate the generic password item so VarlockEnclave becomes the owner. If recreation failed after the delete, fix-access could permanently lose the user's secret.\n\nCatch recreation failures, immediately attempt to write the saved service/account/value back, and return a specific ownershipTransferFailed error that reports whether the secret value was restored. This keeps fix-access from silently turning a repair attempt into destructive migration.
Change takeOwnership to avoid deleting the original item until a replacement has been created and verified. The flow now creates a temporary generic-password item with the same secret, reads it back to verify it is usable, deletes the original item only after that check succeeds, renames the temporary item to the original service/account, then verifies the final item.\n\nIf the swap fails after the original delete, keep the previous recovery behavior: restore the saved secret value at the original service/account when possible, and report whether that recovery succeeded.
Restore fix-access to the legacy ACL-editing behavior so it preserves existing per-app Keychain access on existing items.

Move the delete-and-recreate ownership rewrite behind a separate keychain take-ownership command. The new command is explicit about the ACL-reset tradeoff and still preserves named-keychain targeting when it recreates generic-password items.

Validation: bun run lint:fix; bun run --filter varlock build:binary; bun run --filter varlock typecheck; autoreview --mode local --engine codex reported no accepted/actionable findings.
Add a macOS Keychain smoke test for the explicit take-ownership path and include it in the keychain smoke scripts.

Validation: bun run lint:fix; bun run --filter varlock build:binary; VARLOCK_RUN_KEYCHAIN_SMOKE=1 bunx vitest run tests/keychain-ownership.test.ts from smoke-tests.
Before running keychain fix-access, assert that a restricted smoke-test item is not readable yet. Add timeout support to the smoke-test runner so the pre-fix read cannot hang indefinitely on Keychain authorization.

Validation: bun run lint:fix; bun run --filter varlock build:binary; VARLOCK_RUN_KEYCHAIN_SMOKE=1 bunx vitest run tests/keychain-fix.test.ts from smoke-tests.
Add a useFallback option to keychain() reads. It defaults to true to preserve the existing /usr/bin/security fallback, but useFallback=false now requires the Swift daemon to read through Apple's framework APIs directly.

Use the option in the keychain fix-access smoke fixture so the test proves fix-access grants VarlockEnclave direct framework access instead of succeeding via the security CLI fallback.

Validation: bun run lint:fix; bun run --filter varlock typecheck; bun run --filter varlock build:binary; VARLOCK_RUN_KEYCHAIN_SMOKE=1 bunx vitest run tests/keychain-fix.test.ts from smoke-tests; autoreview --mode local --engine codex.
Kill and clear Varlock daemon state before and after every macOS Keychain smoke test so stale daemon access state cannot affect prompt behavior or test results.

Validation: bun run lint:fix; bun run --filter varlock build:binary; VARLOCK_RUN_KEYCHAIN_SMOKE=1 bunx vitest run tests/keychain-fix.test.ts from smoke-tests.
Replace the separate post-test keychain daemon cleanup helper with resetVarlockDaemon in both beforeEach and afterEach. The important behavior is killing the daemon and clearing pid/socket state; directory removal is unnecessary.

Validation: bun run lint:fix; bun run --filter varlock build:binary; VARLOCK_RUN_KEYCHAIN_SMOKE=1 bunx vitest run tests/keychain-fix.test.ts from smoke-tests.
@bjesuiter
bjesuiter force-pushed the deno-compatibility branch from 5e42121 to bdd1a0a Compare June 29, 2026 11:20
@theoephraim

Copy link
Copy Markdown
Member

I'm not sure about deferring to security as a fallback. Maybe it could be a per item opt-in if it really solves some very specific use case?

Also I had thought that the main reason to read from existing keychain items is the case where a user is already using that item in other contexts, or for example where an IT department is managing those items for the user. In this case I dont think recreating the item will be a good idea.

One thing to note - it could be that you are seeing more prompts than would actually occur since you are not signing the built binary whereas the real one will be signed.

WalksWithASwagger

This comment was marked as resolved.

@dmno-dev dmno-dev deleted a comment from WalksWithASwagger Jul 18, 2026
@theoephraim

Copy link
Copy Markdown
Member

@pullfrog review this

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues — one minor cleanup inline.

Reviewed changes — this PR adds Deno IPC/completion support and reworks the macOS Keychain flows (fix-access, a new non-destructive cloneToOwned, a new delete, and a /usr/bin/security read fallback), backed by gated smoke tests.

  • Allow Deno clients over IPC — add deno to PeerIdentity.swift's allowedBinaryNames, and generate a Deno-aware completion script (getDenoCompletionExec in cli-executable.ts) via the new @bomb.sh/tab dependency.
  • Rework fix-access to a native read promptfixAccessForRefs now performs a keychainGet(useFallback=false) read (guiding the user to "Always Allow") instead of mutating item ACLs; the keychain-fix-access daemon action and KeychainFixAccessResult type are removed.
  • Add security CLI read fallbackKeychainManager.getItem/getItemOfClass gain a useFallback flag; on errSecAuthFailed/errSecInteractionNotAllowed (or other errors) they fall back to getGenericPasswordViaSecurityCLI. The useFallback flag is threaded through the resolver, daemon client, and daemon dispatch, and documented in the keychain docs.
  • Add keychain cloneToOwned — non-destructive command that reads a source secret and recreates it as a Varlock-owned item (optionally writing the env ref); replaces the previous destructive take-ownership approach.
  • Add keychain delete — new CLI command, daemon keychain-delete action, KeychainManager.deleteGenericPassword, and KeychainDeleteResult type. setGenericPassword also gains an optional keychainName.
  • Add gated smoke tests + helpers — opt-in Deno (VARLOCK_RUN_DENO_COMPAT=1) and Keychain (VARLOCK_RUN_KEYCHAIN_SMOKE=1) suites, a shared resetVarlockDaemon helper, stdin input support in run-varlock, and new fixtures/scripts.
  • Docs + changesets — keychain docs updated for the new commands and useFallback, plus three patch bump files.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Claude Opus𝕏

@bjesuiter

Copy link
Copy Markdown
Contributor Author

I'm not sure about deferring to security as a fallback. Maybe it could be a per item opt-in if it really solves some very specific use case?

=> The CodexBar App by Peter Steinberger does this to avoid too many access prompts when accessing things like: Anthropic OAuth Token for claude code to read usage from the token. In this example, CodexBar is not the owner of the secret too, so i figured, this fallback might be useful for us here.

But I admit, getting into the weeds of keychain behavior feels like a lot of trial and error. As I implemented the fix it seemed to help first, but I later realized your point of a non-signed dev build locally.

One thing to note - it could be that you are seeing more prompts than would actually occur since you are not signing the built binary whereas the real one will be signed.

After fixing that by signing the builds with my own apple developer key, the prompt behavior got better already.
Obviously, my signed version cannot read original varlock owned secrets, but the behavior between the "security" cli and my self-signed varlock helper build should be consistent with the behavior your originally signed varlock build should show.
So i assume that the test cases i added should still benefit us by avoiding regressions in keychain behavior :)

Also I had thought that the main reason to read from existing keychain items is the case where a user is already using that item in other contexts, or for example where an IT department is managing those items for the user. In this case I dont think recreating the item will be a good idea.

=> yes, I agree, the copy-then-own method seems not good, so i ripped it out again.
I'll look into the fallback to security cli again to check if we need this or not.

@bjesuiter

bjesuiter commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up on the Keychain behavior: I removed the /usr/bin/security read fallback. Secret reads now use Security.framework directly again, so macOS evaluates access for VarlockEnclave itself.

For the original case—someone wants to use a secret that already exists in Keychain and is not owned by Varlock—the best flow is to choose Always Allow when macOS prompts. That adds VarlockEnclave to that item's ACL, so subsequent reads should no longer prompt. Allow Once only authorizes the current read. (you probably know this, but I wanted to conserve this here for people who might read this thread :D)

I also dug further into CodexBar. Delegating reads to /usr/bin/security was not a general prompt-suppression mechanism and CodexBar later disabled that production path because security find-generic-password -w can itself prompt. Other parts of its strategy are more relevant: gating/caching reads and caching credentials in a CodexBar-owned Keychain item. The latter is conceptually close to our explicit, non-destructive cloneToOwned flow.

@theoephraim what do you think of keeping these as the two explicit options?

  1. Read an existing/externally managed item in place via Security.framework and choose Always Allow once.
  2. Optionally use cloneToOwned when the user deliberately wants a Varlock-owned cached copy, while leaving the original untouched.
    => i could also remove the cloneToOwned again and we think about this caching mechanic more ^^

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — reviewed the native Keychain read revision added since the prior Pullfrog review.

  • Removed the command-line fallback — deleted the /usr/bin/security generic-password fallback so secret reads consistently authorize VarlockEnclave through Security.framework.
  • Removed fallback configuration — removed useFallback from the resolver, daemon client, daemon dispatch, documentation, changesets, and generated smoke fixture.
  • Clarified persistent access behavior — updated the Keychain guidance to explain how choosing Always Allow grants VarlockEnclave persistent access, while Allow Once applies only to the current read.

Pullfrog  | View workflow run | Using azure/gpt-5.6-sol𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes focused on the signed-helper requirement added since the previous Pullfrog pass.

  • Added a Keychain smoke preflight: gated macOS Keychain runs now fail before test execution when the packaged VarlockEnclave.app is missing, invalidly signed, ad-hoc signed, or lacks a stable Team ID.
  • Added signing remediation guidance: the failure message points local contributors to the repository's APPLE_SIGNING_IDENTITY build path, which matches the native build script.

Pullfrog  | View workflow run | Using azure/gpt-5.6-sol𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants