Skip to content

feat(appkit): execution identity and resource provisioning base - #592

Open
MarioCadenas wants to merge 13 commits into
mainfrom
resource-and-execution-base-pr
Open

MarioCadenas wants to merge 13 commits into
mainfrom
resource-and-execution-base-pr

Conversation

@MarioCadenas

@MarioCadenas MarioCadenas commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

This adds the shared execution-identity and resource-provisioning foundations (S1 and S2). It is the bottom of the stack, targeting main; later tickets will build on this PR.

  • S1: replace the active UserContext with immutable CallerContext and a principal discriminant containing only the user variant. Add getCurrentPrincipalKey() (app or user:<id>) and getCurrentActorId() (the initiating user, or undefined in service scope). Rename the context creation and scope helpers, retaining deprecated compatibility exports and flat identity accessors with one-time warnings.
  • S2: add the authoritative SCOPE_BY_TYPE mapping, APP_ONLY_RESOURCE_TYPES for secret, database, and postgres, and the optional plugin scopes field validated against the seven capability-only scopes.
  • Review follow-up: remove warehouseId from CallerContext, its factory, and its immutable snapshot. Move warehouse discovery and binding ownership into the internal app-resource layer, independently of execution identity. Export getWarehouseId() from @databricks/appkit and migrate Analytics to the resource accessor. Retain ServiceContextState.warehouseId, the old context-module helper, and the deprecated user-context property as compatibility accessors with one-time warnings. Explicit legacy warehouse overrides remain isolated outside caller identity. SQL still uses the active SP or user client.

This is behavior-preserving for SP/OBO execution: SP stays the default, forwarded-header handling and credential selection are unchanged, and group is deferred. getCurrentUserId() retains the existing bare user/SP IDs so cache keys and telemetry remain unchanged. Existing Lakebase OBO routing is preserved, while the v1 capability contract still marks its resources app-only as specified by the design. Warehouse environment lookup and development discovery are unchanged.

Internal warehouse consumers, test setup, and ordinary assertions now use resource bindings or the canonical accessor. Only two focused service-context compatibility tests read the deprecated property. ServiceContextState.warehouseId is retained for one release with its one-time warning before removal. Exported test fixtures preserve the legacy field for external test callers, but never read it to configure resources.

Scope is limited to S1 and S2 plus the requested warehouse decoupling and deprecation path. Top-level asUser, tool dispatch, cache-key migration, sync resolution, generated JSON-schema wiring, CLI generators, and templates remain follow-up work. The existing ServiceContext.initialize() entry point still coordinates startup for compatibility, but warehouse state and discovery now belong to app resources. This does not implement general plugin-resource injection or resource registry integration.

Design: design-docs/execution-identity-e2e.md, sections 5.1.1 and 5.2.3. Tickets: task-39o and task-ouk. Section 5.1.1 currently shows warehouseId on CallerContext; removing it and deprecating the service-context property follow the owner's subsequent review requests. The untracked design document is not modified or included in this PR.

Validation on the PR branch:

  • pnpm run generate:types followed by git diff --exit-code: passed after committing the changes.
  • pnpm -r typecheck: passed.
  • env -u NO_COLOR pnpm exec vitest run --project appkit --project shared: 4,583 passed, one existing skipped test.
  • pnpm -r --filter=!docs build:package: passed.
  • pnpm docs:build: passed.
  • pnpm check:fix and pnpm check: passed, with existing unrelated lint warnings.
  • pnpm knip: passed. Commit hooks also passed lint-staged and commitlint.

Regression tests cover warehouse-free callers, unchanged SQL warehouse/client selection for SP and OBO, missing warehouse errors, nested/concurrent legacy override isolation, the package-root accessor, immutable resource snapshots, failed and concurrent initialization, and reset behavior. The Analytics guide documents the migration path. Generated API documentation remains excluded. The generated plugin manifest JSON schema is included to keep CI's freshness check aligned with the S2 scopes contract; no new generator or template wiring is added.

@MarioCadenas
MarioCadenas requested a review from a team as a code owner September 23, 2026 10:54
@MarioCadenas
MarioCadenas requested a review from ditadi September 23, 2026 10:54
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

📦 Bundle size report

Compared against bundle-size-baseline.json (main).

@databricks/appkit

npm tarball (packed): 1.2 MB (+8.7 KB) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist raw gzip
JS (runtime) 1.2 MB (+8.0 KB) 431 KB (+3.0 KB)
Type declarations 449 KB (+3.1 KB) 163 KB (+1.4 KB)
Source maps 2.4 MB (+17 KB) 808 KB (+6.2 KB)
Other 11 KB 3.7 KB
Total 4.0 MB (+28 KB) 1.4 MB (+11 KB)
Per-entry composition (own code — deps external (as shipped))
Entry Initial (gz) Lazy (gz) Total (gz) node_modules (min) Own code (min)
. 97 KB (+840 B) 2.5 KB 100 KB (+840 B) external 318 KB (+2.6 KB)
./beta 94 KB (+694 B) 479 B (+1 B) 95 KB (+695 B) external 284 KB (+1.9 KB)
./testing 39 KB (+808 B) 31 KB (+4 B) 70 KB (+812 B) external 205 KB (+2.4 KB)
./tsdown 520 B 0 B 520 B external 813 B
./type-generator 23 KB 0 B 23 KB external 65 KB

Chunks:

Entry Chunk Load Size (gz)
. index.js initial 93 KB
. utils.js initial 4.6 KB
. remote-tunnel-manager.js lazy 2.5 KB
./beta beta.js initial 77 KB
./beta stream-manager.js initial 5.8 KB
./beta databricks.js initial 3.3 KB
./beta wide-event-emitter.js initial 3.1 KB
./beta configuration.js initial 2.3 KB
./beta service-context.js initial 1.9 KB
./beta client.js initial 593 B
./beta client-options.js initial 219 B
./beta supervisor-api.js lazy 191 B
./beta databricks.js lazy 165 B
./beta index.js lazy 123 B
./testing manifest.js initial 26 KB
./testing index.js initial 10 KB
./testing wide-event-emitter.js initial 2.9 KB
./testing index.js lazy 27 KB
./testing remote-tunnel-manager.js lazy 2.5 KB
./testing utils.js lazy 1.8 KB
./tsdown index.js initial 520 B
./type-generator index.js initial 23 KB

@databricks/appkit-ui

npm tarball (packed): 350 KB — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist raw gzip
JS (runtime) 395 KB 132 KB
Type declarations 229 KB 84 KB
Source maps 766 KB 253 KB
CSS 16 KB 3.2 KB
Total 1.4 MB 472 KB
Per-entry composition (consumer bundle — deps bundled, peerDeps external)
Entry Initial (gz) Lazy (gz) Total (gz) node_modules (min) Own code (min)
./js 5.3 KB 49 KB 55 KB 208 KB 14 KB
./js/beta 20 B 0 B 20 B 0 B 0 B
./react 432 KB 49 KB 481 KB 1.3 MB 177 KB
./react/beta 1.0 KB 0 B 1.0 KB 0 B 1.9 KB

Chunks:

Entry Chunk Load Size (gz)
./js index.js initial 5.2 KB
./js chunk initial 120 B
./js apache-arrow lazy 49 KB
./js/beta beta.js initial 20 B
./react index.js initial 430 KB
./react tslib initial 2.1 KB
./react apache-arrow lazy 49 KB
./react/beta beta.js initial 1.0 KB

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

🤖 AppKit PR bot

🔬 Run evals

Start an eval for this PR from the evals-monitor app: Go to Evals Monitor →

📦 Try this PR's app template

Scaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh auth login — and the Databricks CLI):

gh run download 37298605556 -R databricks/appkit -n appkit-template-0.82.0-pr.1def48f-resource-and-execution-base-pr-592 -D appkit-pr-592 \
  && unzip -o "appkit-pr-592/appkit-template-0.82.0-pr.1def48f-resource-and-execution-base-pr-592.zip" -d "appkit-pr-592" \
  && databricks apps init --template "appkit-pr-592"

The template pins @databricks/appkit and @databricks/appkit-ui to tarballs built from this branch, so the scaffolded app runs against this PR's code.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Resolve the shared warehouse from the app-level service context without
carrying resource fields on CallerContext. Preserve deprecated user-context
warehouse access and isolate explicit legacy overrides outside caller identity.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Include the capability-only scopes already defined by the manifest schema
so the generated-file freshness check passes.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Separate warehouse bindings and discovery from execution identity.
Expose getWarehouseId at the package root and retain deprecated context
accessors with one-time warnings. Preserve SP defaults, startup behavior,
and explicit legacy user-context overrides.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Build mock resource bindings independently of service identity and migrate
ordinary tests to the resource accessor. Keep the deprecated public field
and focused compatibility coverage for one release before removal.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Keep canonical caller snapshots free of legacy fields and resource bindings.
Preserve deprecated accessors and aliases while naming warehouse ownership explicitly.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
@MarioCadenas
MarioCadenas force-pushed the resource-and-execution-base-pr branch from 9efc40f to 12d7eb1 Compare September 29, 2026 15:03
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Co-authored-by: Isaac <no-reply@databricks.com>
MarioCadenas added a commit that referenced this pull request Oct 1, 2026
Brings in main through #592, including #625. Resolves the conflict in
docs/docs/plugins/execution-context.md by keeping this branch's caller
scope and agents sections and keeping #625's surface table, updated for
behavior this branch changes: the agents HTTP routes now open user scope,
so plugin-toolkit and hand-rolled tool calls run as the user, while the
model call stays on the service principal because the adapter builds its
own client. Analytics `.obo.sql` queries are listed as a user-lane surface.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
MarioCadenas and others added 2 commits October 1, 2026 14:06
`Principal` was introduced in this stack as a deprecated alias of
`CallerPrincipal`, but it was never shipped, so there is nothing to stay
backward compatible with. Remove the alias and its context barrel
re-export; the only consumer (standalone runAgent) switches to
`CallerPrincipal`.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
);
});

if (response.warehouses.length === 0) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be using warehouses instead of response.warehouses ?

Databricks list APIs omit empty arrays, so a workspace with no usable
warehouses returns `{}`. discoverWarehouseId already defaulted the sorted
list to `[]` but then read `response.warehouses.length`, which threw a
TypeError instead of the intended ConfigurationError. Check the guarded
`warehouses` list instead.

Adds a regression test covering both an omitted `warehouses` key and an
empty list.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants