Skip to content

Lazy-load app context in the public_command_metadata analytics hook - #8286

Draft
isaacroldan wants to merge 2 commits into
background-analyticsfrom
analytics-timing-test
Draft

Lazy-load app context in the public_command_metadata analytics hook#8286
isaacroldan wants to merge 2 commits into
background-analyticsfrom
analytics-timing-test

Conversation

@isaacroldan

@isaacroldan isaacroldan commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

WHY are these changes introduced?

Building the analytics payload after every command takes ~150–190ms, roughly as long as the Monorail HTTP send itself. Almost all of that time is oclif dynamically importing the public_command_metadata hook module: the hook eagerly imported services/app-context.js, which pulls in a large chunk of the @shopify/app module graph — even for commands like shopify version run outside any app project.

This is stacked on #7615: that PR moves analytics delivery out of the critical path, but the payload is still built in the main process, so the hook-import cost remains in the foreground.

WHAT is this pull request doing?

  • The hook now defers import('../services/app-context.js') until it is actually needed: api_key not already set in metadata, and a shopify.app*.toml found by walking up from the current directory.
  • The shopify.app*.toml glob moves to the import-free constants.ts and is shared with models/project/project.ts instead of being duplicated.
  • Hook tests use real temporary directories instead of a mocked cwd, with a new test covering the not-in-an-app-project skip.

Behavior is unchanged inside app projects: the hook still loads the app context there, and real app commands stay free because the modules are already loaded and api_key is already set.

Benchmarks

shopify version (production bundle, non-app directory, macOS, avg of 3 runs):

Wall clock User CPU
main 1.50s 0.49s
#7615 alone 0.40s 0.48s
#7615 + this PR 0.23s 0.22s

Isolated effect of this change (measured against main before stacking): buildPayload drops from ~190ms to ~14ms. With #7615 the payload build is the bulk of what's left in the foreground, so this PR removes ~40% of the remaining time — and halves the CPU spent parsing/evaluating modules that were never used.

How to test your changes?

  • pnpm nx bundle cli
  • Outside an app project: time node packages/cli/bin/run.js version — compare against main
  • Inside an app project: shopify version --verbose and check the analytics payload still includes app metadata (api_key, etc.)

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows)
  • I've considered possible documentation changes
  • I've considered analytics changes to measure impact

🤖 Generated with Claude Code

@github-actions github-actions Bot added the no-changelog This PR doesn't include a changeset entry. Is an internal only change not relevant to end users. label Aug 10, 2026
The analytics postrun hook eagerly imported app-context.js, pulling in a
large module graph on every command — ~150-190ms of buildPayload time even
for commands like 'shopify version' outside an app project.

The hook now checks for a shopify.app*.toml in the directory tree before
dynamically importing the app context, so commands outside an app project
skip the import entirely.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@isaacroldan
isaacroldan force-pushed the analytics-timing-test branch from b455b2d to 8fb2bd4 Compare August 10, 2026 12:10
@isaacroldan
isaacroldan changed the base branch from main to background-analytics August 10, 2026 12:10
@github-actions github-actions Bot added Area: @shopify/app @shopify/app package issues and removed no-changelog This PR doesn't include a changeset entry. Is an internal only change not relevant to end users. labels Aug 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/private/node/otel-metrics.d.ts
@@ -2,7 +2,7 @@ import { OtelService } from '../../public/node/vendor/otel-js/service/types.js';
 import { DefaultOtelServiceOptions } from '../../public/node/vendor/otel-js/service/DefaultOtelService/DefaultOtelService.js';
 type MetricRecorder = 'console' | {
     type: 'otel';
-    otel: Pick<OtelService, 'record'>;
+    otel: Pick<OtelService, 'getMeterProvider' | 'record'>;
 };
 interface Timing {
     active: number;
packages/cli-kit/dist/public/node/abort.d.ts
@@ -1,24 +1,16 @@
+import { AbortController as NodeAbortController, AbortSignal as NodeAbortControllerSignal } from 'node-abort-controller';
 /**
  * The AbortController interface represents a controller object that allows you to abort one or more Web requests as and when desired.
  *
  * - MDN Documentation: https://developer.mozilla.org/en-US/docs/Web/API/AbortController
  *
- * This class exists to keep the historical `@shopify/cli-kit/node/abort` import path working
- * now that Node provides AbortController natively.
+ * This class is necessary because AbortController support was added to Node 15 and the minimum
+ * version that we support is Node 14.
  */
-export declare class AbortController extends globalThis.AbortController {
+export declare class AbortController extends NodeAbortController {
 }
 /**
  * The AbortSignal interface represents a signal object that allows you to communicate with a DOM request (such as a fetch request) and abort it if required via an AbortController object.
- *
- * Note that AbortSignal cannot be constructed directly. Get one from an AbortController's
- * `signal` property or from the static helpers such as `AbortSignal.timeout()`.
  */
-export declare const AbortSignal: {
-    new (): globalThis.AbortSignal;
-    prototype: globalThis.AbortSignal;
-    abort(reason?: any): globalThis.AbortSignal;
-    any(signals: globalThis.AbortSignal[]): globalThis.AbortSignal;
-    timeout(milliseconds: number): globalThis.AbortSignal;
-};
-export type AbortSignal = globalThis.AbortSignal;
\ No newline at end of file
+export declare class AbortSignal extends NodeAbortControllerSignal {
+}
\ No newline at end of file
packages/cli-kit/dist/public/node/analytics.d.ts
@@ -6,6 +6,7 @@ interface ReportAnalyticsEventOptions {
     errorMessage?: string;
     exitMode: CommandExitMode;
 }
+export declare function sendAnalyticsEventFromStdin(): Promise<void>;
 /**
  * Report an analytics event, sending it off to Monorail -- Shopify's internal analytics service.
  *
packages/cli-kit/dist/public/node/error.d.ts
@@ -1,6 +1,7 @@
 import { OutputMessage } from './output.js';
 import { type InlineToken, type TokenItem } from '../../private/node/ui/components/token-item.js';
 import type { AlertCustomSection } from './ui.js';
+export { ExtendableError } from 'ts-error';
 export declare enum FatalErrorType {
     Abort = 0,
     AbortSilent = 1,
packages/cli-kit/dist/public/node/fs.d.ts
@@ -86,6 +86,8 @@ export declare function appendFile(path: string, content: string): Promise<void>
 export declare function appendFileSync(path: string, data: string): void;
 export interface WriteOptions {
     encoding: BufferEncoding;
+    mode?: number;
+    flag?: string;
 }
 /**
  * Writes content to file at path.
packages/cli-kit/dist/private/node/session/exchange.d.ts
@@ -1,9 +1,10 @@
 import { ApplicationToken, IdentityToken } from './schema.js';
 import { API } from '../api.js';
 import { Result } from '../../../public/node/result.js';
-export declare class InvalidGrantError extends Error {
+import { ExtendableError } from '../../../public/node/error.js';
+export declare class InvalidGrantError extends ExtendableError {
 }
-export declare class InvalidRequestError extends Error {
+export declare class InvalidRequestError extends ExtendableError {
 }
 export interface ExchangeScopes {
     admin: string[];
packages/cli-kit/dist/public/node/plugins/tunnel.d.ts
@@ -1,3 +1,4 @@
+import { ExtendableError } from '../error.js';
 import { OutputMessage } from '../output.js';
 import { FanoutHookFunction, PluginReturnsForHook } from '../plugins.js';
 import { Result } from '../result.js';
@@ -21,7 +22,7 @@ export type TunnelStatusType = {
     message: TokenItem | OutputMessage;
     tryMessage?: TokenItem | OutputMessage | null;
 };
-export declare class TunnelError extends Error {
+export declare class TunnelError extends ExtendableError {
     type: TunnelErrorType;
     constructor(type: TunnelErrorType, message?: string);
 }
packages/cli-kit/dist/public/node/vendor/otel-js/service/types.d.ts
@@ -1,5 +1,5 @@
-import type { Counter, Histogram, MeterProvider, MetricAttributes, MetricOptions, UpDownCounter } from '@opentelemetry/api';
-import type { ViewOptions } from '@opentelemetry/sdk-metrics';
+import type { Counter, Histogram, MetricAttributes, MetricOptions, UpDownCounter } from '@opentelemetry/api';
+import type { MeterProvider, ViewOptions } from '@opentelemetry/sdk-metrics';
 export type CustomMetricLabels<TLabels extends Record<TKeys, MetricAttributes>, TKeys extends string = keyof TLabels & string> = {
     [P in TKeys]: TLabels[P] extends MetricAttributes ? TLabels[P] : never;
 };

@gonzaloriestra
gonzaloriestra force-pushed the background-analytics branch 4 times, most recently from a610c01 to 91a8307 Compare August 12, 2026 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: @shopify/app @shopify/app package issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants