From 1da68b068c86480de7e7dc45681b728abc9ed487 Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Wed, 28 Oct 2026 16:45:37 -0400 Subject: [PATCH 01/14] feat(migrate): record the source, funnel stage, and target app and instance in telemetry Co-Authored-By: Claude Opus 5.5 --- .changeset/migrate-telemetry.md | 5 + README.md | 5 +- .../cli-core/src/commands/migrate/README.md | 30 +++++ .../src/commands/migrate/export/clerk.ts | 8 ++ .../src/commands/migrate/export/index.ts | 10 ++ .../commands/migrate/export/shared.test.ts | 29 +++- .../src/commands/migrate/export/shared.ts | 8 +- .../src/commands/migrate/index.test.ts | 59 ++++++++ .../cli-core/src/commands/migrate/index.ts | 7 + .../cli-core/src/commands/migrate/run.test.ts | 118 +++++++++++++++- packages/cli-core/src/commands/migrate/run.ts | 29 +++- .../cli-core/src/commands/migrate/runs.ts | 3 + .../src/commands/migrate/telemetry.ts | 33 +++++ .../src/commands/migrate/undo.test.ts | 51 ++++++- .../cli-core/src/commands/migrate/undo.ts | 18 ++- packages/cli-core/src/lib/telemetry.test.ts | 105 ++++++++++++++- packages/cli-core/src/lib/telemetry.ts | 126 ++++++++++++++++-- 17 files changed, 625 insertions(+), 19 deletions(-) create mode 100644 .changeset/migrate-telemetry.md create mode 100644 packages/cli-core/src/commands/migrate/telemetry.ts diff --git a/.changeset/migrate-telemetry.md b/.changeset/migrate-telemetry.md new file mode 100644 index 000000000..42ad0bfaa --- /dev/null +++ b/.changeset/migrate-telemetry.md @@ -0,0 +1,5 @@ +--- +"clerk": patch +--- + +Record in usage telemetry which platform a `clerk migrate` run's users come from, how far the run got, and the Clerk app and instance it acted on, and say so in the first-run telemetry notice. diff --git a/README.md b/README.md index 6984c4ded..a6eb2527a 100644 --- a/README.md +++ b/README.md @@ -102,8 +102,9 @@ Commands: ## Telemetry The Clerk CLI collects usage telemetry: command name, flag names, duration, outcome, -environment signals (OS, install method, terminal), a random machine identifier — and -your workspace and app IDs when a project is linked. It never collects command +environment signals (OS, install method, terminal), a random machine identifier, the +platform a migration's users come from, and the workspace, app and instance IDs of the +Clerk app a command acts on or the project is linked to. It never collects command arguments, option values, file paths, or personal data, except the text you choose to send with `clerk feedback`, which is scrubbed on a best-effort basis for common secret formats, home-directory paths and email addresses before it leaves your machine; diff --git a/packages/cli-core/src/commands/migrate/README.md b/packages/cli-core/src/commands/migrate/README.md index 5edba5995..de23476f2 100644 --- a/packages/cli-core/src/commands/migrate/README.md +++ b/packages/cli-core/src/commands/migrate/README.md @@ -1510,6 +1510,36 @@ stamping every user with today's. | `skipLegalChecks` | `boolean` | Skip legal acceptance checks | | `skipPasswordChecks` | `boolean` | Skip password requirements on import | +## What telemetry records + +Every `clerk migrate` event carries the fields every command sends, plus four that say what the run did. Each is null when the run never learned it. + +| Field | What it holds | +| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `migrate_source` | Where the users come from: a built-in source or export platform (`auth0`, `clerk`, …), or `custom` for a source you wrote. Never its path or its key. | +| `instance_id` | The Clerk instance the command acted on (`ins_…`). For `export clerk`, the instance it read. | +| `app_id` | Its app (`app_…`): from `--app`, or from the linked profile when that profile owns the instance. A key for another app's instance sends null, not the linked app. | +| `stage` | How far the run got. One funnel per subcommand, below. | + +`workspace_id` is the linked profile's only when `app_id` is that profile's app. A `key_…` stand-in, the hash a run records when `GET /v1/instance` fails, is never sent: neither is an `--instance` alias or an app label. + +Stages, in the order each subcommand runs them. The last one set is the one sent, so a failure or a Ctrl-C reports where the run stopped: + +- `import`: `migrate_source`, `migrate_target_setup` (sign-in, link), `migrate_input`, `migrate_target`, `migrate_resume`, `migrate_load`, `migrate_checks`, `migrate_confirm`, `migrate_import`, `done`. Terminal branches: `migrate_already_imported`, `migrate_dry_run`, `migrate_nothing_to_import`. +- `export`: `migrate_pick_platform` (bare `clerk migrate export` only), `migrate_export_read` (credentials and reading users), `migrate_export_write`, `done`. +- `undo`: `migrate_run_lookup`, `migrate_target`, `migrate_inspect`, `migrate_confirm`, `migrate_delete`, `done`. Terminal branch: `migrate_dry_run`. + +`runs` and `sources` are one step each and send no stage. + +A run that finishes part-way exits nonzero without throwing, and carries a code so it is not an uncoded error: + +| Ending | `error_code` | Exit | +| ------------------------------------------ | -------------------------- | ---- | +| `import --dry-run` the checks would refuse | `migrate_checks_rejected` | 2 | +| `import` where some users failed | `migrate_import_partial` | 1 | +| `undo` with deletes left | `migrate_undo_incomplete` | 1 | +| `export` the platform cut short | `migrate_export_truncated` | 1 | + ## API endpoints | Method | Path | Used by | diff --git a/packages/cli-core/src/commands/migrate/export/clerk.ts b/packages/cli-core/src/commands/migrate/export/clerk.ts index 48f32bad8..fd2bdc3de 100644 --- a/packages/cli-core/src/commands/migrate/export/clerk.ts +++ b/packages/cli-core/src/commands/migrate/export/clerk.ts @@ -28,6 +28,7 @@ import { } from "../lib/target.ts"; import { resolveClerkSource } from "./clerk-source.ts"; import { finishExport, startExportRun } from "./shared.ts"; +import { recordMigrateTarget } from "../telemetry.ts"; /** BAPI's maximum page size for `GET /v1/users`. */ const PAGE_SIZE = 500; @@ -296,6 +297,13 @@ export async function exportClerk(options: ExportClerkOptions): Promise { instanceId: identity.instanceId, ...(keySource ? { keySource } : {}), }; + // The source instance: for this command, the one it acts on. + // `--app` named the app unless a key overrode it; a linked profile's app + // is filled in from its instances. + recordMigrateTarget({ + appId: keyFrom ? undefined : options.app, + instanceId: identity.instanceId, + }); if (!options.json) printTarget(target); const users = await withSpinner("Fetching users from Clerk...", async (spinner) => diff --git a/packages/cli-core/src/commands/migrate/export/index.ts b/packages/cli-core/src/commands/migrate/export/index.ts index bef1ba1e0..224bf9fab 100644 --- a/packages/cli-core/src/commands/migrate/export/index.ts +++ b/packages/cli-core/src/commands/migrate/export/index.ts @@ -12,6 +12,8 @@ import { exportWorkOs } from "./workos.ts"; import type { DbExportOptions } from "./db-options.ts"; import { RUNS_DIR_DESCRIPTION, RUNS_DIR_FLAG } from "../lib/run-store.ts"; import { exportPlatformKeys, exportPlatforms, getExportPlatform } from "./registry.ts"; +import { setTelemetryStage } from "../../../lib/telemetry.ts"; +import { recordMigrateSource } from "../telemetry.ts"; /** * Bare `clerk migrate export` — pick a platform, then run its export. @@ -33,6 +35,7 @@ export async function exportPicker(options: Record = {}): Promi ); } + setTelemetryStage("migrate_pick_platform"); const platform = await select({ message: "Which platform are you exporting from?", choices: exportPlatforms.map((entry) => ({ @@ -46,6 +49,8 @@ export async function exportPicker(options: Record = {}): Promi // Unreachable via the picker; a guard so a registry edit cannot silently // produce a choice with nothing behind it. if (!entry) throwUsageError(`Unknown export platform "${platform}".`); + recordMigrateSource(platform); + setTelemetryStage("migrate_export_read"); await entry.run(options); } @@ -118,6 +123,11 @@ export function registerMigrateExport(migrateCommand: Command<[], Record { + // The bare picker records both once a platform is chosen. + if (actionCommand !== exportCommand) { + recordMigrateSource(actionCommand.name()); + setTelemetryStage("migrate_export_read"); + } const { output } = actionCommand.opts() as { output?: unknown }; if (typeof output === "string" && !output.toLowerCase().endsWith(".json")) { throwUsageError( diff --git a/packages/cli-core/src/commands/migrate/export/shared.test.ts b/packages/cli-core/src/commands/migrate/export/shared.test.ts index 3c2a03aa4..5fa8fa531 100644 --- a/packages/cli-core/src/commands/migrate/export/shared.test.ts +++ b/packages/cli-core/src/commands/migrate/export/shared.test.ts @@ -3,7 +3,7 @@ import fs from "node:fs"; import os from "node:os"; import path from "node:path"; import { quoteArg } from "../../../lib/json-body.ts"; -import { useCaptureLog } from "../../../test/lib/stubs.ts"; +import { captureTelemetryPayload, useCaptureLog } from "../../../test/lib/stubs.ts"; import { readEnvelope } from "../lib/export-file.ts"; import { latestUserLines, readRun } from "../lib/run-store.ts"; import { finishExport, formatImportCommand, startExportRun } from "./shared.ts"; @@ -196,6 +196,33 @@ describe("finishExport", () => { } }); + test("a finished export ends at done", async () => { + const { payload } = await captureTelemetryPayload("migrate export auth0", async () => { + const run = await startExportRun({ runsDir }, { platform: "auth0" }); + finishExport({ run, options: {}, users, coverage }); + }); + expect(payload).toMatchObject({ outcome: "success", stage: "done" }); + }); + + test("a truncated export is coded", async () => { + const { payload } = await captureTelemetryPayload("migrate export auth0", async () => { + const run = await startExportRun({ runsDir }, { platform: "auth0" }); + finishExport({ run, options: {}, users, coverage, truncated: true }); + }); + expect(payload).toMatchObject({ + outcome: "error", + stage: "done", + error_code: "migrate_export_truncated", + }); + }); + + test("starting the export run records migrate_export_write", async () => { + const { payload } = await captureTelemetryPayload("migrate export auth0", async () => { + await startExportRun({ runsDir }, { platform: "auth0" }); + }); + expect(payload.stage).toBe("migrate_export_write"); + }); + test("prints the import command by run ID", async () => { const run = await startExportRun({ runsDir }, { platform: "clerk" }); diff --git a/packages/cli-core/src/commands/migrate/export/shared.ts b/packages/cli-core/src/commands/migrate/export/shared.ts index b3dc4bfac..6ab26f018 100644 --- a/packages/cli-core/src/commands/migrate/export/shared.ts +++ b/packages/cli-core/src/commands/migrate/export/shared.ts @@ -12,6 +12,7 @@ import fs from "node:fs"; import path from "node:path"; import { dim, green, yellow } from "../../../lib/color.ts"; import { CliError } from "../../../lib/errors.ts"; +import { declareSoftExitOutcome, setTelemetryStage } from "../../../lib/telemetry.ts"; import { log } from "../../../lib/log.ts"; import { carryFlags, withFlags } from "../lib/carry-flags.ts"; import { ENVELOPE_VERSION, type ExportEnvelope } from "../lib/export-file.ts"; @@ -45,6 +46,7 @@ export async function startExportRun( options: ExportCommonOptions, target: RunTarget, ): Promise { + setTelemetryStage("migrate_export_write"); const runsDir = await resolveRunsDir(options.runsDir, { write: true }); return startRun(runsDir, { kind: "export", target, source: target.platform }); } @@ -199,7 +201,11 @@ export function finishExport(input: FinishExportInput): FinishedExport { const skipped = record.counts.skipped ?? 0; // Partial, like an import with failures: the file holds what came back, and // the exit says it is not everyone, where `truncated` alone is easy to miss. - if (input.truncated) process.exitCode = 1; + setTelemetryStage("done"); + if (input.truncated) { + process.exitCode = 1; + declareSoftExitOutcome("error", "migrate_export_truncated"); + } if (options.json) { log.data( diff --git a/packages/cli-core/src/commands/migrate/index.test.ts b/packages/cli-core/src/commands/migrate/index.test.ts index 2426b1cfa..fc5430d9a 100644 --- a/packages/cli-core/src/commands/migrate/index.test.ts +++ b/packages/cli-core/src/commands/migrate/index.test.ts @@ -6,6 +6,7 @@ import { getMode, setMode } from "../../mode.ts"; import { createProgram } from "../../cli-program.ts"; import { exportPlatformKeys } from "./export/registry.ts"; import { isAssumeYes, setAssumeYes } from "./lib/assume-yes.ts"; +import { captureTelemetryPayload } from "../../test/lib/stubs.ts"; let saved: string | undefined; beforeAll(() => { @@ -302,3 +303,61 @@ describe("the migrate group's -y hook", () => { expect(await parse(["migrate", "import", missing, "--secret-key", "sk_test_x"])).toBe(false); }); }); + +describe("what the migrate hooks record for telemetry", () => { + let missing: string; + let missingDb: string; + beforeAll(() => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "clerk-migrate-telemetry-")); + missing = path.join(dir, "none.json"); + missingDb = path.join(dir, "none.sqlite"); + }); + + // The action is allowed to fail: only what the hooks record is under test. + async function payloadOf(argv: string[]) { + const program = createProgram(); + program.exitOverride(); + const { payload } = await captureTelemetryPayload( + "migrate", + () => + program.parseAsync(["node", "clerk", ...argv]).then( + () => undefined, + () => undefined, + ), + { result: { outcome: "error", exitCode: 1 } }, + ); + return payload; + } + + test("--app and --instance IDs are recorded before anything resolves", async () => { + const payload = await payloadOf([ + "migrate", + "import", + missing, + "--app", + "app_x", + "--instance", + "ins_y", + ]); + expect(payload).toMatchObject({ app_id: "app_x", instance_id: "ins_y" }); + }); + + test("an --instance alias is not an ID, and is dropped", async () => { + const payload = await payloadOf([ + "migrate", + "import", + missing, + "--app", + "app_x", + "--instance", + "dev", + ]); + expect(payload).toMatchObject({ app_id: "app_x", instance_id: null }); + }); + + test("an export subcommand records its platform as the source", async () => { + const payload = await payloadOf(["migrate", "export", "supabase", "--db-url", missingDb]); + expect(payload.migrate_source).toBe("supabase"); + expect(payload.stage).toBe("migrate_export_read"); + }); +}); diff --git a/packages/cli-core/src/commands/migrate/index.ts b/packages/cli-core/src/commands/migrate/index.ts index f03b8fda7..25c6740f9 100644 --- a/packages/cli-core/src/commands/migrate/index.ts +++ b/packages/cli-core/src/commands/migrate/index.ts @@ -3,6 +3,7 @@ import { isExperimentEnabled, requireExperiment } from "../../lib/experimental.t import { parseIntegerOption } from "../../lib/option-parsers.ts"; import { setMode } from "../../mode.ts"; import { setAssumeYes } from "./lib/assume-yes.ts"; +import { recordMigrateTarget } from "./telemetry.ts"; import { registerMigrateExport } from "./export/index.ts"; import { RUNS_DIR_DESCRIPTION, RUNS_DIR_FLAG } from "./lib/run-store.ts"; import { run } from "./run.ts"; @@ -70,6 +71,12 @@ export function registerMigrate(program: Program, env: NodeJS.ProcessEnv = proce const opts = actionCommand.optsWithGlobals(); setAssumeYes(Boolean(opts.yes)); if (opts.json && !opts.mode) setMode("agent"); + // A hint, so a run that fails before its target resolves still names it. + // Aliases like `--instance dev` and app labels are dropped. + recordMigrateTarget({ + appId: typeof opts.app === "string" ? opts.app : undefined, + instanceId: typeof opts.instance === "string" ? opts.instance : undefined, + }); }); // Named, not `isDefault`. `import` and `export` are the two directions this diff --git a/packages/cli-core/src/commands/migrate/run.test.ts b/packages/cli-core/src/commands/migrate/run.test.ts index 067c90e9d..345fbe0e2 100644 --- a/packages/cli-core/src/commands/migrate/run.test.ts +++ b/packages/cli-core/src/commands/migrate/run.test.ts @@ -12,10 +12,14 @@ import { import fs from "node:fs"; import os from "node:os"; import path from "node:path"; -import { _setConfigDir } from "../../lib/config.ts"; +import { _setConfigDir, setProfile } from "../../lib/config.ts"; import { type CliError, EXIT_CODE } from "../../lib/errors.ts"; import * as plapi from "../../lib/plapi.ts"; -import { credentialStoreStubs, useCaptureLog } from "../../test/lib/stubs.ts"; +import { + captureTelemetryPayload, + credentialStoreStubs, + useCaptureLog, +} from "../../test/lib/stubs.ts"; // Every test below names its own `--secret-key`, which short-circuits the // signed-in check — except the one that asserts what happens without it. @@ -3113,6 +3117,116 @@ describe("run", () => { expect(requests).toHaveLength(0); }); }); + + describe("telemetry", () => { + const send = async (options: Parameters[0]) => + (await captureTelemetryPayload("migrate import", () => run(options), { captureError: true })) + .payload; + + test("a finished import ends at done, naming its source and instance", async () => { + const payload = await send(baseOptions); + expect(payload).toMatchObject({ + outcome: "success", + stage: "done", + migrate_source: "clerk", + instance_id: "ins_1", + }); + }); + + test("a key-only import borrows the app of the linked profile that owns its instance", async () => { + await setProfile(workDir, { + workspaceId: "ws_1", + appId: "app_1", + instances: { development: "ins_1" }, + }); + const payload = await send(baseOptions); + expect(payload).toMatchObject({ + app_id: "app_1", + workspace_id: "ws_1", + instance_id: "ins_1", + }); + }); + + // A stand-in is a hash of the secret key. + test("a stand-in instance is never sent", async () => { + stubClerk({ instanceDown: true }); + const payload = await send(baseOptions); + expect(payload.instance_id).toBeNull(); + expect(JSON.stringify(payload)).not.toContain(keyInstanceId("sk_test_x")); + }); + + test("some users failing is a partial import, coded", async () => { + stubClerk({ failing: new Set(["u2"]) }); + const payload = await send(baseOptions); + expect(payload).toMatchObject({ + outcome: "error", + stage: "done", + error_code: "migrate_import_partial", + }); + }); + + test.each([ + ["a dry run", { dryRun: true }, "migrate_dry_run"], + ["no consent where nobody can be asked", { yes: false }, "migrate_confirm"], + ["an unknown source", { source: "nope" }, "migrate_source"], + ])("%s stops at its stage", async (_label, extra, stage) => { + const payload = await send({ ...baseOptions, ...extra }); + expect(payload.stage).toBe(stage); + }); + + test("a dry run the checks would refuse is coded", async () => { + fs.writeFileSync( + path.join(workDir, "export.json"), + JSON.stringify([...export2, { id: "u3" }]), + ); + const payload = await send({ ...baseOptions, dryRun: true }); + expect(payload).toMatchObject({ + outcome: "error", + stage: "migrate_dry_run", + error_code: "migrate_checks_rejected", + }); + }); + + test("an import already done stops at migrate_already_imported", async () => { + await run(baseOptions); + const payload = await send(baseOptions); + expect(payload.stage).toBe("migrate_already_imported"); + }); + + test("a file with no users stops at migrate_nothing_to_import", async () => { + const payload = await send({ ...baseOptions, input: "users.json" }); + expect(payload.stage).toBe("migrate_nothing_to_import"); + }); + + test("an envelope names the source when no flag does", async () => { + const file = path.join(workDir, "envelope.json"); + fs.writeFileSync(file, JSON.stringify({ clerkMigrate: 1, source: "auth0", users: [] })); + const { source: _source, ...noSource } = baseOptions; + const payload = await send({ ...noSource, input: "envelope.json" }); + expect(payload.migrate_source).toBe("auth0"); + }); + + test("a source you wrote is custom, never its path or name", async () => { + const own = `./own-telemetry-${Date.now()}.ts`; + fs.writeFileSync( + path.join(workDir, own), + `export default { + key: "secretname", + label: "Secret", + transformer: { id: "userId", primary_email_address: "email" }, + carries: { + passwords: { level: "no", note: "None." }, + mfa: { level: "no", note: "None." }, + metadata: { level: "no", note: "None." }, + }, + };`, + ); + const payload = await send({ ...baseOptions, source: own }); + expect(payload.migrate_source).toBe("custom"); + expect(JSON.stringify(payload)).not.toContain("secretname"); + expect(JSON.stringify(payload)).not.toContain("own-telemetry"); + }); + }); }); describe("explainErrors", () => { diff --git a/packages/cli-core/src/commands/migrate/run.ts b/packages/cli-core/src/commands/migrate/run.ts index 97f16a7fc..7cb522091 100644 --- a/packages/cli-core/src/commands/migrate/run.ts +++ b/packages/cli-core/src/commands/migrate/run.ts @@ -106,6 +106,8 @@ import { import { getSource, resolveSource, sourceKeys } from "./sources/registry.ts"; import type { ImportSummary, User } from "./types.ts"; import { promptForFile, promptForFirebaseHashConfig, promptForSource } from "./wizard.ts"; +import { recordMigrateSource, recordMigrateTarget } from "./telemetry.ts"; +import { declareSoftExitOutcome, setTelemetryStage } from "../../lib/telemetry.ts"; import { login } from "../auth/login.ts"; import { link } from "../link/index.ts"; @@ -912,9 +914,13 @@ type ImportLock = { release?: () => void }; async function runImport(rawOptions: MigrateRunOptions, lock: ImportLock): Promise { // A mistyped --source fails before any sign-in or link. + setTelemetryStage("migrate_source"); let options = await applySource(rawOptions); + recordMigrateSource(options.source, { custom: Boolean(options.sourceHash) }); + setTelemetryStage("migrate_target_setup"); await ensureImportTarget(options); + setTelemetryStage("migrate_input"); const input = await resolveInput(options); options = { ...options, file: input.file }; // Hashed before the envelope is read: the envelope and the users are read @@ -936,6 +942,8 @@ async function runImport(rawOptions: MigrateRunOptions, lock: ImportLock): Promi if (!options.source && canPrompt(options) && fileExists(input.file)) { options = { ...options, source: await promptForSource() }; } + // The file's own word, or the answer at the prompt. + recordMigrateSource(options.source, { custom: Boolean(options.sourceHash) }); const { source, file } = validateRunOptions(options); // The flags every suggested command needs to reach this runs folder and instance. @@ -989,7 +997,10 @@ async function runImport(rawOptions: MigrateRunOptions, lock: ImportLock): Promi await withGutter( "Migrating users to Clerk", async ({ setNextSteps }) => { + setTelemetryStage("migrate_target"); const { secretKey, target, maxAllowedUsers } = await resolveClerkTarget(options); + recordMigrateTarget(target); + setTelemetryStage("migrate_resume"); if (!options.json) printTarget(target); // Every user is already here, so each would be refused as a duplicate. if (input.exportInstanceId && input.exportInstanceId === target.instanceId) { @@ -1061,6 +1072,7 @@ async function runImport(rawOptions: MigrateRunOptions, lock: ImportLock): Promi } if (resume.kind === "complete") { + setTelemetryStage("migrate_already_imported"); // Imported by path and named now by its export run ID: the record // links the export, so the cleanup names the folder holding the data. const record = @@ -1151,6 +1163,7 @@ async function runImport(rawOptions: MigrateRunOptions, lock: ImportLock): Promi } } + setTelemetryStage("migrate_load"); const loaded = await withSpinner(`Loading users from ${file}...`, async () => loadUsersFromFile(file, source, { context: { firebaseHashConfig } }), ); @@ -1189,6 +1202,7 @@ async function runImport(rawOptions: MigrateRunOptions, lock: ImportLock): Promi throwChanged(filePath); } + setTelemetryStage("migrate_checks"); const [settings, existingUsers] = await withSpinner("Checking the instance...", async () => Promise.all([ fetchInstanceSettings(secretKey), @@ -1310,13 +1324,17 @@ async function runImport(rawOptions: MigrateRunOptions, lock: ImportLock): Promi if (!options.json) printChecks(checks); if (options.dryRun) { + setTelemetryStage("migrate_dry_run"); if (options.json) preview({ dryRun: true }); else { log.blank(); log.info(dim("Dry run: nothing was written.")); } // The exit code says what the real run would do. - if (refused) process.exitCode = 2; + if (refused) { + process.exitCode = 2; + declareSoftExitOutcome("error", "migrate_checks_rejected"); + } return; } @@ -1395,6 +1413,7 @@ async function runImport(rawOptions: MigrateRunOptions, lock: ImportLock): Promi withoutPassword.length === 0 && attachOnly.length === 0 ) { + setTelemetryStage("migrate_nothing_to_import"); // Settled, so a continued run is finished rather than left interrupted, // and the JSON reports it as finished, as run.json now does. // Continued by export run ID, the run records the export it now reads. @@ -1430,6 +1449,7 @@ async function runImport(rawOptions: MigrateRunOptions, lock: ImportLock): Promi attachOnly.length > 0 ? `, attach identifiers to ${plural(attachOnly.length, "user")}` : ""; const skipping = checks.rejects.length + checks.skipped.length + withoutPassword.length; const skips = skipping > 0 ? ` and skip ${plural(skipping, "user")}` : ""; + setTelemetryStage("migrate_confirm"); if (!options.yes) { if (!canPrompt(options)) { if (options.json) preview({ consent: "required" }); @@ -1494,6 +1514,7 @@ async function runImport(rawOptions: MigrateRunOptions, lock: ImportLock): Promi }); } + setTelemetryStage("migrate_import"); const summary: ImportSummary = checks.importable.length > 0 || attachOnly.length > 0 ? await withProgress( @@ -1532,7 +1553,11 @@ async function runImport(rawOptions: MigrateRunOptions, lock: ImportLock): Promi throwUserAbort(); } const record = run.finish({ notSent: summary.notSent }); - if (summary.failed > 0) process.exitCode = 1; + setTelemetryStage("done"); + if (summary.failed > 0) { + process.exitCode = 1; + declareSoftExitOutcome("error", "migrate_import_partial"); + } if (options.json) { log.data( diff --git a/packages/cli-core/src/commands/migrate/runs.ts b/packages/cli-core/src/commands/migrate/runs.ts index 427c7fe93..d8b68dc28 100644 --- a/packages/cli-core/src/commands/migrate/runs.ts +++ b/packages/cli-core/src/commands/migrate/runs.ts @@ -9,6 +9,7 @@ import path from "node:path"; import { bold, dim, green, red, yellow } from "../../lib/color.ts"; import { throwUsageError } from "../../lib/errors.ts"; +import { recordMigrateSource, recordMigrateTarget } from "./telemetry.ts"; import { quoteArg } from "../../lib/json-body.ts"; import { log } from "../../lib/log.ts"; import { tallyError } from "./lib/failures.ts"; @@ -314,6 +315,8 @@ export async function runs(id: string | undefined, options: RunsOptions = {}): P const stored = readRun(runsDir, id); if (!stored) throwUsageError(noRunMessage(runsDir, id, carryFlags({ runsDir: options.runsDir }))); const record = withLiveCounts(runsDir, stored); + recordMigrateSource(record.source, { custom: Boolean(record.sourceHash) }); + recordMigrateTarget(record.target); // The record says where the import's key came from, and so which flags // reach that instance again. diff --git a/packages/cli-core/src/commands/migrate/telemetry.ts b/packages/cli-core/src/commands/migrate/telemetry.ts new file mode 100644 index 000000000..929921a60 --- /dev/null +++ b/packages/cli-core/src/commands/migrate/telemetry.ts @@ -0,0 +1,33 @@ +import { + MIGRATE_TELEMETRY_SOURCES, + setTelemetryMigrateSource, + setTelemetryTarget, + type MigrateTelemetrySource, +} from "../../lib/telemetry.ts"; +import type { RunTarget } from "./lib/run-store.ts"; + +/** + * Record the Clerk app and instance a migrate command acted on. A `key_` + * stand-in or an alias never reaches the event: `setTelemetryTarget` drops + * anything that is not an `app_`/`ins_` ID. + */ +export function recordMigrateTarget(target: Pick): void { + setTelemetryTarget({ appId: target.appId, instanceId: target.instanceId }); +} + +/** + * Record where the users come from. A source someone wrote is `custom`, as is + * any key this CLI does not ship: its name is theirs, and may say more than a + * platform does. + */ +export function recordMigrateSource( + key: string | undefined, + options: { custom?: boolean } = {}, +): void { + if (!key) return; + setTelemetryMigrateSource(!options.custom && isBuiltInSource(key) ? key : "custom"); +} + +function isBuiltInSource(key: string): key is MigrateTelemetrySource { + return key !== "custom" && (MIGRATE_TELEMETRY_SOURCES as readonly string[]).includes(key); +} diff --git a/packages/cli-core/src/commands/migrate/undo.test.ts b/packages/cli-core/src/commands/migrate/undo.test.ts index 8a40ac858..be2bed9da 100644 --- a/packages/cli-core/src/commands/migrate/undo.test.ts +++ b/packages/cli-core/src/commands/migrate/undo.test.ts @@ -14,7 +14,7 @@ import path from "node:path"; import { EXIT_CODE, type CliError } from "../../lib/errors.ts"; import { quoteArg } from "../../lib/json-body.ts"; import { _resetInterruptState, abortInFlight, beginInterrupt } from "../../lib/signals.ts"; -import { useCaptureLog } from "../../test/lib/stubs.ts"; +import { captureTelemetryPayload, useCaptureLog } from "../../test/lib/stubs.ts"; import { continueRun, latestUserLines, @@ -795,3 +795,52 @@ describe("deleting", () => { }); }); }); + +describe("telemetry", () => { + const send = async (runId: string, options: Parameters[1]) => + ( + await captureTelemetryPayload("migrate undo", () => undo(runId, options), { + captureError: true, + }) + ).payload; + + test("a finished undo ends at done, naming the import's source and instance", async () => { + const payload = await send(importRun().id, withDir({ yes: true })); + expect(payload).toMatchObject({ + outcome: "success", + stage: "done", + migrate_source: "clerk", + instance_id: "ins_1", + }); + }); + + test("an undo with deletes left is coded", async () => { + failing.add("user_a"); + const payload = await send(importRun().id, withDir({ yes: true })); + expect(payload).toMatchObject({ + outcome: "error", + stage: "done", + error_code: "migrate_undo_incomplete", + }); + }); + + test.each([ + ["a dry run", { dryRun: true }, "migrate_dry_run"], + ["no consent where nobody can be asked", {}, "migrate_confirm"], + ])("%s stops at its stage", async (_label, extra, stage) => { + const payload = await send(importRun().id, withDir(extra)); + expect(payload.stage).toBe(stage); + }); + + test("an unknown run stops at migrate_run_lookup", async () => { + const payload = await send("20260101-000000-nope", withDir({ yes: true })); + expect(payload.stage).toBe("migrate_run_lookup"); + }); + + // Recorded before resolution, so a failed one still names the run's instance. + test("a target that fails to resolve still names the run's instance", async () => { + const record = importRun({ target: { instanceId: "ins_recorded", env: "development" } }); + const payload = await send(record.id, withDir({ yes: true, instance: "prod" })); + expect(payload).toMatchObject({ stage: "migrate_target", instance_id: "ins_recorded" }); + }); +}); diff --git a/packages/cli-core/src/commands/migrate/undo.ts b/packages/cli-core/src/commands/migrate/undo.ts index 45f172111..4b5587d75 100644 --- a/packages/cli-core/src/commands/migrate/undo.ts +++ b/packages/cli-core/src/commands/migrate/undo.ts @@ -22,6 +22,8 @@ import { BapiError, CliError, throwUsageError, throwUserAbort } from "../../lib/ import { log } from "../../lib/log.ts"; import { interruptedExitCode } from "../../lib/signals.ts"; import { confirm } from "../../lib/prompts.ts"; +import { declareSoftExitOutcome, setTelemetryStage } from "../../lib/telemetry.ts"; +import { recordMigrateSource, recordMigrateTarget } from "./telemetry.ts"; import { withSpinner, type SpinnerControls } from "../../lib/spinner.ts"; import { failureOf, tallyError } from "./lib/failures.ts"; import { carryFlags, runsCommand, undoCommand } from "./lib/carry-flags.ts"; @@ -332,14 +334,20 @@ type UndoOutcome = export async function undo(runId: string, options: UndoOptions = {}): Promise { const runsDir = await resolveRunsDir(options.runsDir); const runsCarry = carryFlags({ runsDir: options.runsDir }); + setTelemetryStage("migrate_run_lookup"); const record = readImportRun(runsDir, runId, runsCarry); + recordMigrateSource(record.source, { custom: Boolean(record.sourceHash) }); + // Before resolution, so a failed one still names the run's instance. + recordMigrateTarget(record.target); const importLines = readUserLines(runsDir, record.id).length; // The flags that found this run and instance, for a refusal's command, run // again straight away in the same shell. const carry = carryFlags(options); const undoAgain = undoCommand(record.id, carry); + setTelemetryStage("migrate_target"); const { secretKey, target } = await resolveClerkTarget(options); + recordMigrateTarget(target); // After a partial undo, run later, maybe without the key exported: from the target. const undoLater = undoCommand(record.id, laterCarry(options.runsDir, target)); if (!options.json) printTarget(target); @@ -360,6 +368,7 @@ export async function undo(runId: string, options: UndoOptions = {}): Promise 0 ? await withSpinner("Checking the imported users...", async (spinner) => @@ -399,6 +408,7 @@ export async function undo(runId: string, options: UndoOptions = {}): Promise 0 ? await withProgress({ total: present.length, verb: "deleted" }, async (progress) => @@ -575,6 +587,7 @@ export async function undo(runId: string, options: UndoOptions = {}): Promise { "in_screen", "in_tmux", "install_method", + "instance_id", "machine_uuid", + "migrate_source", "mode", "os", "outcome", @@ -815,6 +824,100 @@ describe("finalizeAndSendTelemetry", () => { expect(payload.pause_step).toBeNull(); expect(payload.components).toEqual({ dns: null, ssl: null, mail: null, oauth: null }); }); + + test("instance_id and migrate_source are present and null on a command that never sets them", async () => { + const payload = await sendAndCapturePayload(() => {}, { outcome: "success", exitCode: 0 }); + expect(payload.instance_id).toBeNull(); + expect(payload.migrate_source).toBeNull(); + }); + }); + + describe("target and migrate source", () => { + const linked = { + workspaceId: "ws_linked", + appId: "app_linked", + instances: { development: "ins_linked_dev", production: "ins_linked_prod" }, + }; + const ok = { outcome: "success", exitCode: 0 } as const; + + test("a command that sets no target reports the linked profile, as before", async () => { + await setProfile(process.cwd(), linked); + const payload = await sendAndCapturePayload(() => {}, ok); + expect(payload.app_id).toBe("app_linked"); + expect(payload.workspace_id).toBe("ws_linked"); + expect(payload.instance_id).toBeNull(); + }); + + test("the target's app overrides the linked profile, and drops its workspace", async () => { + await setProfile(process.cwd(), linked); + const payload = await sendAndCapturePayload( + () => setTelemetryTarget({ appId: "app_other", instanceId: "ins_other" }), + ok, + ); + expect(payload.app_id).toBe("app_other"); + expect(payload.instance_id).toBe("ins_other"); + expect(payload.workspace_id).toBeNull(); + }); + + test("a key-only target borrows the profile's app when the profile owns the instance", async () => { + await setProfile(process.cwd(), linked); + const payload = await sendAndCapturePayload( + () => setTelemetryTarget({ instanceId: "ins_linked_prod" }), + ok, + ); + expect(payload.app_id).toBe("app_linked"); + expect(payload.workspace_id).toBe("ws_linked"); + expect(payload.instance_id).toBe("ins_linked_prod"); + }); + + test("a key-only target for another instance names no app", async () => { + await setProfile(process.cwd(), linked); + const payload = await sendAndCapturePayload( + () => setTelemetryTarget({ instanceId: "ins_elsewhere" }), + ok, + ); + expect(payload.app_id).toBeNull(); + expect(payload.workspace_id).toBeNull(); + expect(payload.instance_id).toBe("ins_elsewhere"); + }); + + // A `key_` stand-in is a hash of the secret key; an alias or label is not an ID. + test("drops anything that is not an app_ or ins_ ID", async () => { + const payload = await sendAndCapturePayload( + () => setTelemetryTarget({ appId: "My App", instanceId: "key_0123456789abcdef" }), + ok, + ); + expect(payload.app_id).toBeNull(); + expect(payload.instance_id).toBeNull(); + expect(JSON.stringify(payload)).not.toContain("key_0123456789abcdef"); + }); + + test("an alias-only hint counts as no target", async () => { + await setProfile(process.cwd(), linked); + const payload = await sendAndCapturePayload( + () => setTelemetryTarget({ instanceId: "dev" }), + ok, + ); + expect(payload.app_id).toBe("app_linked"); + expect(payload.instance_id).toBeNull(); + }); + + test("a later target replaces an earlier one whole", async () => { + const payload = await sendAndCapturePayload(() => { + setTelemetryTarget({ appId: "app_hint", instanceId: "ins_hint" }); + setTelemetryTarget({ instanceId: "ins_resolved" }); + }, ok); + expect(payload.app_id).toBeNull(); + expect(payload.instance_id).toBe("ins_resolved"); + }); + + test("the last migrate source set is the one sent", async () => { + const payload = await sendAndCapturePayload(() => { + setTelemetryMigrateSource("auth0"); + setTelemetryMigrateSource("custom"); + }, ok); + expect(payload.migrate_source).toBe("custom"); + }); }); describe("pause step", () => { diff --git a/packages/cli-core/src/lib/telemetry.ts b/packages/cli-core/src/lib/telemetry.ts index 12fa386d7..180b0a3f5 100644 --- a/packages/cli-core/src/lib/telemetry.ts +++ b/packages/cli-core/src/lib/telemetry.ts @@ -111,9 +111,54 @@ export type TelemetryStage = | "domain_pending" | "oauth_pending" | "complete" + // `clerk migrate import`. `already_imported`, `dry_run` and + // `nothing_to_import` are terminal branches; a partial import still ends at + // `done`, and its `outcome` says so. + | "migrate_source" + | "migrate_target_setup" + | "migrate_input" + | "migrate_target" + | "migrate_resume" + | "migrate_already_imported" + | "migrate_load" + | "migrate_checks" + | "migrate_dry_run" + | "migrate_nothing_to_import" + | "migrate_confirm" + | "migrate_import" + // `clerk migrate export [platform]`. `pick_platform` only on the bare picker. + | "migrate_pick_platform" + | "migrate_export_read" + | "migrate_export_write" + // `clerk migrate undo`, which shares `migrate_target`, `migrate_dry_run` and + // `migrate_confirm` with import. + | "migrate_run_lookup" + | "migrate_inspect" + | "migrate_delete" // shared terminal marker | "done"; +/** + * Where a migration's users come from: a built-in source or export platform, + * or `custom` for a source someone wrote. Closed like {@link TelemetryStage}, + * so a path to a custom source can never reach the payload. + */ +export const MIGRATE_TELEMETRY_SOURCES = [ + "auth0", + "authjs", + "betterauth", + "clerk", + "firebase", + "supabase", + "workos", + "custom", +] as const; + +export type MigrateTelemetrySource = (typeof MIGRATE_TELEMETRY_SOURCES)[number]; + +/** The Clerk app and instance a command acted on. Only real `app_`/`ins_` IDs are kept. */ +type TelemetryTarget = { appId?: string; instanceId?: string }; + /** Structural slice of Commander's Command — avoids its generic types. */ export type TelemetryCommand = { name(): string; @@ -134,6 +179,10 @@ type TelemetryContext = { pauseStep: TelemetryPauseStep | null; /** Per-component readiness, each field written only by an observation. */ components: TelemetryComponents; + /** What the command acted on — see setTelemetryTarget. Empty when nothing said. */ + target: TelemetryTarget; + /** Where a migration's users come from — see setTelemetryMigrateSource. */ + migrateSource: MigrateTelemetrySource | null; }; /** What a command wants recorded for its soft exit. `deploy status` declares no code: nothing was thrown. */ @@ -246,6 +295,8 @@ export function startCommandTelemetry(actionCommand: TelemetryCommand): void { softExit: null, pauseStep: null, components: emptyComponents(), + target: {}, + migrateSource: null, }; } catch (error) { log.debug(`telemetry: failed to start context: ${error}`); @@ -293,6 +344,53 @@ export function setTelemetryOAuthComplete(complete: boolean): void { if (context) context.components = { ...context.components, oauth: complete }; } +/** + * The Clerk app and instance the command acted on. Overrides the linked + * profile's app. Each call replaces the last, so a resolved target replaces an + * early hint from the flags. Anything that is not an `app_`/`ins_` ID — an + * `--instance dev` alias, an app label, a `key_` stand-in hashed from a secret + * key — is dropped here, so no call site can send one. + */ +export function setTelemetryTarget(target: { appId?: string; instanceId?: string }): void { + if (!context) return; + context.target = { + ...(target.appId?.startsWith("app_") ? { appId: target.appId } : {}), + ...(target.instanceId?.startsWith("ins_") ? { instanceId: target.instanceId } : {}), + }; +} + +/** Record where a migration's users come from. Last call wins. */ +export function setTelemetryMigrateSource(source: MigrateTelemetrySource): void { + if (context) context.migrateSource = source; +} + +/** + * The app and workspace an event names. A command's own target wins; a + * key-only target borrows the linked profile's app when the profile owns its + * instance; a command that set no target reports the linked profile as + * before. The workspace is the profile's only when the app is too, so an + * event never pairs one app's workspace with another app's ID. + */ +function resolveEventApp( + target: TelemetryTarget, + profile: + | { appId: string; workspaceId: string; instances: Record } + | undefined, +): { appId: string | null; workspaceId: string | null } { + const targetSet = target.appId !== undefined || target.instanceId !== undefined; + const profileOwnsInstance = + target.instanceId !== undefined && + profile !== undefined && + Object.values(profile.instances).includes(target.instanceId); + const appId = + target.appId ?? + (profileOwnsInstance ? profile.appId : undefined) ?? + (targetSet ? undefined : profile?.appId) ?? + null; + const workspaceId = appId !== null && appId === profile?.appId ? profile.workspaceId : null; + return { appId, workspaceId }; +} + /** * Declare how a run that exits nonzero without throwing should be recorded. * Last call wins; ignored if the run throws or exits 0. Declare it under the @@ -427,7 +525,7 @@ async function buildAndSend( // The notice tells the user "Nothing has been sent during this run" — honor it. if (await showTelemetryNoticeOnce()) return; - const envelope = await buildTelemetryEnvelope(); + const envelope = await buildTelemetryEnvelope(current.target); // The internal post step, not the exported sender: the opt-out was read // just above, and this whole job runs under a one-second deadline. @@ -445,6 +543,9 @@ async function buildAndSend( pause_step: current.pauseStep, components: current.components, duration_ms: Date.now() - current.startedAt, + // Migrate's; null on other commands. + instance_id: current.target.instanceId ?? null, + migrate_source: current.migrateSource, ...envelope, }, }, @@ -488,10 +589,14 @@ export type TelemetryEnvelopePreview = Omit & /** * Read the envelope fields. Creates the machine id on a machine that has * none, so a caller that only previews an event still writes the config file; - * a preview that must not do that uses {@link peekTelemetryEnvelope}. + * a preview that must not do that uses {@link peekTelemetryEnvelope}. The + * per-run usage event passes the command's target, which decides `app_id` and + * `workspace_id`; without one they are the linked project's. */ -export async function buildTelemetryEnvelope(): Promise { - return envelopeWith(await ensureMachineUuid()); +export async function buildTelemetryEnvelope( + target: TelemetryTarget = {}, +): Promise { + return envelopeWith(await ensureMachineUuid(), target); } /** @@ -506,8 +611,10 @@ export async function peekTelemetryEnvelope(): Promise async function envelopeWith( machineUuid: Id, + target: TelemetryTarget = {}, ): Promise & { machine_uuid: Id }> { const resolved = await resolveProfile(process.cwd()).catch(() => undefined); + const app = resolveEventApp(target, resolved?.profile); return { machine_uuid: machineUuid, install_method: detectInstallMethod(process.env, process.execPath), @@ -520,8 +627,8 @@ async function envelopeWith( in_tmux: detectInTmux(process.env), in_screen: detectInScreen(process.env), env: getCurrentEnvName(), - workspace_id: resolved?.profile.workspaceId ?? null, - app_id: resolved?.profile.appId ?? null, + workspace_id: app.workspaceId, + app_id: app.appId, }; } @@ -617,9 +724,12 @@ export async function showTelemetryNoticeOnce( "The Clerk CLI collects usage telemetry to help improve the CLI: command name, flag names,", ); log.info( - "duration, outcome, the step a multi-step command reached, a random machine identifier —", + "duration, outcome, the step a multi-step command reached, a random machine identifier,", + ); + log.info( + "the platform a migration's users come from, and the workspace, app and instance IDs of the", ); - log.info("and your workspace and app IDs when a project is linked."); + log.info("Clerk app a command acts on or the project is linked to."); log.info(lastLine); log.info("Opt out: `clerk telemetry disable` — details: https://clerk.com/docs/telemetry"); log.blank(); From 294bba5450b0689ad3130a34f238df3fe6d77192 Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Wed, 28 Oct 2026 17:30:11 -0400 Subject: [PATCH 02/14] test(migrate): cover telemetry for runs , a prompted source, and a declined import Co-Authored-By: Claude Opus 5.5 --- .../commands/migrate/run-interactive.test.ts | 20 +++++++++++++++- .../src/commands/migrate/runs.test.ts | 24 ++++++++++++++++++- 2 files changed, 42 insertions(+), 2 deletions(-) diff --git a/packages/cli-core/src/commands/migrate/run-interactive.test.ts b/packages/cli-core/src/commands/migrate/run-interactive.test.ts index 1c5cd18ea..8409ef19e 100644 --- a/packages/cli-core/src/commands/migrate/run-interactive.test.ts +++ b/packages/cli-core/src/commands/migrate/run-interactive.test.ts @@ -22,7 +22,7 @@ import { import fs from "node:fs"; import os from "node:os"; import path from "node:path"; -import { listageStubs, useCaptureLog } from "../../test/lib/stubs.ts"; +import { captureTelemetryPayload, listageStubs, useCaptureLog } from "../../test/lib/stubs.ts"; const mockSelect = mock(async () => "clerk" as unknown); const mockText = mock(async () => "export.json" as unknown); @@ -513,3 +513,21 @@ describe("unverified identifiers", () => { expect(statuses()).toEqual([undefined, ["verified", "reserved"]]); }); }); + +describe("telemetry", () => { + test("a source picked at the prompt is recorded", async () => { + const { payload } = await captureTelemetryPayload("migrate import", () => + run({ input: "export.json", secretKey: "sk_test_x" }), + ); + expect(mockSelect).toHaveBeenCalledTimes(1); + expect(payload).toMatchObject({ migrate_source: "clerk", stage: "done" }); + }); + + test("declining at the confirm prompt is an abort at migrate_confirm", async () => { + confirmAnswer = false; + const { payload } = await captureTelemetryPayload("migrate import", () => run(importOptions), { + captureError: true, + }); + expect(payload).toMatchObject({ outcome: "abort", stage: "migrate_confirm" }); + }); +}); diff --git a/packages/cli-core/src/commands/migrate/runs.test.ts b/packages/cli-core/src/commands/migrate/runs.test.ts index d9b541333..e9be4cd45 100644 --- a/packages/cli-core/src/commands/migrate/runs.test.ts +++ b/packages/cli-core/src/commands/migrate/runs.test.ts @@ -3,7 +3,7 @@ import fs from "node:fs"; import os from "node:os"; import path from "node:path"; import { quoteArg } from "../../lib/json-body.ts"; -import { useCaptureLog } from "../../test/lib/stubs.ts"; +import { captureTelemetryPayload, useCaptureLog } from "../../test/lib/stubs.ts"; import { continueRun, type RunRecord, @@ -427,3 +427,25 @@ describe("runs ", () => { ); }); }); + +describe("telemetry", () => { + test("showing one run records its source and instance", async () => { + const record = partialImport({ target: { ...TARGET, appId: "app_1" } }); + const { payload } = await captureTelemetryPayload("migrate runs", () => + runs(record.id, { runsDir }), + ); + expect(payload).toMatchObject({ + migrate_source: "clerk", + instance_id: "ins_1", + app_id: "app_1", + }); + }); + + test("a run imported through a source you wrote records custom", async () => { + const record = partialImport({ sourceHash: "abc123" }); + const { payload } = await captureTelemetryPayload("migrate runs", () => + runs(record.id, { runsDir }), + ); + expect(payload.migrate_source).toBe("custom"); + }); +}); From 40764075ec7f0bbb5abe19131c26ef82be5eec12 Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Wed, 28 Oct 2026 18:53:47 -0400 Subject: [PATCH 03/14] docs(readme): name the migrate values telemetry takes from options Co-Authored-By: Claude Opus 5.5 --- README.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index a6eb2527a..5b6c7e173 100644 --- a/README.md +++ b/README.md @@ -104,10 +104,13 @@ Commands: The Clerk CLI collects usage telemetry: command name, flag names, duration, outcome, environment signals (OS, install method, terminal), a random machine identifier, the platform a migration's users come from, and the workspace, app and instance IDs of the -Clerk app a command acts on or the project is linked to. It never collects command -arguments, option values, file paths, or personal data, except the text you choose to -send with `clerk feedback`, which is scrubbed on a best-effort basis for common secret -formats, home-directory paths and email addresses before it leaves your machine; +Clerk app a command acts on or the project is linked to. It doesn't collect raw command +arguments, option values, file paths, or personal data, with two exceptions. `clerk migrate` +reports where a migration's users come from as a built-in platform name or `custom`, and +the app and instance IDs it acts on, which can come from `--app` and `--instance`. +`clerk feedback` sends the text you choose to send, which is scrubbed on a best-effort +basis for common secret formats, home-directory paths and email addresses before it +leaves your machine; `clerk feedback --dry-run` shows exactly what would be sent. The first run only shows a disclosure notice and sends nothing (CI environments send from the first run), except `clerk feedback`, which shows the notice and then sends the report you asked it to send From eced172e7dcc319ecf13f5686738c8b2ac8f9c5c Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Fri, 30 Oct 2026 15:58:30 -0400 Subject: [PATCH 04/14] fix(telemetry): send a null app for a key_ stand-in, not the linked one When `GET /v1/instance` failed on an import keyed by `--secret-key` or `CLERK_SECRET_KEY`, `setTelemetryTarget` dropped the `key_` stand-in, the target ended up empty, and the event fell back to the linked profile's app and workspace: an app the run may never have touched. A stand-in now counts as a target that's set but unnamed, so `app_id` and `workspace_id` go out as null, as the README says. An `--instance` alias still reports the linked app. Co-Authored-By: Claude Opus 5.5 --- packages/cli-core/src/lib/telemetry.test.ts | 12 ++++++++++++ packages/cli-core/src/lib/telemetry.ts | 16 ++++++++++++---- 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/packages/cli-core/src/lib/telemetry.test.ts b/packages/cli-core/src/lib/telemetry.test.ts index 8a8f65722..91a8c81a6 100644 --- a/packages/cli-core/src/lib/telemetry.test.ts +++ b/packages/cli-core/src/lib/telemetry.test.ts @@ -892,6 +892,18 @@ describe("finalizeAndSendTelemetry", () => { expect(JSON.stringify(payload)).not.toContain("key_0123456789abcdef"); }); + // A key whose instance Clerk didn't confirm may be any app's, not the linked one. + test("a key_ stand-in counts as a target, not the linked profile", async () => { + await setProfile(process.cwd(), linked); + const payload = await sendAndCapturePayload( + () => setTelemetryTarget({ instanceId: "key_0123456789abcdef" }), + ok, + ); + expect(payload.app_id).toBeNull(); + expect(payload.workspace_id).toBeNull(); + expect(payload.instance_id).toBeNull(); + }); + test("an alias-only hint counts as no target", async () => { await setProfile(process.cwd(), linked); const payload = await sendAndCapturePayload( diff --git a/packages/cli-core/src/lib/telemetry.ts b/packages/cli-core/src/lib/telemetry.ts index 180b0a3f5..2d22a28f9 100644 --- a/packages/cli-core/src/lib/telemetry.ts +++ b/packages/cli-core/src/lib/telemetry.ts @@ -157,7 +157,12 @@ export const MIGRATE_TELEMETRY_SOURCES = [ export type MigrateTelemetrySource = (typeof MIGRATE_TELEMETRY_SOURCES)[number]; /** The Clerk app and instance a command acted on. Only real `app_`/`ins_` IDs are kept. */ -type TelemetryTarget = { appId?: string; instanceId?: string }; +type TelemetryTarget = { + appId?: string; + instanceId?: string; + /** A `key_` stand-in: a key Clerk didn't name an instance for, so not the linked app's. */ + standIn?: true; +}; /** Structural slice of Commander's Command — avoids its generic types. */ export type TelemetryCommand = { @@ -356,6 +361,7 @@ export function setTelemetryTarget(target: { appId?: string; instanceId?: string context.target = { ...(target.appId?.startsWith("app_") ? { appId: target.appId } : {}), ...(target.instanceId?.startsWith("ins_") ? { instanceId: target.instanceId } : {}), + ...(target.instanceId?.startsWith("key_") ? { standIn: true as const } : {}), }; } @@ -368,8 +374,9 @@ export function setTelemetryMigrateSource(source: MigrateTelemetrySource): void * The app and workspace an event names. A command's own target wins; a * key-only target borrows the linked profile's app when the profile owns its * instance; a command that set no target reports the linked profile as - * before. The workspace is the profile's only when the app is too, so an - * event never pairs one app's workspace with another app's ID. + * before. A `key_` stand-in is a target, so it never borrows the linked + * profile's app. The workspace is the profile's only when the app is too, so + * an event never pairs one app's workspace with another app's ID. */ function resolveEventApp( target: TelemetryTarget, @@ -377,7 +384,8 @@ function resolveEventApp( | { appId: string; workspaceId: string; instances: Record } | undefined, ): { appId: string | null; workspaceId: string | null } { - const targetSet = target.appId !== undefined || target.instanceId !== undefined; + const targetSet = + target.appId !== undefined || target.instanceId !== undefined || target.standIn === true; const profileOwnsInstance = target.instanceId !== undefined && profile !== undefined && From 1e94d695e1de7a6e030fcbd485db3981c7374d6e Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Fri, 30 Oct 2026 15:59:53 -0400 Subject: [PATCH 05/14] fix(telemetry): check migrate_source against the source registries `migrate_source` was checked against a list in `lib/telemetry.ts` that nothing tied to the registries, so a new built-in source or export platform would have reported as `custom`. `recordMigrateSource` now checks `sourceKeys()` and `exportPlatformKeys()` instead, in `commands/migrate/telemetry.ts` so `lib/telemetry.ts` doesn't load every source on every command. It stays the only caller of `setTelemetryMigrateSource`, so the payload still only sees a shipped source or `custom`. Co-Authored-By: Claude Opus 5.5 --- .../src/commands/migrate/telemetry.test.ts | 30 +++++++++++++++++++ .../src/commands/migrate/telemetry.ts | 15 +++++----- packages/cli-core/src/lib/telemetry.ts | 29 +++++------------- 3 files changed, 45 insertions(+), 29 deletions(-) create mode 100644 packages/cli-core/src/commands/migrate/telemetry.test.ts diff --git a/packages/cli-core/src/commands/migrate/telemetry.test.ts b/packages/cli-core/src/commands/migrate/telemetry.test.ts new file mode 100644 index 000000000..8534576a9 --- /dev/null +++ b/packages/cli-core/src/commands/migrate/telemetry.test.ts @@ -0,0 +1,30 @@ +import { afterAll, afterEach, describe, expect, spyOn, test } from "bun:test"; +import * as telemetry from "../../lib/telemetry.ts"; +import { exportPlatformKeys } from "./export/registry.ts"; +import { sourceKeys } from "./sources/registry.ts"; +import { recordMigrateSource } from "./telemetry.ts"; + +describe("recordMigrateSource", () => { + const setSource = spyOn(telemetry, "setTelemetryMigrateSource"); + afterEach(() => setSource.mockClear()); + afterAll(() => setSource.mockRestore()); + + // Read from the registries, so a new built-in reports itself, not `custom`. + test.each([...new Set([...sourceKeys(), ...exportPlatformKeys()])])( + "reports the built-in %s by name", + (key) => { + recordMigrateSource(key); + expect(setSource).toHaveBeenCalledWith(key); + }, + ); + + test("reports a source someone wrote as custom, whatever its key", () => { + recordMigrateSource("auth0", { custom: true }); + expect(setSource).toHaveBeenCalledWith("custom"); + }); + + test("reports a key this CLI doesn't ship as custom", () => { + recordMigrateSource("okta"); + expect(setSource).toHaveBeenCalledWith("custom"); + }); +}); diff --git a/packages/cli-core/src/commands/migrate/telemetry.ts b/packages/cli-core/src/commands/migrate/telemetry.ts index 929921a60..809444cfc 100644 --- a/packages/cli-core/src/commands/migrate/telemetry.ts +++ b/packages/cli-core/src/commands/migrate/telemetry.ts @@ -1,10 +1,7 @@ -import { - MIGRATE_TELEMETRY_SOURCES, - setTelemetryMigrateSource, - setTelemetryTarget, - type MigrateTelemetrySource, -} from "../../lib/telemetry.ts"; +import { setTelemetryMigrateSource, setTelemetryTarget } from "../../lib/telemetry.ts"; +import { exportPlatformKeys } from "./export/registry.ts"; import type { RunTarget } from "./lib/run-store.ts"; +import { sourceKeys } from "./sources/registry.ts"; /** * Record the Clerk app and instance a migrate command acted on. A `key_` @@ -28,6 +25,8 @@ export function recordMigrateSource( setTelemetryMigrateSource(!options.custom && isBuiltInSource(key) ? key : "custom"); } -function isBuiltInSource(key: string): key is MigrateTelemetrySource { - return key !== "custom" && (MIGRATE_TELEMETRY_SOURCES as readonly string[]).includes(key); +// The registries, not a list of their own: a new built-in source or export +// platform reports itself without anyone remembering to add it here. +function isBuiltInSource(key: string): boolean { + return [...sourceKeys(), ...exportPlatformKeys()].includes(key); } diff --git a/packages/cli-core/src/lib/telemetry.ts b/packages/cli-core/src/lib/telemetry.ts index 2d22a28f9..3f82c8b5d 100644 --- a/packages/cli-core/src/lib/telemetry.ts +++ b/packages/cli-core/src/lib/telemetry.ts @@ -138,24 +138,6 @@ export type TelemetryStage = // shared terminal marker | "done"; -/** - * Where a migration's users come from: a built-in source or export platform, - * or `custom` for a source someone wrote. Closed like {@link TelemetryStage}, - * so a path to a custom source can never reach the payload. - */ -export const MIGRATE_TELEMETRY_SOURCES = [ - "auth0", - "authjs", - "betterauth", - "clerk", - "firebase", - "supabase", - "workos", - "custom", -] as const; - -export type MigrateTelemetrySource = (typeof MIGRATE_TELEMETRY_SOURCES)[number]; - /** The Clerk app and instance a command acted on. Only real `app_`/`ins_` IDs are kept. */ type TelemetryTarget = { appId?: string; @@ -187,7 +169,7 @@ type TelemetryContext = { /** What the command acted on — see setTelemetryTarget. Empty when nothing said. */ target: TelemetryTarget; /** Where a migration's users come from — see setTelemetryMigrateSource. */ - migrateSource: MigrateTelemetrySource | null; + migrateSource: string | null; }; /** What a command wants recorded for its soft exit. `deploy status` declares no code: nothing was thrown. */ @@ -365,8 +347,13 @@ export function setTelemetryTarget(target: { appId?: string; instanceId?: string }; } -/** Record where a migration's users come from. Last call wins. */ -export function setTelemetryMigrateSource(source: MigrateTelemetrySource): void { +/** + * Record where a migration's users come from: a built-in source or export + * platform, or `custom`. Last call wins. `recordMigrateSource` in + * `commands/migrate/telemetry.ts` is the only caller, and it checks the key + * against the registries, so a custom source's path or name never gets here. + */ +export function setTelemetryMigrateSource(source: string): void { if (context) context.migrateSource = source; } From 4a64a6dbd4dd5556b073b26980e217b8750ca09c Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Fri, 30 Oct 2026 16:01:48 -0400 Subject: [PATCH 06/14] feat(telemetry): mark clerk migrate's requests in the User-Agent `clerk migrate`'s Backend API requests looked like any other CLI request in the `User-Agent`, so the logs couldn't tell a migration's user creates from `clerk users create`. The migrate group now marks its requests, and the `User-Agent` gains a `migrate` segment. Like `AIAgent`, the segment is for analytics, so the fetch layer leaves it out under the telemetry opt-outs and until the disclosure notice has shown. Co-Authored-By: Claude Opus 5.5 --- .changeset/migrate-user-agent.md | 5 ++++ .../cli-core/src/commands/migrate/README.md | 2 ++ .../cli-core/src/commands/migrate/index.ts | 3 ++ packages/cli-core/src/lib/fetch.test.ts | 28 +++++++++++++++++-- packages/cli-core/src/lib/fetch.ts | 24 ++++++++++++---- packages/cli-core/src/lib/user-agent.test.ts | 5 ++++ packages/cli-core/src/lib/user-agent.ts | 8 ++++-- 7 files changed, 66 insertions(+), 9 deletions(-) create mode 100644 .changeset/migrate-user-agent.md diff --git a/.changeset/migrate-user-agent.md b/.changeset/migrate-user-agent.md new file mode 100644 index 000000000..de29ea9b1 --- /dev/null +++ b/.changeset/migrate-user-agent.md @@ -0,0 +1,5 @@ +--- +"clerk": patch +--- + +Mark `clerk migrate`'s Backend API requests with a `migrate` segment in the `User-Agent`, so Clerk can count the users each migration creates. Like usage telemetry, the CLI leaves it out when telemetry is off. diff --git a/packages/cli-core/src/commands/migrate/README.md b/packages/cli-core/src/commands/migrate/README.md index de23476f2..faa3b4024 100644 --- a/packages/cli-core/src/commands/migrate/README.md +++ b/packages/cli-core/src/commands/migrate/README.md @@ -1540,6 +1540,8 @@ A run that finishes part-way exits nonzero without throwing, and carries a code | `undo` with deletes left | `migrate_undo_incomplete` | 1 | | `export` the platform cut short | `migrate_export_truncated` | 1 | +`clerk migrate` also adds a `migrate` segment to the `User-Agent` of its Backend API requests (`Clerk-CLI/ (…; migrate)`), so Clerk's logs can count the users each migration creates. The CLI follows the same rules as for telemetry: it leaves the segment out when telemetry is off, and until the telemetry notice has shown. + ## API endpoints | Method | Path | Used by | diff --git a/packages/cli-core/src/commands/migrate/index.ts b/packages/cli-core/src/commands/migrate/index.ts index 25c6740f9..45fdb0e7f 100644 --- a/packages/cli-core/src/commands/migrate/index.ts +++ b/packages/cli-core/src/commands/migrate/index.ts @@ -1,5 +1,6 @@ import type { Program } from "../../cli-program.ts"; import { isExperimentEnabled, requireExperiment } from "../../lib/experimental.ts"; +import { markMigrateRequests } from "../../lib/fetch.ts"; import { parseIntegerOption } from "../../lib/option-parsers.ts"; import { setMode } from "../../mode.ts"; import { setAssumeYes } from "./lib/assume-yes.ts"; @@ -69,6 +70,8 @@ export function registerMigrate(program: Program, env: NodeJS.ProcessEnv = proce // With globals: `export auth0 --json` lands on the export group's own // --json, which the subcommand's opts() never sees. const opts = actionCommand.optsWithGlobals(); + // So the Backend API logs can tell a migration's creates from `clerk users create`. + markMigrateRequests(); setAssumeYes(Boolean(opts.yes)); if (opts.json && !opts.mode) setMode("agent"); // A hint, so a run that fails before its target resolves still names it. diff --git a/packages/cli-core/src/lib/fetch.test.ts b/packages/cli-core/src/lib/fetch.test.ts index 505131b9e..3f51b2302 100644 --- a/packages/cli-core/src/lib/fetch.test.ts +++ b/packages/cli-core/src/lib/fetch.test.ts @@ -2,7 +2,7 @@ import { test, expect, describe, afterEach, beforeEach, mock } from "bun:test"; import { mkdtemp, rm } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { _resetUserAgentCache, loggedFetch } from "./fetch.ts"; +import { _resetUserAgentCache, loggedFetch, markMigrateRequests } from "./fetch.ts"; import { _resetInterruptState, abortInFlight, beginInterrupt, interruptSignal } from "./signals.ts"; import { _setConfigDir, markTelemetryNoticeShown, setTelemetryDisabled } from "./config.ts"; @@ -48,7 +48,7 @@ describe("loggedFetch", () => { }); }); -describe("AIAgent segment honors the telemetry opt-out", () => { +describe("AIAgent and migrate segments honor the telemetry opt-out", () => { let configDir: string; let originalClaudecode: string | undefined; let originalCi: string | undefined; @@ -112,6 +112,30 @@ describe("AIAgent segment honors the telemetry opt-out", () => { await setTelemetryDisabled(true); expect(await sentUserAgent()).not.toContain("AIAgent/"); }); + + // Marks `clerk migrate`'s Backend API requests, under the same opt-outs. + test("migrate segment present once clerk migrate marks its requests", async () => { + await markTelemetryNoticeShown(); + markMigrateRequests(); + expect(await sentUserAgent()).toContain("; migrate)"); + }); + + test("migrate segment omitted when nothing marks the requests", async () => { + await markTelemetryNoticeShown(); + expect(await sentUserAgent()).not.toContain("migrate"); + }); + + test("migrate segment omitted under an opt-out", async () => { + await markTelemetryNoticeShown(); + await setTelemetryDisabled(true); + markMigrateRequests(); + expect(await sentUserAgent()).not.toContain("migrate"); + }); + + test("migrate segment omitted before the disclosure notice has been shown", async () => { + markMigrateRequests(); + expect(await sentUserAgent()).not.toContain("migrate"); + }); }); describe("loggedFetch interrupt signal", () => { diff --git a/packages/cli-core/src/lib/fetch.ts b/packages/cli-core/src/lib/fetch.ts index f879acff2..ffb8ef1e2 100644 --- a/packages/cli-core/src/lib/fetch.ts +++ b/packages/cli-core/src/lib/fetch.ts @@ -16,28 +16,42 @@ import { optOutEnvVar } from "./env-signals.ts"; import { getTelemetryDisabled, getTelemetryNoticeShown } from "./config.ts"; let userAgentPromise: Promise | undefined; +let migrateRequests = false; /** Test-only: recompute the User-Agent on the next request. */ export function _resetUserAgentCache(): void { userAgentPromise = undefined; + migrateRequests = false; +} + +/** + * Mark the rest of this process's requests as `clerk migrate`'s: the + * User-Agent gains a `migrate` segment, under the same opt-outs as `AIAgent`. + */ +export function markMigrateRequests(): void { + migrateRequests = true; + userAgentPromise = undefined; } // Not `async`: the body hands back a memoized promise, and wrapping it per call // would allocate a fresh one on every request for no gain. // oxlint-disable-next-line typescript/promise-function-async function resolveUserAgent(): Promise { - // The AIAgent segment exists purely for analytics classification, so it - // honors the telemetry opt-outs (env vars and `clerk telemetry disable`) - // and, like the events, stays absent until the disclosure notice has been + // The AIAgent and migrate segments exist purely for analytics, so they + // honor the telemetry opt-outs (env vars and `clerk telemetry disable`) + // and, like the events, stay absent until the disclosure notice has been // shown — CI exempt, matching showTelemetryNoticeOnce. userAgentPromise ??= (async () => { const beforeDisclosure = !process.env.CI && !(await getTelemetryNoticeShown().catch(() => false)); - const omitAgentSegment = + const omitAnalyticsSegments = optOutEnvVar(process.env) !== null || beforeDisclosure || (await getTelemetryDisabled().catch(() => true)); - return buildUserAgent(process.env, { agentToken: !omitAgentSegment }); + return buildUserAgent(process.env, { + agentToken: !omitAnalyticsSegments, + migrate: migrateRequests && !omitAnalyticsSegments, + }); })(); return userAgentPromise; } diff --git a/packages/cli-core/src/lib/user-agent.test.ts b/packages/cli-core/src/lib/user-agent.test.ts index 63e850760..a19423c40 100644 --- a/packages/cli-core/src/lib/user-agent.test.ts +++ b/packages/cli-core/src/lib/user-agent.test.ts @@ -22,6 +22,11 @@ describe("buildUserAgent", () => { expect(ua).not.toMatch(/; ci\)/); }); + test("appends a migrate segment for clerk migrate's requests", () => { + expect(buildUserAgent({}, { migrate: true })).toMatch(/; migrate\)$/); + expect(buildUserAgent({})).not.toContain("migrate"); + }); + test("uses only printable ASCII characters", () => { expect(buildUserAgent({})).toMatch(/^[\x20-\x7e]+$/); }); diff --git a/packages/cli-core/src/lib/user-agent.ts b/packages/cli-core/src/lib/user-agent.ts index 6bd023112..755f51fa8 100644 --- a/packages/cli-core/src/lib/user-agent.ts +++ b/packages/cli-core/src/lib/user-agent.ts @@ -4,13 +4,16 @@ * we fall through to Bun's default `User-Agent: Bun/`, which is * indistinguishable from any other Bun-based client. * - * Format: `Clerk-CLI/ (Bun/; -[; ci][; AIAgent/])` + * Format: `Clerk-CLI/ (Bun/; -[; ci][; AIAgent/][; migrate])` * - : darwin | linux | win32 | … (process.platform) * - : arm64 | x64 | … (process.arch) * - `ci` segment is appended when running under a recognized CI environment. * - `AIAgent/` segment is appended when an AI agent is detected — * analytics-purposed, so the fetch layer omits it when telemetry is * opted out or not yet disclosed. + * - `migrate` segment is appended to `clerk migrate`'s requests, so the + * Backend API logs can count a migration's user creates. Analytics too, + * so the fetch layer omits it under the same rules as `AIAgent`. */ import { detectAiAgent, type EnvLike } from "./env-signals.ts"; @@ -18,7 +21,7 @@ import { CURRENT_VERSION } from "./version.ts"; export function buildUserAgent( env: EnvLike = process.env, - options: { agentToken?: boolean } = {}, + options: { agentToken?: boolean; migrate?: boolean } = {}, ): string { const segments = [`Bun/${Bun.version}`, `${process.platform}-${process.arch}`]; if (env.CI) segments.push("ci"); @@ -26,5 +29,6 @@ export function buildUserAgent( const agent = detectAiAgent(env); if (agent) segments.push(`AIAgent/${agent}`); } + if (options.migrate) segments.push("migrate"); return `Clerk-CLI/${CURRENT_VERSION} (${segments.join("; ")})`; } From cc565f2df3c3d71df95d7be5ac61be5582a9c01a Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Fri, 30 Oct 2026 16:02:40 -0400 Subject: [PATCH 07/14] docs(migrate): say which telemetry fields migrate adds and when each is null The README said migrate events carry "plus four" fields, though `app_id` and `stage` go out on every command, and that each is null when the run never learned it, though `app_id` falls back to the linked project. It now says migrate adds `migrate_source` and `instance_id` and gives `app_id` and `stage` values of its own, with the null cases in each row. Co-Authored-By: Claude Opus 5.5 --- packages/cli-core/src/commands/migrate/README.md | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/packages/cli-core/src/commands/migrate/README.md b/packages/cli-core/src/commands/migrate/README.md index faa3b4024..ec01b0adc 100644 --- a/packages/cli-core/src/commands/migrate/README.md +++ b/packages/cli-core/src/commands/migrate/README.md @@ -1512,14 +1512,14 @@ stamping every user with today's. ## What telemetry records -Every `clerk migrate` event carries the fields every command sends, plus four that say what the run did. Each is null when the run never learned it. +`clerk migrate` adds two fields to the ones every command sends, `migrate_source` and `instance_id`, and gives `app_id` and `stage` values of its own: -| Field | What it holds | -| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `migrate_source` | Where the users come from: a built-in source or export platform (`auth0`, `clerk`, …), or `custom` for a source you wrote. Never its path or its key. | -| `instance_id` | The Clerk instance the command acted on (`ins_…`). For `export clerk`, the instance it read. | -| `app_id` | Its app (`app_…`): from `--app`, or from the linked profile when that profile owns the instance. A key for another app's instance sends null, not the linked app. | -| `stage` | How far the run got. One funnel per subcommand, below. | +| Field | What it holds | +| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `migrate_source` | Where the users come from: a built-in source or export platform (`auth0`, `clerk`, …), or `custom` for a source you wrote. Never its path or its key. Null when the run stopped before it knew the source. | +| `instance_id` | The Clerk instance the command acted on (`ins_…`). For `export clerk`, the instance it read. Null when the run stopped before it resolved one, or when Clerk didn't confirm it. | +| `app_id` | Its app (`app_…`): from `--app`, or from the linked profile when that profile owns the instance. When the run learned nothing about its target, the linked project's app, as for every command. Null for a key for another app's instance, or one whose instance Clerk didn't confirm. | +| `stage` | How far the run got: one funnel per subcommand, below. Null for `runs` and `sources`, which are one step each. | `workspace_id` is the linked profile's only when `app_id` is that profile's app. A `key_…` stand-in, the hash a run records when `GET /v1/instance` fails, is never sent: neither is an `--instance` alias or an app label. @@ -1529,8 +1529,6 @@ Stages, in the order each subcommand runs them. The last one set is the one sent - `export`: `migrate_pick_platform` (bare `clerk migrate export` only), `migrate_export_read` (credentials and reading users), `migrate_export_write`, `done`. - `undo`: `migrate_run_lookup`, `migrate_target`, `migrate_inspect`, `migrate_confirm`, `migrate_delete`, `done`. Terminal branch: `migrate_dry_run`. -`runs` and `sources` are one step each and send no stage. - A run that finishes part-way exits nonzero without throwing, and carries a code so it is not an uncoded error: | Ending | `error_code` | Exit | From ce7947a5820d73ecf47d330ca0a190822f240076 Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Fri, 30 Oct 2026 16:02:52 -0400 Subject: [PATCH 08/14] docs(migrate): put the telemetry exclusions and stages in the active voice Applies the review's wording for the two passive sentences, with an Oxford comma added to the list of values telemetry never sends. Co-Authored-By: Claude Opus 5.5 --- packages/cli-core/src/commands/migrate/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/cli-core/src/commands/migrate/README.md b/packages/cli-core/src/commands/migrate/README.md index ec01b0adc..997a26bbb 100644 --- a/packages/cli-core/src/commands/migrate/README.md +++ b/packages/cli-core/src/commands/migrate/README.md @@ -1521,9 +1521,9 @@ stamping every user with today's. | `app_id` | Its app (`app_…`): from `--app`, or from the linked profile when that profile owns the instance. When the run learned nothing about its target, the linked project's app, as for every command. Null for a key for another app's instance, or one whose instance Clerk didn't confirm. | | `stage` | How far the run got: one funnel per subcommand, below. Null for `runs` and `sources`, which are one step each. | -`workspace_id` is the linked profile's only when `app_id` is that profile's app. A `key_…` stand-in, the hash a run records when `GET /v1/instance` fails, is never sent: neither is an `--instance` alias or an app label. +`workspace_id` is the linked profile's only when `app_id` is that profile's app. Telemetry never sends a `key_…` stand-in (the hash a run records when `GET /v1/instance` fails), an `--instance` alias, or an app label. -Stages, in the order each subcommand runs them. The last one set is the one sent, so a failure or a Ctrl-C reports where the run stopped: +Stages, in the order each subcommand runs them. Telemetry sends the last stage set, so a failure or a Ctrl-C reports where the run stopped: - `import`: `migrate_source`, `migrate_target_setup` (sign-in, link), `migrate_input`, `migrate_target`, `migrate_resume`, `migrate_load`, `migrate_checks`, `migrate_confirm`, `migrate_import`, `done`. Terminal branches: `migrate_already_imported`, `migrate_dry_run`, `migrate_nothing_to_import`. - `export`: `migrate_pick_platform` (bare `clerk migrate export` only), `migrate_export_read` (credentials and reading users), `migrate_export_write`, `done`. From 9282b96d92a26445f38b771aef0facd7c248b3d2 Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Fri, 30 Oct 2026 16:03:26 -0400 Subject: [PATCH 09/14] docs(readme): drop the repeated telemetry exceptions and the passive voice Applies the review's wording for the root README's telemetry paragraph: the exceptions sentence no longer repeats the fields the first sentence lists, and `clerk feedback` scrubs the text itself. Oxford commas added to its two lists. Co-Authored-By: Claude Opus 5.5 --- README.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 5b6c7e173..19395df56 100644 --- a/README.md +++ b/README.md @@ -103,14 +103,13 @@ Commands: The Clerk CLI collects usage telemetry: command name, flag names, duration, outcome, environment signals (OS, install method, terminal), a random machine identifier, the -platform a migration's users come from, and the workspace, app and instance IDs of the +platform a migration's users come from, and the workspace, app, and instance IDs of the Clerk app a command acts on or the project is linked to. It doesn't collect raw command arguments, option values, file paths, or personal data, with two exceptions. `clerk migrate` -reports where a migration's users come from as a built-in platform name or `custom`, and -the app and instance IDs it acts on, which can come from `--app` and `--instance`. -`clerk feedback` sends the text you choose to send, which is scrubbed on a best-effort -basis for common secret formats, home-directory paths and email addresses before it -leaves your machine; +reports its source as a built-in platform name or `custom`, and the IDs it acts on can come +from `--app` and `--instance`. `clerk feedback` sends the text you choose to send, and +scrubs it on a best-effort basis for common secret formats, home-directory paths, and email +addresses before it leaves your machine; `clerk feedback --dry-run` shows exactly what would be sent. The first run only shows a disclosure notice and sends nothing (CI environments send from the first run), except `clerk feedback`, which shows the notice and then sends the report you asked it to send From cf4df262e5f25282c8cdb9accf9050892e28c7db Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Fri, 30 Oct 2026 16:03:34 -0400 Subject: [PATCH 10/14] docs(changeset): split the migrate telemetry changeset's long sentence Applies the review's suggestion as written. Co-Authored-By: Claude Opus 5.5 --- .changeset/migrate-telemetry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.changeset/migrate-telemetry.md b/.changeset/migrate-telemetry.md index 42ad0bfaa..5c740a425 100644 --- a/.changeset/migrate-telemetry.md +++ b/.changeset/migrate-telemetry.md @@ -2,4 +2,4 @@ "clerk": patch --- -Record in usage telemetry which platform a `clerk migrate` run's users come from, how far the run got, and the Clerk app and instance it acted on, and say so in the first-run telemetry notice. +Record in usage telemetry which platform a `clerk migrate` run's users come from, how far the run got, and the Clerk app and instance it acted on. The first-run telemetry notice now names these fields. From 70b7d1c73caf528402e5c874c635b1c7e7784e6d Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Fri, 30 Oct 2026 17:41:47 -0400 Subject: [PATCH 11/14] docs(migrate): say the migrate User-Agent segment skips the notice in CI The README said the CLI leaves the `migrate` segment out until the telemetry notice has shown. Like the events and the `AIAgent` segment, CI is exempt, so the segment goes out on a CI's first run. Applies the review's wording. Co-Authored-By: Claude Opus 5.5 --- packages/cli-core/src/commands/migrate/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/cli-core/src/commands/migrate/README.md b/packages/cli-core/src/commands/migrate/README.md index 997a26bbb..95f62cbd5 100644 --- a/packages/cli-core/src/commands/migrate/README.md +++ b/packages/cli-core/src/commands/migrate/README.md @@ -1538,7 +1538,7 @@ A run that finishes part-way exits nonzero without throwing, and carries a code | `undo` with deletes left | `migrate_undo_incomplete` | 1 | | `export` the platform cut short | `migrate_export_truncated` | 1 | -`clerk migrate` also adds a `migrate` segment to the `User-Agent` of its Backend API requests (`Clerk-CLI/ (…; migrate)`), so Clerk's logs can count the users each migration creates. The CLI follows the same rules as for telemetry: it leaves the segment out when telemetry is off, and until the telemetry notice has shown. +`clerk migrate` also adds a `migrate` segment to the `User-Agent` of its Backend API requests (`Clerk-CLI/ (…; migrate)`), so Clerk's logs can count the users each migration creates. The CLI follows the same rules as for telemetry: it leaves the segment out when telemetry is off and, outside CI, until the telemetry notice has shown. ## API endpoints From 82333ef31d9c071c1f7b5bd8c6f5f7b4c8d7782c Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Fri, 30 Oct 2026 17:42:06 -0400 Subject: [PATCH 12/14] docs(migrate): say instance_id carries the --instance hint before the run resolves MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The migrate group records `--app` and `--instance` as a telemetry target before anything resolves, so a run that stops early still names what it was pointed at. The README said `instance_id` is null in that case. It now says a run that hasn't resolved its target sends the `ins_…` or `app_…` you passed, unchecked. Co-Authored-By: Claude Opus 5.5 --- packages/cli-core/src/commands/migrate/README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/packages/cli-core/src/commands/migrate/README.md b/packages/cli-core/src/commands/migrate/README.md index 95f62cbd5..3c806407d 100644 --- a/packages/cli-core/src/commands/migrate/README.md +++ b/packages/cli-core/src/commands/migrate/README.md @@ -1514,12 +1514,12 @@ stamping every user with today's. `clerk migrate` adds two fields to the ones every command sends, `migrate_source` and `instance_id`, and gives `app_id` and `stage` values of its own: -| Field | What it holds | -| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `migrate_source` | Where the users come from: a built-in source or export platform (`auth0`, `clerk`, …), or `custom` for a source you wrote. Never its path or its key. Null when the run stopped before it knew the source. | -| `instance_id` | The Clerk instance the command acted on (`ins_…`). For `export clerk`, the instance it read. Null when the run stopped before it resolved one, or when Clerk didn't confirm it. | -| `app_id` | Its app (`app_…`): from `--app`, or from the linked profile when that profile owns the instance. When the run learned nothing about its target, the linked project's app, as for every command. Null for a key for another app's instance, or one whose instance Clerk didn't confirm. | -| `stage` | How far the run got: one funnel per subcommand, below. Null for `runs` and `sources`, which are one step each. | +| Field | What it holds | +| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `migrate_source` | Where the users come from: a built-in source or export platform (`auth0`, `clerk`, …), or `custom` for a source you wrote. Never its path or its key. Null when the run stopped before it knew the source. | +| `instance_id` | The Clerk instance the command acted on (`ins_…`). For `export clerk`, the instance it read. Before the run resolves its instance, the `--instance` you passed, unchecked, when it's an `ins_…` ID. Null when there's neither, or when Clerk didn't confirm the instance. | +| `app_id` | Its app (`app_…`): from `--app`, or from the linked profile when that profile owns the instance. Before the run resolves its target, the `--app` you passed, unchecked, when it's an `app_…` ID. When the run learned nothing about its target, the linked project's app, as for every command. Null for a key for another app's instance, or one whose instance Clerk didn't confirm. | +| `stage` | How far the run got: one funnel per subcommand, below. Null for `runs` and `sources`, which are one step each. | `workspace_id` is the linked profile's only when `app_id` is that profile's app. Telemetry never sends a `key_…` stand-in (the hash a run records when `GET /v1/instance` fails), an `--instance` alias, or an app label. From 4ddcd194091c2b600a06c7084fc3c970b0a57036 Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Fri, 30 Oct 2026 18:26:16 -0400 Subject: [PATCH 13/14] fix(telemetry): leave the analytics User-Agent segments out of dev builds Usage events skip dev builds, but the gate for the `AIAgent` and `migrate` User-Agent segments didn't, so our own testing with a dev build would count as migrations in the Backend API logs. The gate now skips dev builds too, unless `CLERK_TELEMETRY_URL` lifts that, as it does for the events. Co-Authored-By: Claude Opus 5.5 --- packages/cli-core/src/lib/fetch.test.ts | 14 ++++++++++++++ packages/cli-core/src/lib/fetch.ts | 8 ++++++-- packages/cli-core/src/lib/user-agent.ts | 4 ++-- 3 files changed, 22 insertions(+), 4 deletions(-) diff --git a/packages/cli-core/src/lib/fetch.test.ts b/packages/cli-core/src/lib/fetch.test.ts index 3f51b2302..2f154637f 100644 --- a/packages/cli-core/src/lib/fetch.test.ts +++ b/packages/cli-core/src/lib/fetch.test.ts @@ -72,6 +72,8 @@ describe("AIAgent and migrate segments honor the telemetry opt-out", () => { delete process.env.CI; delete process.env.CLERK_TELEMETRY_DISABLED; delete process.env.DO_NOT_TRACK; + // These run as a dev build; this variable lifts the guard, as for telemetry itself. + process.env.CLERK_TELEMETRY_URL = "https://telemetry.example.test"; }); afterEach(async () => { @@ -84,6 +86,7 @@ describe("AIAgent and migrate segments honor the telemetry opt-out", () => { else process.env.CI = originalCi; delete process.env.CLERK_TELEMETRY_DISABLED; delete process.env.DO_NOT_TRACK; + delete process.env.CLERK_TELEMETRY_URL; await rm(configDir, { recursive: true, force: true }); }); @@ -136,6 +139,17 @@ describe("AIAgent and migrate segments honor the telemetry opt-out", () => { markMigrateRequests(); expect(await sentUserAgent()).not.toContain("migrate"); }); + + // Our own testing with dev builds must not count as usage, as with the events. + test("both segments omitted from a dev build, even in CI", async () => { + await markTelemetryNoticeShown(); + process.env.CI = "1"; + delete process.env.CLERK_TELEMETRY_URL; + markMigrateRequests(); + const userAgent = await sentUserAgent(); + expect(userAgent).not.toContain("AIAgent/"); + expect(userAgent).not.toContain("migrate"); + }); }); describe("loggedFetch interrupt signal", () => { diff --git a/packages/cli-core/src/lib/fetch.ts b/packages/cli-core/src/lib/fetch.ts index ffb8ef1e2..861ec31c1 100644 --- a/packages/cli-core/src/lib/fetch.ts +++ b/packages/cli-core/src/lib/fetch.ts @@ -14,6 +14,7 @@ import { withNetworkAccess } from "./host-execution.ts"; import { buildUserAgent } from "./user-agent.ts"; import { optOutEnvVar } from "./env-signals.ts"; import { getTelemetryDisabled, getTelemetryNoticeShown } from "./config.ts"; +import { IS_DEV_BUILD } from "./version.ts"; let userAgentPromise: Promise | undefined; let migrateRequests = false; @@ -39,13 +40,16 @@ export function markMigrateRequests(): void { function resolveUserAgent(): Promise { // The AIAgent and migrate segments exist purely for analytics, so they // honor the telemetry opt-outs (env vars and `clerk telemetry disable`) - // and, like the events, stay absent until the disclosure notice has been - // shown — CI exempt, matching showTelemetryNoticeOnce. + // and, like the events, stay absent from dev builds (unless + // CLERK_TELEMETRY_URL lifts that, as in telemetryEnabled, which this module + // can't import) and until the disclosure notice has been shown — CI exempt, + // matching showTelemetryNoticeOnce. userAgentPromise ??= (async () => { const beforeDisclosure = !process.env.CI && !(await getTelemetryNoticeShown().catch(() => false)); const omitAnalyticsSegments = optOutEnvVar(process.env) !== null || + (IS_DEV_BUILD && !process.env.CLERK_TELEMETRY_URL) || beforeDisclosure || (await getTelemetryDisabled().catch(() => true)); return buildUserAgent(process.env, { diff --git a/packages/cli-core/src/lib/user-agent.ts b/packages/cli-core/src/lib/user-agent.ts index 755f51fa8..c6b299d26 100644 --- a/packages/cli-core/src/lib/user-agent.ts +++ b/packages/cli-core/src/lib/user-agent.ts @@ -9,8 +9,8 @@ * - : arm64 | x64 | … (process.arch) * - `ci` segment is appended when running under a recognized CI environment. * - `AIAgent/` segment is appended when an AI agent is detected — - * analytics-purposed, so the fetch layer omits it when telemetry is - * opted out or not yet disclosed. + * analytics-purposed, so the fetch layer omits it from dev builds and when + * telemetry is opted out or not yet disclosed. * - `migrate` segment is appended to `clerk migrate`'s requests, so the * Backend API logs can count a migration's user creates. Analytics too, * so the fetch layer omits it under the same rules as `AIAgent`. From 01321930be81e8e95311afb96ead30e6bfd13a8b Mon Sep 17 00:00:00 2001 From: Roy Anger Date: Fri, 30 Oct 2026 18:27:04 -0400 Subject: [PATCH 14/14] docs(migrate): say every request a migrate run sends carries the segment The flag lives in `loggedFetch` for the whole process, so Platform API, Frontend API, and export requests carry `migrate` too, as well as the Backend API requests the docs named. Co-Authored-By: Claude Opus 5.5 --- .changeset/migrate-user-agent.md | 2 +- packages/cli-core/src/commands/migrate/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.changeset/migrate-user-agent.md b/.changeset/migrate-user-agent.md index de29ea9b1..9461a6139 100644 --- a/.changeset/migrate-user-agent.md +++ b/.changeset/migrate-user-agent.md @@ -2,4 +2,4 @@ "clerk": patch --- -Mark `clerk migrate`'s Backend API requests with a `migrate` segment in the `User-Agent`, so Clerk can count the users each migration creates. Like usage telemetry, the CLI leaves it out when telemetry is off. +Mark the requests `clerk migrate` sends with a `migrate` segment in the `User-Agent`, so Clerk can count the users each migration creates. Like usage telemetry, the CLI leaves it out when telemetry is off. diff --git a/packages/cli-core/src/commands/migrate/README.md b/packages/cli-core/src/commands/migrate/README.md index 3c806407d..4231033fc 100644 --- a/packages/cli-core/src/commands/migrate/README.md +++ b/packages/cli-core/src/commands/migrate/README.md @@ -1538,7 +1538,7 @@ A run that finishes part-way exits nonzero without throwing, and carries a code | `undo` with deletes left | `migrate_undo_incomplete` | 1 | | `export` the platform cut short | `migrate_export_truncated` | 1 | -`clerk migrate` also adds a `migrate` segment to the `User-Agent` of its Backend API requests (`Clerk-CLI/ (…; migrate)`), so Clerk's logs can count the users each migration creates. The CLI follows the same rules as for telemetry: it leaves the segment out when telemetry is off and, outside CI, until the telemetry notice has shown. +`clerk migrate` also adds a `migrate` segment to the `User-Agent` of the requests it sends (`Clerk-CLI/ (…; migrate)`), so Clerk's logs can count the users each migration creates. The CLI follows the same rules as for telemetry: it leaves the segment out when telemetry is off and, outside CI, until the telemetry notice has shown. ## API endpoints