Skip to content

feat(migrations): Unify schema and data journals#896

Draft
dcramer wants to merge 8 commits into
mainfrom
codex/unified-migration-journals
Draft

feat(migrations): Unify schema and data journals#896
dcramer wants to merge 8 commits into
mainfrom
codex/unified-migration-journals

Conversation

@dcramer

@dcramer dcramer commented Jul 13, 2026

Copy link
Copy Markdown
Member

Summary

  • add the publishable @sentry/junior-migrations package for mixed SQL and TypeScript Drizzle journals
  • generate 0001_name.ts data migrations through the same Drizzle Kit numbering workflow used for schema migrations
  • run core and enabled-plugin migrations through one ordered, resumable journal runner
  • expose the exact host database adapter as context.database
  • keep one-off transformations in migration files while allowing append-only infrastructure helpers from @sentry/junior/migration-helpers/v1
  • move every existing hardcoded core and scheduler data migration, including conversation usage repair from current main, into the journal
  • remove plugin migrationTasks, context.tasks, duplicate host migration implementations, and unchanged Drizzle snapshots for data-only entries
  • include the new package in Craft, CI, release alignment, package validation, and documentation

Migration contract

Each journal entry resolves to exactly one immutable <tag>.sql or <tag>.ts file. TypeScript entries target MigrationV1, receive the host database/state adapters, and own their one-off record transformations. They may import only external packages and the versioned @sentry/junior/migration-helpers/v1 infrastructure surface; unversioned Junior runtime imports, relative imports, dynamic imports, and runtime re-exports are rejected.

Drizzle Kit still owns schema snapshots and journal numbering. The data-migration generator removes the unchanged custom snapshot after creating a TypeScript entry; a verified follow-up schema generation continues from the latest real schema snapshot and preserves journal order.

Existing installations adopt their legacy core/plugin migration ledgers once, then all pending SQL and TypeScript entries execute through runMigrationJournal. TypeScript progress and results are persisted in the Drizzle ledger and failed entries resume with the same source hash.

Adversarial cleanup

After rebasing onto current main, the PR initially measured 7,447 additions. The dominant bloat was a 3,575-line whole-app bundle for legacy history plus 1,292 lines of unchanged custom snapshots. The final diff is 3,889 additions: the history migration is now 1,128 lines of conversion/storage code, data snapshots are removed, and main's usage repair is journaled rather than left as a separate upgrade path.

Test Plan

  • pnpm --filter @sentry/junior-migrations typecheck
  • pnpm --filter @sentry/junior typecheck
  • pnpm --filter @sentry/junior-migrations test — 9 tests
  • integrated core/plugin/scheduler/history and real-database migration suite — 107 tests
  • pnpm lint
  • pnpm release:check
  • pnpm docs:check
  • git diff --check

Garfield adversarial review

Three review cycles addressed strict historical scheduler decoding, malformed existing-row repair, safe migration paths, symlink rejection, final source hash verification, finite JSON progress/result validation, original error preservation, runner-only lock capabilities, dead helper APIs, shared state projection, package version alignment, and stale docs. Independent verification confirmed the validated fixes are committed in this PR.

Final framework guarantees

  • migration-helpers/v1 is a permanent compatibility path; breaking helper changes require a new versioned path, and shipped migration source hashes remain immutable.
  • Plugin TypeScript migrations receive plugin-scoped state by default while historical junior:<plugin>: keys remain available for legacy conversion.
  • Schema bootstrap is explicit (mode: "schema-bootstrap" or bootstrapPluginSchemas) and cannot be selected by omission.
  • Real PostgreSQL contracts cover failed progress persistence, resume-to-completion, and concurrent journal serialization.

@vercel

vercel Bot commented Jul 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
junior-docs Ready Ready Preview Jul 16, 2026 11:55pm

Request Review

Comment thread packages/junior-migrations/package.json
Comment thread packages/junior/src/chat/conversations/sql/migrations.ts Outdated
Comment thread packages/junior/migrations/0005_redis_conversation_state.ts
Comment thread packages/junior/migrations/0006_conversations_to_sql.ts Outdated
Comment thread packages/junior/migrations/0007_conversation_history_to_sql.ts Outdated
Comment thread packages/junior-migrations/src/runner.ts
Comment thread packages/junior-scheduler/migrations/0002_scheduler_state_to_sql.ts
dcramer and others added 5 commits July 16, 2026 11:52
Add a publishable migration package that executes SQL and TypeScript entries
from one Drizzle-compatible journal. Route core and plugin upgrades through the
shared runner so schema changes and resumable data migrations use one ordered
ledger.

Keep historical scripts isolated behind versioned capabilities, and wire the
package into release, CI, Craft, documentation, and package validation.

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Remove the migration task registry and keep historical data migration code in each journal entry. This prevents future runtime refactors or deletions from breaking pending upgrades.

Expose versioned migration capabilities, move scheduler and core backfills into their migration files, and update migration tests and documentation.

Co-Authored-By: GPT-5.6 Codex <noreply@openai.com>
Expose the exact host database adapter through MigrationContextV1 instead of reconstructing a partial SQL capability.

Rebuild the SQL-heavy frozen migrations without connection setup and driver implementations so migrations retain transformation logic without duplicating database infrastructure.

Co-Authored-By: GPT-5.6 Codex <noreply@openai.com>
Allow migrations to import append-only infrastructure helpers from @sentry/junior/migration-helpers/v1 while continuing to reject unversioned Junior runtime imports.

Keep one-off Redis transformations and backfill decisions in their journal entries, replace generated capsules for migrations 0004 through 0006 with readable source, and publish explicit helper types without private module references.

Co-Authored-By: GPT-5.6 Codex <noreply@openai.com>
Rebase the mixed journal onto the latest schema, move the new conversation usage repair into the journal, and preserve its focused tests.

Remove unchanged Drizzle snapshots for data entries, rebuild the legacy history migration around versioned infrastructure helpers, and reject runtime re-export bypasses.

Co-Authored-By: GPT-5.6 Codex <noreply@openai.com>
Comment thread packages/junior-migrations/src/journal.ts
Freeze complete scheduler migration schemas, repair malformed existing scheduler rows, and add focused regression coverage.

Harden migration path and JSON boundaries, narrow the runner-only locking interface, remove dead helper surface, deduplicate state projection, and preserve original migration errors.

Co-Authored-By: GPT-5.6 Codex <noreply@openai.com>
Comment thread packages/junior/migrations/0008_conversation_history_to_sql.ts
Comment thread packages/junior/migrations/0008_conversation_history_to_sql.ts Outdated
Require explicit schema-bootstrap execution, scope plugin migration state by default while preserving historical plugin keys, and document permanent helper-version compatibility.

Add real PostgreSQL progress, resume, and lock serialization contracts plus host-wiring isolation coverage.

Co-Authored-By: GPT-5.6 Codex <noreply@openai.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 360518a. Configure here.

Comment thread packages/junior/src/chat/migrations/state-v1.ts Outdated
Pass the real core state adapter into TypeScript migrations so Redis-native index and lock detection continues to work.

Add multiline import guard coverage and remove the stale generated temp-path comment from the history migration.

Co-Authored-By: GPT-5.6 Codex <noreply@openai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant