Skip to content

docs: add guide for upgrading from PostgreSQL to IvorySQL - #340

Open
JerseyYee wants to merge 5 commits into
IvorySQL:masterfrom
JerseyYee:docs/pg-upgrade-guide
Open

JerseyYee wants to merge 5 commits into
IvorySQL:masterfrom
JerseyYee:docs/pg-upgrade-guide

Conversation

@JerseyYee

Copy link
Copy Markdown

Summary

Adds a new bilingual guide for migrating an existing PostgreSQL installation to IvorySQL, resolving #226.

The guide covers:

  • Comparison of the three upgrade paths (pg_dump/pg_restore, pg_dumpall, logical replication) and when to use each
  • Why pg_upgrade is not recommended for vanilla PostgreSQL → IvorySQL upgrades
  • Pre-upgrade checklist, including IvorySQL-specific considerations (dual-port configuration, Oracle-compatible-mode empty-string semantics, identifier casing)
  • Step-by-step instructions for each path, including global object migration and a near-zero-downtime logical replication cutover procedure
  • Post-upgrade validation, including sequence synchronization (which logical replication does not cover) and Oracle-mode smoke tests
  • FAQ covering the most likely pitfalls (empty-string semantics, identifier casing, extension compatibility, ownership)

Both Chinese and English versions are included, with nav entries added under the Data Migration section.

Closes #226

@github-actions
github-actions Bot temporarily deployed to pull request October 17, 2026 12:10 Inactive
@github-actions

Copy link
Copy Markdown

🚀 IvorySQL-Docs Preview Ready

@jiaoshuntian

Copy link
Copy Markdown
Contributor

Thank you for your contribution. These documents are very valuable to IvorySQL, and they are exactly what the project needs right now.

@jiaoshuntian

Copy link
Copy Markdown
Contributor

Thanks for adding this bilingual guide. Before merging, could you address the following points in both versions?

  1. Compatibility-mode configuration: ivorysql.compatible_mode is marked GUC_DISALLOW_IN_FILE, so it should not be set in ivorysql.conf. Please remove that configuration entry and explain how the connection port selects the compatibility behavior.
  2. Replica identity: The guide currently configures REPLICA IDENTITY FULL on the target. Source tables without a primary key need an appropriate replica identity before publication; otherwise, UPDATE/DELETE operations fail on the publisher.
  3. Offline migration: Please explicitly require stopping source writes before the final dump and keeping them stopped until cutover. Otherwise, writes made after the dump snapshot will not reach IvorySQL.
  4. Logical replication cutover: Checking latest_end_lsn alone does not establish that migration is complete. Please provide concrete checks confirming that all initial table copies have finished and the final source transactions have been applied before disabling the subscription.
  5. Sequence synchronization: The script only handles sequences in public. Please cover all migrated application schemas and generate schema-qualified sequence names to avoid missing or updating the wrong sequence.
  6. Global objects: Please import the required global objects and prepare tablespaces before restoring the database. The current order fails for tables using custom tablespaces. Alternatively, document using --no-tablespaces when restoring into the default tablespace is intended.

The structure looks useful; these changes would make the migration procedures safer and more reliable to follow.

- Remove ivorysql.compatible_mode from ivorysql.conf examples (GUC_DISALLOW_IN_FILE); document that compatibility mode is determined by the connection port
- Move global objects/tablespaces migration before data restore and add --no-tablespaces guidance
- Require stopping source writes before final dump for offline paths
- Fix replica identity guidance: set REPLICA IDENTITY FULL on the source for PK-less tables, add detection query
- Expand cutover checks: pg_subscription_rel srsubstate='r' plus LSN comparison and data validation
- Sequence sync script now covers all non-system schemas with qualified names
@JerseyYee

Copy link
Copy Markdown
Author

Thanks @jiaoshuntian for the careful review! All six points have been addressed in the latest commit (CN and EN):

  1. Compatible mode config: removed ivorysql.compatible_mode from ivorysql.conf examples (GUC_DISALLOW_IN_FILE); now verified via SHOW ivorysql.compatible_mode; on both ports, with a note that compatibility behavior is determined by the connection port.

  2. Replica identity: moved to the source side (new section "Set replica identity on the source"). PK-less tables now get REPLICA IDENTITY FULL on the publisher, plus a detection query listing user tables that lack a replica identity (relreplident = 'd' with no primary key).

  3. Offline migration stop-writes: both Path 1 and Path 2 now carry an IMPORTANT note requiring writes on the source to be stopped before the final dump and kept stopped until cutover/validation completes.

  4. Cutover checks: expanded to concrete steps - verify pg_subscription_rel shows srsubstate = 'r' for every subscribed table, then compare the source pg_current_wal_lsn() (recorded at write stop) with latest_end_lsn on IvorySQL, and run row-count/sampling validation before ALTER SUBSCRIPTION ... DISABLE.

  5. Sequence sync: the script now covers all non-system schemas using format('%I.%I', schemaname, sequencename) for fully qualified names, and filters out sequences with last_value IS NULL (never used, no sync needed).

  6. Global objects ordering: "Migrate global objects and tablespaces" now precedes data restore, with instructions to edit globals.sql and pre-create tablespace directories; pg_restore uses --no-tablespaces by default with guidance on preserving custom tablespaces instead.

@github-actions
github-actions Bot temporarily deployed to pull request October 19, 2026 12:16 Inactive
@github-actions

Copy link
Copy Markdown

🚀 IvorySQL-Docs Preview Ready

@jiaoshuntian

Copy link
Copy Markdown
Contributor

Thanks for addressing the previous feedback. I tested commit 6a34e92 with the assistance of Codex, using PostgreSQL 16.15 as the source database and IvorySQL 5.6 as the target database. All three migration paths successfully transferred the test data, and logical replication handled both the initial copy and subsequent INSERT/UPDATE/DELETE operations.

Three points still need correction in both language versions:

  1. The readiness query fails: srsubid::regsubscription produces ERROR: type "regsubscription" does not exist. Please join pg_subscription_rel with pg_subscription and select subname instead.
  2. The LSN check should not require exact equality: After stopping application writes, I ran only CHECKPOINT. The subscriber’s LSN advanced beyond the recorded position while the data remained consistent, so the equality condition stayed false. Please document a concrete final-apply completion check that accommodates progress beyond the boundary.
  3. Restore errors do not automatically stop psql -f: With a missing tablespace directory, the restore reported errors, continued, and returned exit code 0. The rows were restored, but the table ended up in the default tablespace. Please correct the statement that such an error aborts the restore, and document fail-fast execution with -v ON_ERROR_STOP=1 after handling expected conflicts such as existing roles.

The other previously raised points passed validation within this test’s scope. All temporary test instances and files have been cleaned up.

- Fix pg_subscription_rel readiness query: regsubscription type does not exist; join pg_subscription and select subname
- LSN final-apply check: no longer require exact equality; run CHECKPOINT after write stop, wait for latest_end_lsn to catch up (>=) and stop advancing
- psql -f for globals.sql and full_cluster.sql now uses -v ON_ERROR_STOP=1; document that default behavior continues on error and may place tables in default tablespace
@JerseyYee

Copy link
Copy Markdown
Author

Thanks again @jiaoshuntian for testing the changes and the precise feedback. All three points are fixed in the latest commit (CN and EN):

  1. Readiness query: replaced srsubid::regsubscription (type does not exist) with a join to pg_subscription selecting subname.

  2. LSN final-apply check: no longer requires exact equality. After stopping writes, run CHECKPOINT on the source to flush all committed transactions, record pg_current_wal_lsn() as L1, then wait for the subscriber's latest_end_lsn to be greater than or equal to L1 and observe that it no longer advances. This accommodates progress beyond the write-stop boundary from internal operations.

  3. psql -f error handling: both globals.sql and full_cluster.sql imports now use -v ON_ERROR_STOP=1. Removed the incorrect claim that a missing tablespace directory aborts the restore; added a note that default psql -f continues on errors and may place affected tables in the default tablespace, so ON_ERROR_STOP=1 is recommended. Conflicts such as pre-existing roles should be handled before running with this flag.

@github-actions
github-actions Bot temporarily deployed to pull request October 20, 2026 12:03 Inactive
@github-actions

Copy link
Copy Markdown

🚀 IvorySQL-Docs Preview Ready

@jiaoshuntian

Copy link
Copy Markdown
Contributor

One issue remains in Path 2: adding ON_ERROR_STOP=1 makes the restore stop at:

ERROR: role "postgres" already exists

The command exits with code 3 before creating the application database, because the dump contains CREATE ROLE postgres; while the target already has that role.

Please add instructions in both language versions to inspect full_cluster.sql and remove conflicting role-creation statements before importing. Path 1 currently explains this only for globals.sql.

After removing just the conflicting CREATE ROLE postgres; statement, the restore succeeded and all 100 test rows were restored. Please keep ON_ERROR_STOP=1 so genuine restore failures remain visible.

- Path 2: document that full_cluster.sql contains CREATE ROLE postgres, which already exists on the IvorySQL target
- Add pre-import steps: list role creation statements, compare with existing target roles, comment out conflicting CREATE ROLE lines
- Explain that ON_ERROR_STOP=1 exits with code 3 before the database is created when a role conflict remains; keep the flag after resolving conflicts
@JerseyYee

Copy link
Copy Markdown
Author

Thanks @jiaoshuntian, good catch - and thanks for re-testing with the change in place.

Fixed in both language versions. Path 2 now has a new section "Resolve role conflicts before import" ahead of the import command:

  • Explains that the globals section of full_cluster.sql always contains CREATE ROLE postgres;, while the IvorySQL target already has that role because initdb creates it.
  • Documents the exact failure mode you observed: with -v ON_ERROR_STOP=1, psql exits at that statement with exit code 3, before the application database is created and before any data is restored.
  • Gives the concrete pre-import sequence: grep -n '^CREATE ROLE' full_cluster.sql, compare with psql -U postgres -p 5432 -c '\du' on the target, then delete or comment out the CREATE ROLE statements for roles that already exist (example for postgres).
  • Notes that only the conflicting CREATE ROLE lines should be removed; the following ALTER ROLE statement should be kept because it aligns role attributes.
  • Keeps ON_ERROR_STOP=1 in the import command, and the note now states that the option should remain after the role conflicts are resolved so genuine restore failures are not hidden.

CN and EN have the same section structure, and no "error aborts the restore" wording remains in either version.

@github-actions
github-actions Bot temporarily deployed to pull request October 21, 2026 12:03 Inactive
@github-actions

Copy link
Copy Markdown

🚀 IvorySQL-Docs Preview Ready

@jiaoshuntian

Copy link
Copy Markdown
Contributor

One additional point needs correction before merging: the paragraph advising against pg_upgrade conflicts with the existing operation guide.

IvorySQL provides a PostgreSQL-to-IvorySQL upgrade path using its own pg_upgrade tool with -g (--using-ora-pg). This is documented in operation_guide.adoc, and the option is also present in the installed IvorySQL 5.6 tool.

Please replace the blanket recommendation against pg_upgrade and the unsupported claim about compatibility components not being initialized correctly. Instead, explain that this path requires the target IvorySQL version’s tool, the appropriate options, compatibility checks, and post-upgrade validation, then link to the operation guide.

This guide can remain focused on logical migration methods without implying that the documented pg_upgrade path is unsupported.

- Remove the blanket recommendation against pg_upgrade and the unsupported initialization claim
- Describe the documented physical upgrade path (target-version tool, -g/--using-ora-pg, compatibility check, post-upgrade validation) and link to operation_guide.adoc
@JerseyYee

Copy link
Copy Markdown
Author

Thanks @jiaoshuntian - corrected in both language versions.

The [IMPORTANT] block advising against pg_upgrade and the claim about compatibility components not being initialized correctly have been removed. In their place, the overview now states that a physical upgrade is supported when the source is native PostgreSQL and the target is IvorySQL, and summarizes what the path requires: the pg_upgrade binary shipped with the target IvorySQL version, the -g (--using-ora-pg) option, the -c compatibility check before upgrading, and post-upgrade validation. It links to operation_guide.adoc for the full procedure.

The guide's scope is now explicitly the three logical migration paths, without implying that the documented pg_upgrade path is unsupported. The wording was cross-checked against the pg_upgrade sections of operation_guide.adoc in both CN and EN.

@github-actions
github-actions Bot temporarily deployed to pull request October 22, 2026 11:15 Inactive
@github-actions

Copy link
Copy Markdown

🚀 IvorySQL-Docs Preview Ready

This branch was previously deployed

1 inactive deployment
pull request — 36183043 Deployed Oct 22, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants