Skip to content

fix: replace a Hive metastore table with a view without renaming the table - #1701

Merged
sd-db merged 4 commits into
mainfrom
fix/hms-table-to-view-no-rename
Oct 9, 2026
Merged

sd-db merged 4 commits into
mainfrom
fix/hms-table-to-view-no-rename

Conversation

@sd-db

@sd-db sd-db commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator

Description

Since #1688, a V1 view model that replaces an existing table goes through the shared replace flow (get_replace_sql). That flow picks safely_replace, which renames the existing table to <name>__dbt_backup before swapping in the new view. V2 has always taken this path, with or without use_safer_relation_operations.

Databricks rejects that rename for Hive metastore managed Delta tables on S3:

[DELTA_ALTER_TABLE_RENAME_NOT_ALLOWED] Operation not allowed: ALTER TABLE RENAME TO is not allowed for managed Delta tables on S3, as eventual consistency on S3 may corrupt the Delta transaction log. ...

So on AWS with the Hive metastore, changing a model from table to view fails by default on 1.12.6 (V1 and V2). 1.12.5 passed for V1 and failed for V2. The only workaround the error suggests is spark.databricks.delta.alterTable.rename.enabledOnAWS, which Databricks warns can corrupt the Delta log and which isn't available on SQL warehouses.

This PR stops the replace flow from renaming an existing Hive metastore table:

  • View replacing an HMS table: uses stage_then_replace. The view is built at a staging name, then the table is dropped and the staged view renamed into place. Renaming a view is allowed on S3, and a failed view build still leaves the table intact, so fix: use shared replacement flow for V1 view conversions #1688's protection holds.
  • Unchanged: Unity Catalog relations (managed UC tables rename fine on S3) and same-type CREATE OR REPLACE replacements.
  • Cache fix: stage_then_replace now marks the dropped relation in the relation cache. Without that, get_relation still reported a table after a successful conversion.

use_safer_relation_operations table and incremental rebuilds still rename tables, which is unchanged and opt-in. The flow docs now note that this needs the AWS flag on the Hive metastore.

Tests:

  • New macro tests in tests/unit/macros/relations/test_replace_macros.py cover strategy selection for HMS vs UC, with and without use_safer_relation_operations. The HMS cases fail on main.
  • TestTableConvertsToView (V1 and V2, including the failed-build-keeps-table case) on an AWS workspace with the Hive metastore: 4/4 pass. On main, the two conversion cases fail with the error above.
  • tests/functional/adapter/relations and views pass on the HMS cluster, the UC cluster and the SQL warehouse. metric_views, materialized_view_tests and streaming_tables pass on the SQL warehouse.

Checklist

  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • I have updated the CHANGELOG.md and added information about my change to the "dbt-databricks next" section.
  • [Optional] I have run the dbt-databricks-pr-ready project skill for this PR and addressed its merge-readiness feedback

…table

Since #1688, a V1 view model that replaces an existing table goes through
the shared replace flow, which renames the table to `__dbt_backup` before
swapping in the new view (V2 always did this). Databricks rejects
`ALTER TABLE ... RENAME TO` for Hive metastore managed Delta tables on S3
(`DELTA_ALTER_TABLE_RENAME_NOT_ALLOWED`), so every table-to-view change
fails on AWS by default.

The replace flow no longer renames an existing Hive metastore table. A
view replacing one is staged, the table is dropped, and the staged view is
renamed into place, so a failed view build still leaves the table intact.
Unity Catalog relations and same-type replacements are unchanged.
`stage_then_replace` now also removes the dropped relation from the
relation cache.
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Coverage report

This PR does not seem to contain any modification to coverable code.

Signed-off-by: Shubham Dhal <shubham.dhal@databricks.com>
@sd-db
sd-db merged commit ca51ad9 into main Oct 9, 2026
9 checks passed
@sd-db
sd-db deleted the fix/hms-table-to-view-no-rename branch October 9, 2026 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants