Repository navigation
fix: replace a Hive metastore table with a view without renaming the table - #1701
Merged
Merged
Conversation
…table Since #1688, a V1 view model that replaces an existing table goes through the shared replace flow, which renames the table to `__dbt_backup` before swapping in the new view (V2 always did this). Databricks rejects `ALTER TABLE ... RENAME TO` for Hive metastore managed Delta tables on S3 (`DELTA_ALTER_TABLE_RENAME_NOT_ALLOWED`), so every table-to-view change fails on AWS by default. The replace flow no longer renames an existing Hive metastore table. A view replacing one is staged, the table is dropped, and the staged view is renamed into place, so a failed view build still leaves the table intact. Unity Catalog relations and same-type replacements are unchanged. `stage_then_replace` now also removes the dropped relation from the relation cache.
saishreeeee
approved these changes
Oct 7, 2026
Signed-off-by: Shubham Dhal <shubham.dhal@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Since #1688, a V1 view model that replaces an existing table goes through the shared replace flow (
get_replace_sql). That flow pickssafely_replace, which renames the existing table to<name>__dbt_backupbefore swapping in the new view. V2 has always taken this path, with or withoutuse_safer_relation_operations.Databricks rejects that rename for Hive metastore managed Delta tables on S3:
So on AWS with the Hive metastore, changing a model from
tabletoviewfails by default on 1.12.6 (V1 and V2). 1.12.5 passed for V1 and failed for V2. The only workaround the error suggests isspark.databricks.delta.alterTable.rename.enabledOnAWS, which Databricks warns can corrupt the Delta log and which isn't available on SQL warehouses.This PR stops the replace flow from renaming an existing Hive metastore table:
stage_then_replace. The view is built at a staging name, then the table is dropped and the staged view renamed into place. Renaming a view is allowed on S3, and a failed view build still leaves the table intact, so fix: use shared replacement flow for V1 view conversions #1688's protection holds.CREATE OR REPLACEreplacements.stage_then_replacenow marks the dropped relation in the relation cache. Without that,get_relationstill reported atableafter a successful conversion.use_safer_relation_operationstable and incremental rebuilds still rename tables, which is unchanged and opt-in. The flow docs now note that this needs the AWS flag on the Hive metastore.Tests:
tests/unit/macros/relations/test_replace_macros.pycover strategy selection for HMS vs UC, with and withoutuse_safer_relation_operations. The HMS cases fail onmain.TestTableConvertsToView(V1 and V2, including the failed-build-keeps-table case) on an AWS workspace with the Hive metastore: 4/4 pass. Onmain, the two conversion cases fail with the error above.tests/functional/adapter/relationsandviewspass on the HMS cluster, the UC cluster and the SQL warehouse.metric_views,materialized_view_testsandstreaming_tablespass on the SQL warehouse.Checklist
CHANGELOG.mdand added information about my change to the "dbt-databricks next" section.dbt-databricks-pr-readyproject skill for this PR and addressed its merge-readiness feedback