Skip to content

CUR2-2107: sui object event deltas followup#9599

Open
tomfutago wants to merge 8 commits intomainfrom
CUR2-2107-sui-object-event-deltas-followup
Open

CUR2-2107: sui object event deltas followup#9599
tomfutago wants to merge 8 commits intomainfrom
CUR2-2107-sui-object-event-deltas-followup

Conversation

@tomfutago
Copy link
Copy Markdown
Contributor

@tomfutago tomfutago commented Apr 24, 2026

Summary

This PR refactors Sui object-event anchoring so tokens_sui_object_event_deltas no longer reconstructs prior object state by scanning recent history directly. Instead, it introduces tokens_sui_coin_object_latest_state as a compact helper table with one latest pre-window Coin object state per object_id, and uses that helper to stitch incremental windows safely. The downstream transfer models keep the same output contract, while the anchor reconstruction work moves into a reusable state layer.

Expected Performance

Structural reasoning, pending the first production-shaped CI/prod run:

  • tokens_sui_object_event_deltas no longer scans recent month-to-date history to reconstruct anchors.
  • Prior-state anchors now come from tokens_sui_coin_object_latest_state, joined to the active window’s window_first_version object set.
  • The old MTD recent_history_anchors scan is eliminated from object_event_deltas.
  • The added window_first_version group-by runs over the already-loaded active window rows, so it should be small relative to the removed history scan.

Rough estimate:

  • Current incremental runtime is around 25m, a more realistic first expectation is “meaningfully below that,” roughly in the 5-15m range depending on merge overhead

@github-actions github-actions Bot added WIP work in progress dbt: tokens covers the Tokens dbt subproject labels Apr 24, 2026
@tomfutago
Copy link
Copy Markdown
Contributor Author

Regression Report: PR #9599

Models tested: 1

Scope note: CI relations were built with the current bounded window 2026-01-01 to 2026-04-27, so this is a bounded-window regression, not full-history parity.

tokens_sui_object_event_deltas

  • Row count: prod 512,633,861, ci 512,633,861 (0.000000%) -- pass
  • Daily coverage: 0 dates with diffs -- pass
  • Aggregates (7d): rows match (26,875,247); sum(balance_delta) matches (16,381,918,215,434,972,864); unique_txs match (19,248,489); unique_objects match (3,468,636) -- pass
  • Key-match: ci_only 0, prod_only 0 -- pass
  • Block range: bounded by CI 2026-01-01 to 2026-04-27, checkpoints 229,588,356 to 269,524,071
  • Uniqueness: 0 duplicate keys, 0 duplicate rows -- pass

Verification queries:

tokens_sui_object_event_deltas

Downstream diffs, briefly: direct_transfers has 304 prod-only rows; owner_net_transfers has 1,246,533 prod-only rows; base_transfers and transfers each have 15,020 prod-only and 1 CI-only row. Since object_event_deltas matches exactly and downstream SQL only changed the temporary start date, these are downstream stale incremental residual artifacts in current prod, not an object_event_deltas regression.

@tomfutago tomfutago marked this pull request as ready for review April 27, 2026 13:02
@cursor
Copy link
Copy Markdown

cursor Bot commented Apr 27, 2026

PR Summary

Medium Risk
Moderate risk because it changes incremental anchoring/windowing logic for tokens_sui_object_event_deltas, which can affect correctness at window boundaries and in delayed-run scenarios.

Overview
tokens_sui_object_event_deltas no longer rebuilds prior object state by scanning month-to-date history; it now joins a new incremental helper, tokens_sui_coin_object_latest_state, to fetch a single pre-window anchor row per object_id.

The old monthly tokens_sui_coin_object_anchor_state model and its seed are removed, schema/docs are updated accordingly, and the new helper enforces "strictly before the incremental window" semantics (with a bounded catch-up window) so anchors cannot overlap the active window.

Reviewed by Cursor Bugbot for commit c01b3b5. Configure here.

@github-actions github-actions Bot added ready-for-review this PR development is complete, please review and removed WIP work in progress labels Apr 27, 2026
@tomfutago
Copy link
Copy Markdown
Contributor Author

bugbot run

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit c01b3b5. Configure here.

@tomfutago tomfutago requested a review from 0xRobin April 27, 2026 13:33
Copy link
Copy Markdown
Contributor

@0xRobin 0xRobin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tomfutago this tightly couples the runs of the latest_state to the event_deltas model which is not ideal.
If they are ever out of sync they will start producing wrong data..
I'm not sure this is the right approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dbt: tokens covers the Tokens dbt subproject ready-for-review this PR development is complete, please review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants