Skip to content

Fix FlinkOrderedListState: preserve elements with same timestamp - #39795

Open
waterWang wants to merge 1 commit into
apache:masterfrom
waterWang:fix/flink-orderedlist-same-timestamp-overwrite
Open

Fix FlinkOrderedListState: preserve elements with same timestamp#39795
waterWang wants to merge 1 commit into
apache:masterfrom
waterWang:fix/flink-orderedlist-same-timestamp-overwrite

Conversation

@waterWang

Copy link
Copy Markdown

Fix FlinkOrderedListState data loss when multiple elements with the same timestamp are added to an OrderedListState.

Root cause: The readAsMap() method builds a TreeMap<Instant, TimestampedValue<T>> keyed by the element's timestamp. When a timestamp is shared by multiple elements, Map.put() silently overwrites the previous entry, destroying data.

Fix: Change the internal structure to SortedMap<Instant, List<TimestampedValue<T>>>, using computeIfAbsent to append same-timestamp elements to a list instead of overwriting. All callers (read(), readRange(), clearRange()) are updated to flatten the list via Iterables.concat().

Fixes #39782

@github-actions

Copy link
Copy Markdown
Contributor

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Java FlinkRunner OrderedListState silently overwrites duplicate elements sharing same timestamps

1 participant