Skip to content

Null-aware LeftMark joins miss logical NULLs in dictionary keys #25691

Description

@viirya

Describe the bug

Null-aware LeftMark hash joins inspect the physical null bitmap of dictionary-encoded join keys instead of their logical nulls. A DictionaryArray key can be physically valid while pointing to a NULL dictionary value, so the join misses that NULL and produces an incorrect FALSE mark where SQL three-valued logic requires NULL.

The problem affects both sides of an uncorrelated null-aware mark join:

  • A logical NULL build key is treated as non-null when the final mark column is built.
  • A logical NULL probe key is not recorded in the shared probe summary, including when it is seen by a different probe partition from the partition that completes the join.

To Reproduce

At the HashJoinExec level, use dictionary keys whose key bitmap contains no nulls but whose dictionary values contain a null:

  • Build values [1, NULL, 4], probe values [1, 2, 3]: the expected marks are [TRUE, NULL, FALSE], but the build-side logical null is marked FALSE.
  • Build values [1, 4], probe values containing [1, NULL]: the expected marks are [TRUE, NULL], but the unmatched build row is marked FALSE.

Expected behavior

Null-aware LeftMark joins should use Arrow logical-null semantics for dictionary arrays on both build and probe sides, producing the same SQL results as non-dictionary arrays.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions