Describe the problem
Generic hash joins reserve hash-bucket capacity from the number of build rows, even when most rows have the same join key. The lookup table stores one entry per distinct hash; duplicate rows are already retained in its separate row-index chain. Sizing both structures by total rows therefore wastes bucket memory and can reject a join whose payload, duplicate chain and useful lookup index fit in the memory pool.
For example, a build with many rows keyed by a small set of state codes still needs every row for join multiplicity, but does not need a hash bucket for every row. This is distinct from deduplicating semi-join input: ordinary joins must preserve all matching rows.
Expected behavior
Keep the row-index chain sized for every build row, but start the generic lookup index small and grow it only when needed. Preserve a fast construction path for mostly unique keys. Account for scratch and old/new index allocations during resizing so the optimization also works under a bounded memory pool.
Scope
The generic hash lookup path in HashJoinExec. The existing perfect-hash ArrayMap is a separate path. This does not add spilling or address build-payload concatenation accounting.
Describe the problem
Generic hash joins reserve hash-bucket capacity from the number of build rows, even when most rows have the same join key. The lookup table stores one entry per distinct hash; duplicate rows are already retained in its separate row-index chain. Sizing both structures by total rows therefore wastes bucket memory and can reject a join whose payload, duplicate chain and useful lookup index fit in the memory pool.
For example, a build with many rows keyed by a small set of state codes still needs every row for join multiplicity, but does not need a hash bucket for every row. This is distinct from deduplicating semi-join input: ordinary joins must preserve all matching rows.
Expected behavior
Keep the row-index chain sized for every build row, but start the generic lookup index small and grow it only when needed. Preserve a fast construction path for mostly unique keys. Account for scratch and old/new index allocations during resizing so the optimization also works under a bounded memory pool.
Scope
The generic hash lookup path in
HashJoinExec. The existing perfect-hashArrayMapis a separate path. This does not add spilling or address build-payload concatenation accounting.