Add minimal_perfect_hash, two_level_hash and minimal_cover_hash policies - #103
Open
jll63 wants to merge 5 commits into
Open
Add minimal_perfect_hash, two_level_hash and minimal_cover_hash policies#103jll63 wants to merge 5 commits into
jll63 wants to merge 5 commits into
Conversation
…hash Three new type_hash policies, from the research for boostorg#57. fast_perfect_hash stays the default: none of these beats it as a general v-table lookup. What they offer is a way out of its one real failure mode - its randomized search costs tens to hundreds of milliseconds on a sparse type id set and gives up outright on a large one, and the table it finds is sized by where the addresses are rather than by how many there are. minimal_perfect_hash (boostorg#56) is hash-and-displace: one slot per type id whatever the addresses are, a search whose cost depends only on the class count, and no instruction-set requirement. It pays with a second dependent load on the dispatch path. It is the one to reach for in a program that dlopens modules registering classes of their own. two_level_hash is the same family with the final reduction replaced by a shift into a power-of-two table: cheaper per dispatch where the compiler hoists the shift amount, at the cost of a table that is a sawtooth between 1.0 and 2.0 slots per type id rather than a flat figure. minimal_cover_hash picks the smallest set of bit positions that still separates the type ids and extracts them with pext. It dispatches as fast as the default and finds its table deterministically, but BMI2 is not a portable requirement, so the header always compiles and naming the policy in a registry is what fails when the instruction is unavailable - with a diagnostic that names both the flag and the portable alternative. Four changes from the prototypes, beyond namespace and naming: - detail::uintptr moves from fast_perfect_hash.hpp to preamble.hpp. Every type_hash policy needs it; only one of them had it. - minimal_perfect_hash and two_level_hash hash `x + 1`. Zero is a fixed point of a multiply, so a type id of 0 would be pinned to slot 0 for every seed and every pilot and the search could fail spuriously. One increment on the dispatch path buys a policy that works for any type id, including the small integers a custom rtti policy may hand out. - two_level_hash drops the M1Candidates knob: scoring first-level multipliers for even buckets was measured and does not pay, because the placement cost is set by the tail, where every remaining bucket faces an almost full table. Its table cap is now relative to the class count rather than an absolute 2^26, which could ask for half a gigabyte. - aux_bytes() is gone from both. It is not part of the TypeHashFn contract; it existed so a benchmark could report the pilot array size. finalize() now releases that array, which the prototypes leaked until the registry died. Docs and tests follow.
minimal_perfect_hash and two_level_hash hashed `x + 1` so that a type id of zero could not be pinned to slot 0. That put an increment on every dispatch to buy something no caller needs: addresses are never zero, so std_rtti and static_rtti could never hit it, and a custom rtti policy handing out small integers can simply not start at zero. So make it a precondition instead. Both policies hash the type id directly again, both document zero as outside their domain, and `initialize` asserts that none of the registered ids is zero when the registry has runtime_checks - one comparison, since the ids are sorted by then, and compiled out entirely otherwise. The dispatch path is back to imul/shr/load/imul/mul/load, with nothing in front of the first multiply.
Four new files, and the first tests in the suite to drive a type_hash policy
directly rather than through dispatch.
test_hash_policies.cpp feeds each policy a fabricated InitializeContext over
chosen type ids, which is the only way to present a distribution deliberately
instead of taking whatever this program's own classes happen to get. Four
distributions: one packed module, one diluted module (v-tables emitted between
the records, which is what a real one looks like), a program with implicitly
linked libraries, and a program plus dlopened modules tens of terabytes apart.
It asserts injectivity and that hash_range brackets every value on all of them,
that the table size is identical for the packed and the dlopened sets - the
property this family of policies exists for - that minimal_perfect_hash<2, 100>
is exactly minimal and the default is ceil(n / 0.95), that two_level_hash's
table is a power of two between n and 2n, that a type id registered by several
modules is not a collision, and that initialize works again after finalize.
The generators keep a per-module cursor so the ids are distinct by construction,
and a test case asserts that much: a repeated id would exercise the policies'
deduplication rather than their hashing, and would make every injectivity count
come out short for no fault of the policy. Getting that wrong the first time is
what caught it.
test_dispatch_{minimal_perfect,two_level,minimal_cover}_hash.cpp run each policy
end to end - single and multiple dispatch over a five-class hierarchy - with
runtime_checks on unconditionally rather than only in a Debug build, so the
control table that `hash` consults is exercised in both configurations, and with
throw_error_handler so that a call passing an unregistered class is observable as
missing_class instead of aborting.
The minimal_cover_hash test needs BMI2 for the whole translation unit, which the
CMake build adds for that one target on x86, and which b2 gets from a new
config//has_bmi2 probe - the same shape as the existing has_reflection one.
Probing beats naming an architecture: an <architecture>x86 conditional does not
match every toolset spelling. Where the instruction is absent the test still
builds, as one case that records why it did nothing.
test_policies.cpp gains static_asserts that each new policy satisfies the
TypeHashFn blueprint, and that `with` replaces a type_hash policy in place
rather than appending a second - which would leave vptr_vector reading the wrong
state.
The tutorial explained dlopen without ever saying what is different about it for type ids, which is the thing that decides whether the default hash policy copes. A new section of shared_libraries.adoc, "Type Ids Across Modules", fills that in: that a type id is `&typeid(X)`; that the Itanium ABI requires pointer identity across modules and the linker delivers it for an implicitly linked library with a copy relocation, so a program and the libraries it links against present one compact set of ids; that dlopen gets none of that, because nothing names a plugin's classes, so its records stay in its own mapping wherever the loader put it; and that RTTI has to keep default visibility for any of it to work, which is why the library's own tests mark their classes BOOST_SYMBOL_VISIBLE. None of that was written down anywhere outside a comment in test/dynamic_loading/classes.hpp. Then what it costs - fast_perfect_hash searching over several far-apart clusters, and vptr_vector sizing its table from the result - and the four ways out, as a table: the three new policies and vptr_map, which sidesteps the question by not hashing at all. Each with the one declaration that selects it, and a note that `with` replaces by category in place, so the ordering rule elsewhere on the page is not something a caller has to think about. Also: three entries in ref_headers.adoc; a paragraph in registries_and_policies.adoc saying there are four type_hash policies and what the other three are for; a forward reference in performance.adoc, whose figures are fast_perfect_hash's specifically; and three tagged snippets in snippets/policies.cpp, which is compiled and run as a test, so the examples in the reference cannot rot. Two fixes the rendered output turned up, neither visible in the source: - `@ref minimal_perfect_hash:` had its colon absorbed into the reference name, so the sentence rendered as "...as in `minimal_perfect_hash` zero is a fixed point...". A colon is valid in a qualified name, so the parser takes it. Reworded to end the sentence with a period instead. Note that five shipped headers have the same construct and lose their colons the same way - initialize.hpp's `@li @ref missing_class:` among them - which is left alone here. - BOOST_OPENMETHOD_HAS_PEXT got no reference page, because its doc comment was separated from both `#define` directives by the `#if` that chooses between them, and a comment on the far side of a directive is not attached. Every @ref to it therefore rendered as plain text. The detection now sets an internal macro and the documented one is a single unconditional `#define` with the comment attached to it. Verified by rendering, not by reading: every @ref in the three new headers resolves to a link, the new section's table and both code blocks render, the three cross-references into its anchor resolve, no page leaks MRDOCS, and no stray backticks survive on any of the four edited pages.
|
An automated preview of the documentation is available at https://103.openmethod.prtest3.cppalliance.org/libs/openmethod/doc/html/index.html If more commits are pushed to the pull request, the docs will rebuild at the same URL. 2026-09-11 19:02:07 UTC |
**minimal_cover_hash's static_assert fired at parse time on GCC 11 and 12, and Clang 13 through 15.** A static_assert whose condition does not depend on the enclosing template may be diagnosed as soon as the template is *defined* rather than when it is instantiated - the standard calls such a template ill-formed, no diagnostic required, and compilers differ on when they report it. The condition was BOOST_OPENMETHOD_HAS_PEXT, a plain 0 or 1, so on those compilers merely *including* the header was an error when the instruction was unavailable, which is the one thing the header promises not to do. It now goes through detail::has_pext<Registry>, a variable template, so the condition is dependent and the check happens on use. Not reproducible here: the oldest local compiler is GCC 13, which defers, and no container runtime is available. Verified instead that including the header is clean on GCC 13, 15 and 16 and Clang 22, that naming the policy in a registry still fails with the same diagnostic, and that it still works under -mbmi2. **test_hash_policies fabricated 64-bit addresses**, so on a 32-bit target reinterpret_cast to type_id truncated them. The multi_module bases differ only in their high bits, so all four collapsed onto one another and the generator emitted duplicates - 988 distinct ids out of 1000. The bases are now derived from sizeof(uintptr_t), with a `spread` parameter saying how far apart the modules sit, so the same layout holds at either width. The fixture's own generators_produce_distinct_ids case caught this first in CI and pointed straight at the cause, which is what it is there for. Reproduced locally with -m32 - the old file fails with exactly CI's [16 != 17] and [988 != 1000], the new one passes at both widths. 164 tests pass in Release and Debug. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqH1U2Ky7ubqKzqSvgxY9y
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
(Written by Claude Code, on behalf of @jll63.)
Three new
type_hashpolicies, plus the tutorial section explaining the situation they address. Closes #56.fast_perfect_hashstays the default and is still the right choice for almost every program — none of these beats it as a general v-table lookup. What they offer is a way out of its one real failure mode: its randomized search costs tens to hundreds of milliseconds on a sparse type id set and gives up outright on a large one, and the table it finds is sized by where the addresses are rather than by how many there are. A program thatdlopens class-registering modules is the one most likely to hit both.minimal_perfect_hash(#56)two_level_hashminimal_cover_hashDocumentation
A new
## Type Ids Across Modulessection inshared_libraries.adoc. The tutorial covereddlopenwithout ever saying what is different about it for type ids, which is what decides whether the default hash copes. It now explains that a type id is&typeid(X); that the Itanium ABI requires pointer identity across modules and the linker delivers it for an implicitly linked library with a copy relocation, so a program and its libraries present one compact set of ids; thatdlopengets none of that; and that RTTI must keep default visibility for any of it to work — which is why the library's own tests mark classesBOOST_SYMBOL_VISIBLE. None of that was written down anywhere outside a comment intest/dynamic_loading/classes.hpp.Then what it costs, and the four ways out as a table — the three policies and
vptr_map, which sidesteps the question by not hashing at all — each with the one declaration that selects it.Also three entries in
ref_headers.adoc, a paragraph inregistries_and_policies.adoc, a forward reference inperformance.adoc(whose figures arefast_perfect_hash's specifically), and three tagged snippets insnippets/policies.cpp, which is compiled and run as a test so the reference examples cannot rot.Notes on the implementation
detail::uintptrmoves topreamble.hpp. Everytype_hashpolicy needs it; onlyfast_perfect_hashhad it.runtime_checks. Zero is a fixed point of a multiply, so it would be pinned to slot 0 for every seed and pilot and the search could fail spuriously. Addresses are never zero, sostd_rttiandstatic_rttiare unaffected; a customrttipolicy handing out small integers must not start at zero. The assert is one comparison — the ids are sorted by then — and compiled out otherwise, leaving the dispatch path untouched.minimal_cover_hashalways compiles, everywhere. BMI2 is not a portable requirement, and becausehashisBOOST_FORCEINLINEthe flag is a whole-program one, so a bare#errorwould break any TU that merely included the header. Instead the header exposesBOOST_OPENMETHOD_HAS_PEXT, and naming the policy in a registry is what fails — with one diagnostic naming both the flag and the portable alternative.two_level_hash's table cap is relative to the class count, not an absolute2^26, which could have asked for half a gigabyte.Tests
164 pass in Release and Debug. Three end-to-end dispatch tests (single and multiple dispatch,
runtime_checkson unconditionally so the control table is exercised in both build types,throw_error_handlerso an unregistered class surfaces asmissing_class), plustest_hash_policies.cpp— the first test here to drive atype_hashpolicy directly, through a fabricatedInitializeContext, over four distributions including a program plusdlopened modules tens of terabytes apart. It asserts injectivity, that the table size is identical for the packed anddlopened sets, exact minimality at<2, 100>, the power-of-two property, that a type id registered by several modules is not a collision, and that initialize works after finalize.minimal_cover_hash's test needs BMI2 for the whole TU: CMake adds it for that one target on x86, and b2 gets it from a newconfig//has_bmi2probe, the same shape as the existinghas_reflectionone. Probing beats naming an architecture — an<architecture>x86conditional did not match. Where the instruction is absent the test still builds, as one case recording why it did nothing.Verified
CMake Release and Debug (164 each), the rendered documentation (every
@refin the new headers resolves, the new section's table and code blocks render, the three cross-references into its anchor resolve, no page leaksMRDOCS, no stray backticks),dev/check-flat.sh, and the snippet target.Not verified locally: b2. Boost.Test's own
unit_test_parameters.ofails to compile on this machine on a pristinedevelopcheckout too — clang 18 against GCC 16's libstdc++ — so the Jamfile changes are confirmed only to parse and resolve targets, not to run. Worth a look in CI.🤖 Generated with Claude Code
https://claude.ai/code/session_01Uegbpo2mmwkWaHsffQtmeB