Repository navigation
perf: allow bavet score calculation to exit early if it can prove the move cannot be accepted - #2694
Christopher-Chianelli wants to merge 8 commits into
Conversation
0a41eaf to
de33f7f
Compare
de33f7f to
1b37495
Compare
1b37495 to
913ce43
Compare
triceo
left a comment
There was a problem hiding this comment.
See my comments on Enterprise.
|
This pull request, as it stands, would leave the Solver documentation stale in the place below. An automated check found it and can be wrong. The drift this comment reported earlier is gone. The new
{property_prefix}timefold.solver.{solver_name_prefix}use-tiered-score-calculation::
Enable xref:commercial-editions/performance-improvements.adoc#tieredScoreCalculation[tiered score calculation] to skip constraints of softer score levels when they cannot change whether a move is accepted.
Defaults to `false`.Separately, the new section in The docs live in A new commit on the branch updates this comment, and drift that is still there when the group merges gets a docs pull request against Generated by Solver Docs Drift Review with Claude. |
|
When it comes to the pessimistic bound, how does the mechanism work when there are rewards in the score? Doesn't the idea of rewards break the pessimistic bound idea? Two other related thoughts:
|
If the hard score is smaller after a move, no amount of rewards to the soft score will make the move an improvement.
That constraint would be calculated in the tier corresponding to the hardest score level it spans. That way, the total medium is available when all hard and medium constraints are evaluated.
Not relevant; we do the comparison after all constraints of a given level are evaluated, so it doesn't matter if a given constraint is a reward or penalty. |
cd1fccf to
eb49a4b
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Tiered scoring can persist partial scores as actual results, select worse fallback moves, trigger false corruption assertions, and cause ClassCastException failures with phase-specific environments.
4 open issues
What changed in this PR
This PR adds support for optional Enterprise tiered score calculation in Bavet. It lets move evaluation skip later score levels when the move cannot reach a supplied score bound.
Changes:
- Add bounded score calculation and connect it to construction heuristic and local search move evaluation.
- Refactor Bavet session creation and add Enterprise builder support.
- Expose the setting through XML, Spring Boot, and Quarkus, with documentation and configuration tests.
| File | Description |
|---|---|
| tools/benchmark/src/main/resources/benchmark.xsd | Add the tiered scoring setting to benchmark XML. |
| spring-integration/spring-boot-autoconfigure/src/test/java/ai/timefold/solver/spring/boot/autoconfigure/TimefoldSolverSingleSolverAutoConfigurationTest.java | Test Spring property mapping. |
| spring-integration/spring-boot-autoconfigure/src/main/java/ai/timefold/solver/spring/boot/autoconfigure/TimefoldSolverAutoConfiguration.java | Apply the Spring setting to solver configuration. |
| spring-integration/spring-boot-autoconfigure/src/main/java/ai/timefold/solver/spring/boot/autoconfigure/config/SolverProperty.java | Register the Spring property. |
| spring-integration/spring-boot-autoconfigure/src/main/java/ai/timefold/solver/spring/boot/autoconfigure/config/SolverProperties.java | Add the Spring property and its documentation. |
| quarkus-integration/quarkus/runtime/src/main/java/ai/timefold/solver/quarkus/TimefoldRecorder.java | Apply the Quarkus runtime setting. |
| quarkus-integration/quarkus/runtime/src/main/java/ai/timefold/solver/quarkus/config/SolverRuntimeConfig.java | Expose the Quarkus runtime property. |
| docs/src/modules/ROOT/pages/commercial-editions/performance-improvements.adoc | Document tiered scoring and configuration options. |
| core/src/test/java/ai/timefold/solver/core/impl/solver/DefaultSolverFactoryTest.java | Test rejection without Enterprise support. |
| core/src/test/java/ai/timefold/solver/core/impl/score/director/ScoreDirectorFactoryFactoryTest.java | Test default behavior and Enterprise requirements. |
| core/src/main/resources/solver.xsd | Add the tiered scoring XML element. |
| core/src/main/java/module-info.java | Export implementation packages to Enterprise. |
| core/src/main/java/ai/timefold/solver/core/impl/solver/DefaultSolverFactory.java | Pass the setting into heuristic configuration. |
| core/src/main/java/ai/timefold/solver/core/impl/score/stream/bavet/DefaultBavetConstraintSessionFactory.java | Extract standard Bavet session creation. |
| core/src/main/java/ai/timefold/solver/core/impl/score/stream/bavet/common/ConstraintNodeBuildHelper.java | Mark scorer lookup as nullable. |
| core/src/main/java/ai/timefold/solver/core/impl/score/stream/bavet/BavetConstraintSession.java | Expose the node-network constructor. |
| core/src/main/java/ai/timefold/solver/core/impl/score/stream/bavet/BavetConstraintFactory.java | Update a Javadoc link after the factory refactor. |
| core/src/main/java/ai/timefold/solver/core/impl/score/stream/bavet/AbstractBavetConstraintSessionFactory.java | Add reusable session and node-network construction hooks. |
| core/src/main/java/ai/timefold/solver/core/impl/score/director/stream/MultiEnvironmentBavetConstraintStreamScoreDirectorFactory.java | Generalize the builder return type. |
| core/src/main/java/ai/timefold/solver/core/impl/score/director/stream/BavetConstraintStreamScoreDirectorFactory.java | Select the Enterprise tiered builder when enabled. |
| core/src/main/java/ai/timefold/solver/core/impl/score/director/InnerScoreDirector.java | Define bounded scoring and temporary-move methods. |
| core/src/main/java/ai/timefold/solver/core/impl/score/director/AbstractScoreDirector.java | Implement bounded move execution and assertion handling. |
| core/src/main/java/ai/timefold/solver/core/impl/phase/custom/DefaultCustomPhaseFactory.java | Pass the tiered scoring flag to custom phases. |
| core/src/main/java/ai/timefold/solver/core/impl/phase/AbstractPhase.java | Add tiered scoring state and step-score recalculation. |
| core/src/main/java/ai/timefold/solver/core/impl/move/MoveDirector.java | Add temporary bounded move evaluation and undo handling. |
| core/src/main/java/ai/timefold/solver/core/impl/localsearch/DefaultLocalSearchPhaseFactory.java | Pass the tiered scoring flag to local search. |
| core/src/main/java/ai/timefold/solver/core/impl/localsearch/decider/LocalSearchDecider.java | Use acceptor bounds during move evaluation. |
| core/src/main/java/ai/timefold/solver/core/impl/localsearch/decider/acceptor/lateacceptance/LateAcceptanceAcceptor.java | Supply a late-acceptance score bound. |
| core/src/main/java/ai/timefold/solver/core/impl/localsearch/decider/acceptor/lateacceptance/DiversifiedLateAcceptanceAcceptor.java | Supply a move-specific diversified late-acceptance bound. |
| core/src/main/java/ai/timefold/solver/core/impl/localsearch/decider/acceptor/Acceptor.java | Define the optional acceptance-bound contract. |
| core/src/main/java/ai/timefold/solver/core/impl/heuristic/HeuristicConfigPolicy.java | Store and copy the tiered scoring flag. |
| core/src/main/java/ai/timefold/solver/core/impl/exhaustivesearch/DefaultExhaustiveSearchPhaseFactory.java | Pass the tiered scoring flag to exhaustive search. |
| core/src/main/java/ai/timefold/solver/core/impl/domain/solution/descriptor/SolutionDescriptor.java | Remove a blank line; no behavior change. |
| core/src/main/java/ai/timefold/solver/core/impl/constructionheuristic/DefaultConstructionHeuristicPhaseFactory.java | Pass the tiered scoring flag to construction heuristics. |
| core/src/main/java/ai/timefold/solver/core/impl/constructionheuristic/decider/forager/DefaultConstructionHeuristicForager.java | Derive a bound from the best move found so far. |
| core/src/main/java/ai/timefold/solver/core/impl/constructionheuristic/decider/forager/ConstructionHeuristicForager.java | Define the optional forager-bound contract. |
| core/src/main/java/ai/timefold/solver/core/impl/constructionheuristic/decider/ConstructionHeuristicDecider.java | Use the forager bound during move evaluation. |
| core/src/main/java/ai/timefold/solver/core/enterprise/TimefoldSolverEnterpriseService.java | Add the Enterprise tiered builder entry point. |
| core/src/main/java/ai/timefold/solver/core/config/score/director/ScoreDirectorFactoryConfig.java | Add the setting, fluent method, and inheritance support. |
| core/src/build/revapi-differences.json | Record the XML property-order compatibility exception. |
Review effort: Balanced
| if ((assertStepScoreFromScratch || assertExpectedStepScore || assertShadowVariablesAreNotStaleAfterStep) | ||
| && isTieredScoreCalculationEnabled) { |
There was a problem hiding this comment.
Besides the Copilot comment, I wonder - should assert modes disable this feature? You could argue that the assert mode should be clean, not smart, the barebones implementation.
| if (lowerBound == null) { | ||
| score = scoreDirector.executeTemporaryMove(moveScope.getMove(), assertMoveScoreFromScratch); | ||
| } else { | ||
| score = scoreDirector.executeTemporaryMoveAboveBound(move, lowerBound, assertMoveScoreFromScratch); |
… move cannot be accepted When enabled via its preview feature, Construction Heuristic's Forager and Local Search's Decider give infomation to the ScoreDirector about the lowest score move it would accept. If after computing a level node network, the score calculator could prove the resulting score would be lower than the minimum score, it skips applying the move to the later levels' node networks.
…rector configuration
…treamScoreDirectorFactory
ed03fef to
995b110
Compare
There was a problem hiding this comment.
It's been a while since I last saw this PR, so forgive me if this was already answered. How does a score bound actually look? Most scores do have MAX and MIN, but BigDecimal does not.
Does Bendable actually split the network across all levels? That seems wasteful, and absolutely guaranteed to ruin node sharing, if there are many levels. (Note we have seen problems with 15 score levels. Crazy, but it exists.)
I am leaving comments on what I see in this PR. I will deal with Enterprise once we agree on how this manifests in the Community solver. Also, plenty of Copilot comments to resolve one way or another.
| this.constraintStreamProfilingEnabled = constraintStreamProfilingEnabled; | ||
| } | ||
|
|
||
| public @Nullable Boolean getUseTieredScoreCalculation() { |
There was a problem hiding this comment.
Constraint streams automatic node sharing is enabled via constraintStreamAutomaticNodeSharing.
Which suggests this feature should be enabled via constraintStreamTieredScoreCalculation.
I would explicitly make this fail fast with score directors other than CS.
| * @see Acceptor#getAcceptedMoveScoreLowerBound(LocalSearchStepScope, boolean) | ||
| */ | ||
| @Nullable | ||
| default <Score_ extends Score<Score_>> Score_ getForagedMoveScoreLowerBound() { |
There was a problem hiding this comment.
The interface has currently three methods:
- addMove
- pickMove
- isQuitEarly
The first two go together, the third one got bolted on, and now we are adding getForagedMoveScoreLoweBound. This interface is not coherent at all, it looks like a jumble of methods randomly thrown together because it was necessary.
I'd rather not continue this pattern. We may need to establish some new layers/abstractions.
You could argue the same applies to the LS acceptor.
| InnerScore<?> score; | ||
| var lowerBound = forager.<Score_> getForagedMoveScoreLowerBound(); | ||
| if (lowerBound != null) { | ||
| score = scoreDirector.executeTemporaryMoveAboveBound(moveScope.getMove(), lowerBound, |
There was a problem hiding this comment.
The endless expansion of execute... methods on score director and move director needs to be stopped. We gotta start using overloads more; otherwise it becomes impossible to track who calls what - entry points into any functionality ought to be minimized.
| * @param allowDecreasingDuringStep if true, the method can return a score that decreases within a step. | ||
| * If false, it will return the lowest possible score it would accept across the entire step. |
There was a problem hiding this comment.
The name of the parameter already makes this pretty much obvious.
What is not obvious is why would this be requested, what purpose does it serve.
There was a problem hiding this comment.
Multithreaded solving during local search for diversified late acceptance. If the bound decreases after a move, then other moves executing with a higher bound may not be fully calculated if it was between the two bounds (so above new, below old).
| * @return null if it can accept a step with any {@link Score}, the minimum score of a potential step otherwise. | ||
| */ | ||
| @Nullable | ||
| default <Score_ extends Score<Score_>> Score_ getAcceptedMoveScoreLowerBound(LocalSearchStepScope<Solution_> stepScope, |
There was a problem hiding this comment.
Why default? Acceptors are under our control, we do not need backwards compatibility. I'd rather be explicit here.
| if ((assertStepScoreFromScratch || assertExpectedStepScore || assertShadowVariablesAreNotStaleAfterStep) | ||
| && isTieredScoreCalculationEnabled) { |
There was a problem hiding this comment.
Besides the Copilot comment, I wonder - should assert modes disable this feature? You could argue that the assert mode should be clean, not smart, the barebones implementation.
| } | ||
|
|
||
| @Override | ||
| public final InnerScore<Score_> calculateScoreAboveBound(Score_ lowerBound) { |
There was a problem hiding this comment.
Same comment as earlier - I'd prefer there to be a single calculateScore method with overloads, because then I can use my IDE to see where calculateScore is used, regardless of the purpose or qualification.
| For instance, if a move broke multiple hard constraints, then Timefold Solver may skip the medium and soft constraints. | ||
| For planning problems where soft constraints take a significant amount of time, this can significantly speed up move evaluation. | ||
|
|
||
| This feature is disabled by default. |
There was a problem hiding this comment.
As far as I know, the policy is to enable perf features on Enterprise by default.
What I would do is mention that this feature may interfere with the efficiency of automatic node sharing and therefore disabling it may counterintuitively bring performance in corner cases where nodes are shared across score levels.
It just a score, typically one either from the current step (CH) or a previous step (LS). If null, it is interepreted as "there is no bound".
Yes, but do note nodes uninvolved in a given level do not appear in that level's network. |


When enabled via its preview feature,
Construction Heuristic's Forager and Local Search's Decider give infomation to the ScoreDirector about the lowest score move it would accept. If after computing a level node network, the score calculator could prove the resulting score would be lower than the minimum score, it skips applying the move to the later levels' node networks.