VB-20: Add flexible sweep configs and per-efSearch benchmark output

[nvzm123]

Summary

This PR updates the benchmark sweep flow to support more flexible algorithm configurations and cleaner efSearch handling across generation, execution, and result post-processing.

The main change is that sweep definitions can now specify algorithms as either the existing dictionary format or a new list-based format. The list format allows multiple entries for the same algorithm with different parameter groups, which makes it easier to express CAGRA/HNSW sweeps where parameters such as graph degree, intermediate graph degree, and HNSW layers need to vary together.

Changes

• Add support for list-based algorithms entries in sweep configs while preserving compatibility with the existing dictionary format.
• Treat efSearch as a per-run search parameter instead of expanding it into separate generated configs.
• Write per-efSearch benchmark outputs into separate result directories when sweeping multiple values.
• Update result conversion and Pareto analysis to read scalar efSearch values from benchmark metrics.
• Use hash-based indexDirPath generation for benchmark configs.
• Skip invalid or duplicate-producing parameter combinations, including:
  • cagraIntermediateGraphDegree < cagraGraphDegree
  • hnswMaxConn > hnswBeamWidth
• Update cuVS dependencies from 26.04.0 to 26.08.0.
• Switch cuVS/CAGRA benchmark setup to use heuristic parameter selection.
• Add Deep1B dataset entries for 1M, 10M, 20M, 40M, and 80M slices.
• Add an example sweeps_3000.json using the new list-based algorithm format.

Compatibility

Existing sweep files that use the previous dictionary-based algorithms format should continue to work. The new list-based format is additive and is intended for sweeps that need multiple parameter groups for the same algorithm.

Notes

This change also updates downstream result processing so converted NVIDIA-format output and Pareto marking correctly handle per-efSearch result directories.

[nvzm123]

cc: @narangvivek10 @kshitizgupta21 @singhmanas1 @cjnolet