[Shreyavemana]

Summary

Implements BM25-ranked tool discovery for large tool catalogs, building on BaseSearchTransform from issue #62. This enables models to find relevant tools using natural-language queries instead of relying on exact tool names or scanning the entire catalog.

Changes Made

Additions

• nitrostack/transforms/search/tokenizer.py — Added a tokenizer that normalizes text, splits punctuation, snake_case and camelCase, and removes common stop words.
• nitrostack/transforms/search/bm25_engine.py — Added a pure-Python Okapi BM25 engine with configurable (k_1) and (b), positive IDF, weighted term frequency, and deterministic tie-breaking by tool name.
• nitrostack/transforms/search/bm25.py — Added Bm25SearchTransform, implementing the rank(query, tools) interface and reusing the existing search infrastructure from issue [transforms] Progressive tool discovery (search_tools / call_tool) #62.
• tests/test_transforms_bm25.py — Added tests for tokenization, BM25 scoring, ranking, and index lifecycle.

Modifications

• nitrostack/transforms/search/__init__.py — Exported Bm25SearchTransform, Bm25Engine, and tokenize.

Implementation Details

• Builds the search index lazily on the first search.
• Uses a SHA-256 catalog fingerprint to detect changes and rebuild the index only when necessary.
• Applies field weights to prioritize tool names, titles, descriptions, and parameter text.
• Counts duplicate query terms only once and sorts equal-scoring results deterministically.
• Returns an empty result list for queries with no matches.
• Adds no new runtime dependencies and does not modify core/app.py.

Tests Added

• Tokenizer behavior, including camelCase, acronyms, punctuation, Unicode, digits, and stop words.
• BM25 score verification against a hand-calculated expected value.
• Positive IDF for common terms, deterministic tie-breaking, duplicate query terms, and invalid parameter validation.
• Golden ranking tests for fixed queries and a fixed tool catalog.
• Index lifecycle tests covering lazy initialization, catalog ordering, and rebuilds after tools are added, removed, or edited.
• Search-tools handler tests covering ranked results, result-limit clamping, and no-match behavior.

Closes #64.