`HFCrossEncoderReranker` returns scores in original input order after reranking

[davidhaley]

Codex found this bug while I was making a plan - double checked with Grok and it found the same bug.

---

HFCrossEncoderReranker returns scores in original input order after reranking

Description

HFCrossEncoderReranker.rank(..., return_score=True) returns reranked documents in descending score order, but returns the accompanying score array in the original input order.

As a result, the returned documents and scores are misaligned:

reranked_docs[i]

does not necessarily correspond to:

scores[i]

This affects RedisVL 0.23.0 and appears to remain present in current releases/main.

Reproduction

Given several documents whose cross-encoder scores are not already in descending input order:

from redisvl.utils.rerank import HFCrossEncoderReranker

reranker = HFCrossEncoderReranker(
    model="cross-encoder/ms-marco-MiniLM-L-6-v2",
)

docs = [
    "irrelevant document",
    "highly relevant document",
    "somewhat relevant document",
]

reranked_docs, scores = reranker.rank(
    query="relevant document",
    docs=docs,
    limit=len(docs),
    return_score=True,
)

for doc, score in zip(reranked_docs, scores):
    print(score, doc)

The documents are returned in reranked order, but the scores remain ordered according to the original docs input.

Cause

The implementation first associates documents with their scores and correctly sorts them:

scores = self._client.predict(...)
scores = [float(score) for score in scores]

docs_with_scores = list(zip(doc_subset, scores))
docs_with_scores.sort(key=lambda x: x[1], reverse=True)

reranked_docs = [doc for doc, _ in docs_with_scores[:limit]]

However, the returned scores are then taken from the original unsorted scores list:

scores = scores[:limit]

if return_score:
    return reranked_docs, scores

Therefore only the documents reflect the sort operation.

Expected behavior

Documents and scores should be returned from the same sorted sequence, such that:

reranked_docs[i]

always corresponds to:

scores[i]

and the scores are ordered consistently with the reranked documents.

Proposed fix

Extract both documents and scores from the sorted docs_with_scores list:

ranked = docs_with_scores[:limit]

reranked_docs = [doc for doc, _ in ranked]
scores = [score for _, score in ranked]

if return_score:
    return reranked_docs, scores

Alternatively:

reranked_docs = [doc for doc, _ in docs_with_scores[:limit]]
scores = [score for _, score in docs_with_scores[:limit]]

Impact

Consumers that attach the returned reranker score to each result will associate incorrect scores with documents whenever the reranker changes the input ordering.

Using return_score=False avoids exposing the incorrect association, but does not solve the issue for applications that need reranker scores for downstream ranking, observability, debugging, or response metadata.

Related behavior

HFCrossEncoderReranker also defaults to:

limit=3

so callers that intend to rerank and return their full candidate set must explicitly provide an appropriate limit, such as limit=len(candidates).

That appears to be intentional API behavior rather than part of this bug.