What happened
limit=0 on the gRPC query path is dropped before it reaches the server, so
the server applies QUERY_DEFAULTS_LIMIT instead of returning nothing.
With 30 objects in a collection and the server started with
QUERY_DEFAULTS_LIMIT=25:
| call |
returned |
collection.query.fetch_objects(limit=0) |
25 objects |
collection.query.fetch_objects(limit=1) |
1 |
collection.query.fetch_objects(limit=2) |
2 |
collection.query.near_vector(..., limit=0) |
25 |
collection.query.hybrid(..., limit=0) |
25 |
GET /v1/objects?class=X&limit=0 |
0 objects |
GraphQL Difftest(limit: 0) |
error: invalid pagination params: invalid default limit: 0 |
bm25 is unaffected, it fails earlier on a different validation path.
Versions
- weaviate-client 4.23.0
- weaviate server 1.38.13 (
semitechnologies/weaviate:latest)
Reproduction
docker run -d -p 8080:8080 -p 50051:50051 \
-e AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=true \
-e PERSISTENCE_DATA_PATH=/var/lib/weaviate \
-e DEFAULT_VECTORIZER_MODULE=none \
-e QUERY_DEFAULTS_LIMIT=25 \
semitechnologies/weaviate:latest
import weaviate
from weaviate.classes.config import Configure, DataType, Property
client = weaviate.connect_to_local()
coll = client.collections.create(
name="LimitZero",
properties=[Property(name="n", data_type=DataType.INT)],
vectorizer_config=Configure.Vectorizer.none(),
)
with coll.batch.fixed_size(batch_size=30) as batch:
for i in range(30):
batch.add_object(properties={"n": i}, vector=[float(i), 0.0, 0.0])
print(len(coll.query.fetch_objects(limit=0).objects)) # 25, expected 0
client.close()
Why it happens
SearchRequest.limit is a proto3 uint32 with no field presence, so 0 is the
type default and is not serialised:
from weaviate.proto.v1 import search_get_pb2
search_get_pb2.SearchRequest(collection="X", limit=0).SerializeToString()
# b'\n\x01X'
search_get_pb2.SearchRequest(collection="X", limit=2).SerializeToString()
# b'\n\x01X\xf0\x01\x02'
limit=0 is byte-identical to sending no limit at all.
I want to be clear that I think this part is intended, not a wire bug.
grpc/proto/v1/search_get.proto in weaviate/weaviate says so:
// affects order and length of results. 0/empty (default value) means disabled
uint32 limit = 30;
uint32 offset = 31;
uint32 autocut = 32;
So 0 meaning "disabled" is a deliberate convention shared with offset and
autocut, and I am not proposing changing the proto.
What I think the actual problem is
The Python signature is limit: Optional[int] = None. None already means
"no limit", so there is nothing in the Python API to suggest 0 is a second
spelling of the same thing. The convention is documented in a .proto file
that a Python user has no reason to open, and it is not in the docstring or
the type.
A caller passing a computed limit — min(page_size, remaining),
wanted - already_fetched, a quota that reaches zero — asks for no objects and
receives a full default page, with no error and no warning.
The client already rejects other degenerate query inputs. contains_any([])
raises WeaviateInvalidInputError: Filter contains_any must have at least one value. limit=0 gets no such treatment.
There is also existing precedent for guarding this exact class of silent
coercion. weaviate/validator.py carries this from #2088:
# bool is a subclass of int, so isinstance(True, int) is True. Reject a
# bool where a plain int is expected so it is not silently coerced to 0/1.
What would you prefer
Two options, and I do not want to pick one for you:
- Reject
limit=0 with WeaviateInvalidInputError inside the existing
if self._validate_arguments: block in collections/grpc/query.py, the way
contains_any([]) is handled. This is a behaviour change for anyone
currently relying on 0 meaning "disabled" from Python.
- Document it — state in the docstring that
0 means no limit, same as None.
I have option 1 written with a test in test_bad_query_inputs, and the unit
suite passes. Happy to open a PR for either once you say which you want, or to
drop it if you would rather leave the behaviour alone.
What happened
limit=0on the gRPC query path is dropped before it reaches the server, sothe server applies
QUERY_DEFAULTS_LIMITinstead of returning nothing.With 30 objects in a collection and the server started with
QUERY_DEFAULTS_LIMIT=25:collection.query.fetch_objects(limit=0)collection.query.fetch_objects(limit=1)collection.query.fetch_objects(limit=2)collection.query.near_vector(..., limit=0)collection.query.hybrid(..., limit=0)GET /v1/objects?class=X&limit=0Difftest(limit: 0)invalid pagination params: invalid default limit: 0bm25is unaffected, it fails earlier on a different validation path.Versions
semitechnologies/weaviate:latest)Reproduction
Why it happens
SearchRequest.limitis a proto3uint32with no field presence, so0is thetype default and is not serialised:
limit=0is byte-identical to sending no limit at all.I want to be clear that I think this part is intended, not a wire bug.
grpc/proto/v1/search_get.protoin weaviate/weaviate says so:// affects order and length of results. 0/empty (default value) means disabled uint32 limit = 30; uint32 offset = 31; uint32 autocut = 32;So
0meaning "disabled" is a deliberate convention shared withoffsetandautocut, and I am not proposing changing the proto.What I think the actual problem is
The Python signature is
limit: Optional[int] = None.Nonealready means"no limit", so there is nothing in the Python API to suggest
0is a secondspelling of the same thing. The convention is documented in a
.protofilethat a Python user has no reason to open, and it is not in the docstring or
the type.
A caller passing a computed limit —
min(page_size, remaining),wanted - already_fetched, a quota that reaches zero — asks for no objects andreceives a full default page, with no error and no warning.
The client already rejects other degenerate query inputs.
contains_any([])raises
WeaviateInvalidInputError: Filter contains_any must have at least one value.limit=0gets no such treatment.There is also existing precedent for guarding this exact class of silent
coercion.
weaviate/validator.pycarries this from #2088:What would you prefer
Two options, and I do not want to pick one for you:
limit=0withWeaviateInvalidInputErrorinside the existingif self._validate_arguments:block incollections/grpc/query.py, the waycontains_any([])is handled. This is a behaviour change for anyonecurrently relying on
0meaning "disabled" from Python.0means no limit, same asNone.I have option 1 written with a test in
test_bad_query_inputs, and the unitsuite passes. Happy to open a PR for either once you say which you want, or to
drop it if you would rather leave the behaviour alone.