fix: Warn when a requested timeout is capped at timeout_max - #962
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #962 +/- ##
==========================================
+ Coverage 94.64% 94.86% +0.22%
==========================================
Files 58 58
Lines 5263 5357 +94
==========================================
+ Hits 4981 5082 +101
+ Misses 282 275 -7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Pijukatel
left a comment
There was a problem hiding this comment.
Can you please verify in the API code that a timeout larger than 360 s has any meaning?
If there is some hard-coded or implementation limit on the API side, it makes no sense to lift the limit in the client.
(But a warning might be good in those cases)
|
Probably, there are mostly idle timeouts, not wall-clock timeouts, so this could make sense. See:
I don't want to waste much time here. IMO, silently truncating it is definitely wrong. If we don't want to allow longer timeouts, we should at least log a warning. However, allowing longer timeouts wouldn't be harmful anyway. I'll wait for your response, but I don't have a strong preference here. |
An explicit per-call
timeouttimedelta larger thantimeout_max(default 360 s), or a tier configured above it, is clamped totimeout_maxon every attempt. For example,dataset.get_items_as_bytes(timeout=timedelta(minutes=30))runs every attempt with a 360 s timeout and surfaces as repeatedimpit.TimeoutException, with no hint that the requested 30 minutes never took effect.The behavior stays unchanged:
timeout_maxremains a hard ceiling for any single request attempt. Only visibility changes._compute_timeoutnow logs a warning when the resolved base timeout exceedstimeout_max, naming both values and pointing attimeout_maxas the knob to raise.The warning goes through
LoggerOnce, a small dedup helper ported from Crawlee for Python, so a recurring cut-off is reported once per timeout kind rather than on every attempt of every call.Docs and docstrings now state that the cap applies to tier and per-call timeouts alike.
✍️ Drafted by Claude Code