fix(core): distinguish WebFetch URL errors from network errors#33393
Open
lexlian wants to merge 1 commit into
Open
fix(core): distinguish WebFetch URL errors from network errors#33393lexlian wants to merge 1 commit into
lexlian wants to merge 1 commit into
Conversation
WebFetch's URL validation collapsed every failure (malformed URL,
wrong scheme, transport failure, timeout, response too large, MIME
rejection, HTML conversion crash) into the same "Unable to fetch"
message. The model could not tell whether it sent a bad URL or hit
an unreachable host.
This commit introduces two-layer URL validation, matching Claude
Code's WebFetch shape:
1. Schema-level parseability check via Schema.makeFilter rejects
inputs that `new URL()` cannot parse. These surface through the
standard tool-input channel as
"Invalid tool input: Invalid URL\n at [\"url\"]".
2. Tool-layer scheme check (kept from the original assertHttpUrl)
rejects non-http/https schemes with a typed InvalidUrlError.
These surface as
"Invalid URL <url>: URL must use http:// or https://".
Network, timeout, body-size, MIME, and HTML-conversion failures
keep the generic "Unable to fetch <url>" message so existing tests
and downstream consumers do not break.
Replaces Effect.try({ catch: (error) => error }) at the validation
site with Effect.gen + Effect.fail(InvalidUrlError) so the tagged
error reaches the boundary mapper without erasure. Removes the
now-unused assertHttpUrl helper.
Adds two new tests (parse error, ftp scheme) and updates the
existing file:///etc/passwd test to expect the new typed message.
Closes anomalyco#33073
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes #33073
Type of change
What does this PR do?
WebFetch's URL validation collapses every failure (malformed URL, wrong scheme, transport failure, timeout, response too large, MIME rejection, HTML conversion crash) into the same
"Unable to fetch <url>"message. The model cannot tell whether it sent a bad URL or hit an unreachable host. The reporter's "Expected behavior" asks for a clear distinction between format errors and network errors.This PR introduces two-layer URL validation, matching Claude Code's WebFetch shape:
Schema-level parseability check via
Schema.makeFilteron theurlfield rejects inputs thatnew URL()cannot parse. These surface through the standard tool-input channel asInvalid tool input: Invalid URL\n at ["url"](the same path every other tool's schema rejection takes viaTool.make's built-inSchema.decodeUnknownstep inpackages/core/src/tool/tool.ts:82).Tool-layer scheme check (kept from the original
assertHttpUrl) rejects non-http/https schemes with a typedInvalidUrlError. These surface asInvalid URL <url>: URL must use http:// or https://, carrying the reason into the message so the model knows why the request was rejected.Network, timeout, body-size, MIME, and HTML-conversion failures keep the generic
Unable to fetch <url>message so existing tests and downstream consumers do not break.Implementation notes
Effect.try({ catch: (error) => error })at the validation site withEffect.gen+Effect.fail(InvalidUrlError)so the tagged error reaches the boundary mapper without erasure throughunknown → E. (Effect.try's catch returns its value as the failure type, but the surroundingEffect.mapError(() => …)was discarding it.)assertHttpUrlhelper (line 81-83 of the old file) — the same check is inlined in the newEffect.gen.Effect.try({ catch: (error) => error })site added by PR fix(core): bound web tool failures #33259 for HTML-to-Markdown conversion (lines 162-165) has the same generic-message anti-pattern but does not cause user-visible failures today. Left alone in this PR; flagged in the solution doc as a follow-up.Why two layers (and not just Schema-level)
Claude Code uses a JSON-schema
format: "url"constraint to reject unparseable URLs before the tool runs, and a separate tool-layer check for the http/https scheme. Two layers produce two distinct error categories — exactly the "format vs. network" distinction the bug report asks for. The tool-layer scheme check is also strictly stricter than Claude Code's (Claude Code leaksfile://to the network and returns a confusing socket error — cross-checked 2026-06-22).How did you verify your code works?
Manual verification: invoked
webfetchwithnot-a-url,ftp://example.com,file:///etc/passwd, and an unreachable host via the live tool:not-a-urlInvalid tool input: Invalid URL\n at ["url"](no permission, no network)ftp://example.comInvalid URL ftp://example.com: URL must use http:// or https://(no permission, no network)file:///etc/passwdInvalid URL file:///etc/passwd: URL must use http:// or https://(no permission, no network)Unable to fetch <url>(permission + network, as before)Screenshots / recordings
Not a UI change.
Checklist
Notes for the reviewer
file://rejection (fromUnable to fetch file:///etc/passwdtoInvalid URL file:///etc/passwd: URL must use http:// or https://). The rejection itself, the absence of permission/network calls, and the read-tool-only path for local files are all preserved unchanged (verified againstpackages/opencode/src/tool/read.ts:241-260andpackages/core/src/location-mutation.ts:37).DEVS/fixes/SOLUTION-33073.md("Cross-check: Claude Code WebFetch" and "Security context: local file access" sections).