From b9991636b70568a9386f9f4adc9a98346afc8145 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Eric=20Meadows-J=C3=B6nsson?= Date: Tue, 21 Jul 2026 19:00:18 +0200 Subject: [PATCH 1/2] Use the request target as the CONNECT :authority in Mint.HTTP2 Mint.HTTP2 hardcoded the :authority pseudo-header of CONNECT requests to the connection's own authority and ignored the path argument, so a CONNECT could only target the server itself. Per RFC 9113 section 8.5 the request target of a CONNECT is the host and port of the tunnel destination, which request/5 now takes from its path argument, mirroring HTTP/1 (HTTP1.request(conn, "CONNECT", "host:443", ...)). Extended CONNECT (RFC 8441) is unchanged: when the headers contain a :protocol pseudo-header, :authority stays the connection authority and the caller passes :scheme and :path explicitly, which is how mint_web_socket drives WebSocket over HTTP/2. The new tests also pin the tunnel stream lifecycle: bytes flow in both directions on the open stream, a content-length header on the 2xx response doesn't end the tunnel, a server END_STREAM closes the whole tunnel, and a client :eof half-closes it while receiving keeps working. The docs describe CONNECT tunnels in Mint.HTTP2.request/5 and how to build HTTP/2 proxying on top of them in the Proxying section. --- lib/mint/http.ex | 39 +++++++ lib/mint/http2.ex | 44 +++++++- test/mint/http2/conn_test.exs | 190 ++++++++++++++++++++++++++++++++-- 3 files changed, 261 insertions(+), 12 deletions(-) diff --git a/lib/mint/http.ex b/lib/mint/http.ex index 45d65f78..1c1de9ee 100644 --- a/lib/mint/http.ex +++ b/lib/mint/http.ex @@ -296,6 +296,39 @@ defmodule Mint.HTTP do this time, `connect/4` returns a `Mint.HTTPError` with reason `{:proxy, :tunnel_timeout}`. Defaults to `30_000` (30 seconds). + ### Proxying over HTTP/2 + + The `:proxy` option always speaks HTTP/1 to the proxy. The connection tunneled + *through* the proxy can still be HTTP/2 (negotiated via ALPN), but the `CONNECT` + exchange with the proxy itself is HTTP/1. Mint doesn't drive an HTTP/2 connection + to the proxy from `connect/4` on purpose: the benefit of HTTP/2 `CONNECT` is + multiplexing many tunnels over a single proxy connection, which means sharing that + connection across Mint connections. That is the job of a connection pool built on + top of Mint, not of a single connection. + + To tunnel through a proxy over HTTP/2, connect to the proxy with `Mint.HTTP2` and + open one `CONNECT` stream per tunnel (see the "CONNECT requests" section in + `Mint.HTTP2.request/5`): + + {:ok, conn} = Mint.HTTP2.connect(:https, proxy_host, proxy_port) + {:ok, conn, ref} = Mint.HTTP2.request(conn, "CONNECT", "example.com:443", [], :stream) + + Once the proxy replies with a `{:status, ref, 200}` response, the stream is a byte + tunnel: send bytes to the target with `Mint.HTTP2.stream_request_body/3` and receive + bytes from the target as `{:data, ref, data}` responses. + + To run TLS (or a whole Mint connection) *inside* such a tunnel, you need a relay: + `:ssl` can't run over an arbitrary byte stream, so a process has to own the proxy + connection and relay bytes between the `CONNECT` stream and a local TCP socket. The + inner connection then dials the relay while keeping the identity of the real host + through the `:hostname` option: + + Mint.HTTP.connect(:https, relay_address, relay_port, hostname: "example.com") + + so that SNI, certificate verification, and the request `Host` header all target the + real host. This is the natural shape for pooling libraries, which can multiplex many + tunnels over one proxy connection. + ## Transport options The options specified in `:transport_opts` are passed to the module that @@ -611,6 +644,12 @@ defmodule Mint.HTTP do `content-length` header yourself. If you're using HTTP/1, Mint will do chunked transfer-encoding when a content-length is not provided (see `Mint.HTTP1.request/5`). + ## CONNECT requests + + For requests with the `CONNECT` method, `path` is the target of the tunnel in + *authority form* (`"host:port"`), not a path. See `Mint.HTTP2.request/5` for HTTP/2 + tunnel semantics, including the extended CONNECT protocol. + ## Examples Mint.HTTP.request(conn, "GET", "/", _headers = [], _body = nil) diff --git a/lib/mint/http2.ex b/lib/mint/http2.ex index 51788816..bc321bd2 100644 --- a/lib/mint/http2.ex +++ b/lib/mint/http2.ex @@ -519,6 +519,36 @@ defmodule Mint.HTTP2 do body by initially passing `:stream` as the body and sending the body in chunks using `stream_request_body/3` and using `get_window_size/2` to get the window size of the request and connection. + + ## CONNECT requests + + You can open a tunnel to a target host through the server with the `CONNECT` + method ([RFC 9113, section 8.5](https://www.rfc-editor.org/rfc/rfc9113.html#section-8.5)). + For `CONNECT` requests, `path` is not a path: it's the host and port of the tunnel + target, such as `"example.com:443"` (IPv6 addresses must be enclosed in square + brackets). Mint sends it as the `:authority` pseudo-header and omits the `:scheme` + and `:path` pseudo-headers. *Available since v1.10.0*. + + Pass `:stream` as the body. Once the server replies with a 2xx status, the stream + becomes a tunnel: iodata you pass to `stream_request_body/3` is delivered to the + tunnel target, and bytes coming from the target are returned by `stream/2` as + `{:data, request_ref, data}` responses. The tunnel counts towards the limit of + concurrent requests for as long as it's open. + + Mint doesn't support the TCP-style half-close that RFC 9113 describes for tunnels: + when the server ends its side of the stream, Mint considers the whole tunnel closed, + returns `{:done, request_ref}`, and doesn't let you send more data for that request. + If instead you end your side first (by passing `:eof` to `stream_request_body/3`), + you keep receiving data until the server ends the stream. + + ### Extended CONNECT + + Mint also supports the extended CONNECT protocol + ([RFC 8441](https://www.rfc-editor.org/rfc/rfc8441.html)), which is used to bootstrap + protocols such as WebSocket over HTTP/2. If you pass a `:protocol` pseudo-header in + `headers`, the request is treated as an extended CONNECT request: you're expected to + also pass the `:scheme` and `:path` pseudo-headers explicitly, and Mint sets + `:authority` to the connection's authority, as in a normal request. """ @impl true @spec request( @@ -1606,9 +1636,21 @@ defmodule Mint.HTTP2 do defp add_pseudo_headers(headers, conn, method, path) do if same_method?(method, "CONNECT") do + # In extended CONNECT (RFC 8441) the caller passes :scheme, :path, and + # :protocol explicitly and :authority is the origin, as in a normal + # request. In plain CONNECT (RFC 9113, section 8.5) the request target + # is the host and port of the tunnel destination, which request/5 + # receives as its "path" argument. + authority = + if List.keymember?(headers, ":protocol", 0) do + conn.authority + else + path + end + [ {":method", method}, - {":authority", conn.authority} + {":authority", authority} | headers ] else diff --git a/test/mint/http2/conn_test.exs b/test/mint/http2/conn_test.exs index e7f85054..b323be46 100644 --- a/test/mint/http2/conn_test.exs +++ b/test/mint/http2/conn_test.exs @@ -1151,18 +1151,18 @@ defmodule Mint.HTTP2Test do assert cookie == "a=b; c=d; e=f; g=h" end - test "a CONNECT request omits :scheme and :path pseudo-headers", %{conn: conn} do - assert {:ok, conn, _ref} = HTTP2.request(conn, "CONNECT", "/", [], nil) + test "a CONNECT request uses path as :authority and omits :scheme and :path", %{conn: conn} do + assert {:ok, conn, _ref} = HTTP2.request(conn, "CONNECT", "example.com:443", [], :stream) - assert_recv_frames [headers(hbf: hbf)] + assert_recv_frames [headers(hbf: hbf) = frame] - refute hbf - |> server_decode_headers() - |> List.keymember?(":scheme", 0) + assert [ + {":method", "CONNECT"}, + {":authority", "example.com:443"}, + {"user-agent", _} + ] = server_decode_headers(hbf) - refute hbf - |> server_decode_headers() - |> List.keymember?(":path", 0) + refute flag_set?(headers(frame, :flags), :headers, :end_stream) assert HTTP2.open?(conn) end @@ -1176,13 +1176,15 @@ defmodule Mint.HTTP2Test do {":protocol", "websocket"} ] - assert {:ok, conn, _ref} = HTTP2.request(conn, "CONNECT", "/", headers, :stream) + assert {:ok, conn, _ref} = HTTP2.request(conn, "CONNECT", "/ws", headers, :stream) assert_recv_frames [headers(hbf: hbf)] + connection_authority = conn.authority + assert [ {":method", "CONNECT"}, - {":authority", _}, + {":authority", ^connection_authority}, {":scheme", _}, {":path", "/ws"}, {":protocol", "websocket"}, @@ -2408,6 +2410,172 @@ defmodule Mint.HTTP2Test do end end + describe "CONNECT tunnels" do + test "tunnels bytes in both directions", %{conn: conn} do + assert {:ok, conn, ref} = HTTP2.request(conn, "CONNECT", "example.com:80", [], :stream) + + assert_recv_frames [headers(stream_id: stream_id)] + + hbf = server_encode_headers([{":status", "200"}]) + + assert {:ok, %HTTP2{} = conn, [{:status, ^ref, 200}, {:headers, ^ref, []}]} = + stream_frames(conn, [ + headers( + stream_id: stream_id, + hbf: hbf, + flags: set_flags(:headers, [:end_headers]) + ) + ]) + + assert {:ok, conn} = HTTP2.stream_request_body(conn, ref, "hello") + assert_recv_frames [data(stream_id: ^stream_id, data: "hello") = data_frame] + refute flag_set?(data(data_frame, :flags), :data, :end_stream) + + assert {:ok, %HTTP2{} = conn, [{:data, ^ref, "world"}]} = + stream_frames(conn, [ + data(stream_id: stream_id, data: "world", flags: set_flags(:data, [])) + ]) + + assert {:ok, conn} = HTTP2.stream_request_body(conn, ref, "again") + assert_recv_frames [data(stream_id: ^stream_id, data: "again")] + + assert HTTP2.open_request_count(conn) == 1 + assert HTTP2.open?(conn) + end + + test "content-length: 0 on the response does not end the tunnel", %{conn: conn} do + assert {:ok, conn, ref} = HTTP2.request(conn, "CONNECT", "example.com:443", [], :stream) + + assert_recv_frames [headers(stream_id: stream_id)] + + hbf = server_encode_headers([{":status", "200"}, {"content-length", "0"}]) + + assert {:ok, %HTTP2{} = conn, responses} = + stream_frames(conn, [ + headers( + stream_id: stream_id, + hbf: hbf, + flags: set_flags(:headers, [:end_headers]) + ), + data(stream_id: stream_id, data: "tunneled", flags: set_flags(:data, [])) + ]) + + assert responses == [ + {:status, ref, 200}, + {:headers, ref, [{"content-length", "0"}]}, + {:data, ref, "tunneled"} + ] + + assert {:ok, conn} = HTTP2.stream_request_body(conn, ref, "out") + assert_recv_frames [data(stream_id: ^stream_id, data: "out")] + + assert HTTP2.open_request_count(conn) == 1 + assert HTTP2.open?(conn) + end + + test "an END_STREAM from the server closes the whole tunnel", %{conn: conn} do + assert {:ok, conn, ref} = HTTP2.request(conn, "CONNECT", "example.com:443", [], :stream) + + assert_recv_frames [headers(stream_id: stream_id)] + + hbf = server_encode_headers([{":status", "200"}]) + + assert {:ok, %HTTP2{} = conn, responses} = + stream_frames(conn, [ + headers( + stream_id: stream_id, + hbf: hbf, + flags: set_flags(:headers, [:end_headers]) + ), + data(stream_id: stream_id, data: "bye", flags: set_flags(:data, [:end_stream])) + ]) + + assert responses == [ + {:status, ref, 200}, + {:headers, ref, []}, + {:data, ref, "bye"}, + {:done, ref} + ] + + assert_recv_frames [rst_stream(stream_id: ^stream_id, error_code: :no_error)] + + assert {:error, %HTTP2{} = conn, error} = HTTP2.stream_request_body(conn, ref, "x") + assert_http2_error error, :unknown_request_to_stream + + assert HTTP2.open_request_count(conn) == 0 + assert HTTP2.open?(conn) + end + + test "an :eof from the client half-closes the tunnel and data can still be received", + %{conn: conn} do + assert {:ok, conn, ref} = HTTP2.request(conn, "CONNECT", "example.com:443", [], :stream) + + assert_recv_frames [headers(stream_id: stream_id)] + + hbf = server_encode_headers([{":status", "200"}]) + + assert {:ok, %HTTP2{} = conn, [{:status, ^ref, 200}, {:headers, ^ref, []}]} = + stream_frames(conn, [ + headers( + stream_id: stream_id, + hbf: hbf, + flags: set_flags(:headers, [:end_headers]) + ) + ]) + + assert {:ok, conn} = HTTP2.stream_request_body(conn, ref, "last") + assert {:ok, conn} = HTTP2.stream_request_body(conn, ref, :eof) + + assert_recv_frames [ + data(stream_id: ^stream_id, data: "last") = data1, + data(stream_id: ^stream_id, data: "") = data2 + ] + + refute flag_set?(data(data1, :flags), :data, :end_stream) + assert flag_set?(data(data2, :flags), :data, :end_stream) + + assert {:ok, %HTTP2{} = conn, [{:data, ^ref, "still coming"}]} = + stream_frames(conn, [ + data(stream_id: stream_id, data: "still coming", flags: set_flags(:data, [])) + ]) + + assert {:ok, %HTTP2{} = conn, [{:data, ^ref, ""}, {:done, ^ref}]} = + stream_frames(conn, [ + data(stream_id: stream_id, data: "", flags: set_flags(:data, [:end_stream])) + ]) + + assert_recv_frames [] + + assert HTTP2.open_request_count(conn) == 0 + assert HTTP2.open?(conn) + end + + test "a CONNECT request with a nil body opens a receive-only tunnel", %{conn: conn} do + assert {:ok, conn, ref} = HTTP2.request(conn, "CONNECT", "example.com:443", [], nil) + + assert_recv_frames [headers(stream_id: stream_id) = headers_frame] + assert flag_set?(headers(headers_frame, :flags), :headers, :end_stream) + + hbf = server_encode_headers([{":status", "200"}]) + + assert {:ok, %HTTP2{} = conn, + [{:status, ^ref, 200}, {:headers, ^ref, []}, {:data, ^ref, "in"}]} = + stream_frames(conn, [ + headers( + stream_id: stream_id, + hbf: hbf, + flags: set_flags(:headers, [:end_headers]) + ), + data(stream_id: stream_id, data: "in", flags: set_flags(:data, [])) + ]) + + assert {:error, %HTTP2{} = conn, error} = HTTP2.stream_request_body(conn, ref, "x") + assert_http2_error error, :request_is_not_streaming + + assert HTTP2.open?(conn) + end + end + describe "stream_request_body/3" do test "streaming a request", %{conn: conn} do {conn, ref} = open_request(conn, :stream) From b59b2d765c97ead629fdc0c1177d7557dc45a05e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Eric=20Meadows-J=C3=B6nsson?= Date: Tue, 21 Jul 2026 19:19:37 +0200 Subject: [PATCH 2/2] Correct the CONNECT tunnel docs The relay pattern in the Proxying section claimed the request Host header targets the real host; the default Host header and :authority contain the connection port, which is the relay's port, so the docs now say so and give the workarounds. Also qualify that the path argument doesn't apply to extended CONNECT requests, and note that servers gate extended CONNECT behind the :enable_connect_protocol setting, which Mint doesn't enforce. --- lib/mint/http.ex | 11 +++++++---- lib/mint/http2.ex | 16 ++++++++++------ 2 files changed, 17 insertions(+), 10 deletions(-) diff --git a/lib/mint/http.ex b/lib/mint/http.ex index 1c1de9ee..277ac4a5 100644 --- a/lib/mint/http.ex +++ b/lib/mint/http.ex @@ -325,9 +325,12 @@ defmodule Mint.HTTP do Mint.HTTP.connect(:https, relay_address, relay_port, hostname: "example.com") - so that SNI, certificate verification, and the request `Host` header all target the - real host. This is the natural shape for pooling libraries, which can multiplex many - tunnels over one proxy connection. + so that SNI and certificate verification target the real host. Note that the default + request `Host` header (HTTP/1) and `:authority` pseudo-header (HTTP/2) contain the + port of the connection, which is the relay's port: either make the relay listen on + the target's port, or (on HTTP/1) pass an explicit `host` header with each request. + This is the natural shape for pooling libraries, which can multiplex many tunnels + over one proxy connection. ## Transport options @@ -648,7 +651,7 @@ defmodule Mint.HTTP do For requests with the `CONNECT` method, `path` is the target of the tunnel in *authority form* (`"host:port"`), not a path. See `Mint.HTTP2.request/5` for HTTP/2 - tunnel semantics, including the extended CONNECT protocol. + tunnel semantics and for the extended CONNECT protocol, where `path` doesn't apply. ## Examples diff --git a/lib/mint/http2.ex b/lib/mint/http2.ex index bc321bd2..21527787 100644 --- a/lib/mint/http2.ex +++ b/lib/mint/http2.ex @@ -524,10 +524,11 @@ defmodule Mint.HTTP2 do You can open a tunnel to a target host through the server with the `CONNECT` method ([RFC 9113, section 8.5](https://www.rfc-editor.org/rfc/rfc9113.html#section-8.5)). - For `CONNECT` requests, `path` is not a path: it's the host and port of the tunnel - target, such as `"example.com:443"` (IPv6 addresses must be enclosed in square - brackets). Mint sends it as the `:authority` pseudo-header and omits the `:scheme` - and `:path` pseudo-headers. *Available since v1.10.0*. + For `CONNECT` requests (other than extended CONNECT, see below), `path` is not a + path: it's the host and port of the tunnel target, such as `"example.com:443"` + (IPv6 addresses must be enclosed in square brackets). Mint sends it as the + `:authority` pseudo-header and omits the `:scheme` and `:path` pseudo-headers. + *Available since v1.10.0*. Pass `:stream` as the body. Once the server replies with a 2xx status, the stream becomes a tunnel: iodata you pass to `stream_request_body/3` is delivered to the @@ -547,8 +548,11 @@ defmodule Mint.HTTP2 do ([RFC 8441](https://www.rfc-editor.org/rfc/rfc8441.html)), which is used to bootstrap protocols such as WebSocket over HTTP/2. If you pass a `:protocol` pseudo-header in `headers`, the request is treated as an extended CONNECT request: you're expected to - also pass the `:scheme` and `:path` pseudo-headers explicitly, and Mint sets - `:authority` to the connection's authority, as in a normal request. + also pass the `:scheme` and `:path` pseudo-headers explicitly (the `path` argument is + not used), and Mint sets `:authority` to the connection's authority, as in a normal + request. Note that servers only allow extended CONNECT after advertising the + `:enable_connect_protocol` setting (see `get_server_setting/2`); Mint doesn't + enforce this, so it's up to you to check the setting first. """ @impl true @spec request(