diff --git a/issues/toyos-has-its-own-network-stack.md b/issues/toyos-has-its-own-network-stack.md index 8a5b0ede3b8..a7a21843c6f 100644 --- a/issues/toyos-has-its-own-network-stack.md +++ b/issues/toyos-has-its-own-network-stack.md @@ -17,7 +17,7 @@ netd runs smoltcp. Its replacement is ToyOS's own stack, built clean-room: reade - Then multi-core, netring (blocked on the owner's ABI ruling), TCP and IP hardening, IPv6, offloads, soak. - This track owns the T14's outbound rows, the machine reaching its router and the internet on the I219, judged from the stick: they arrive on this stack and not on smoltcp (owner: "no smoltcp."), and until they do no T14 row reads the wired card (`issues/the-host-cannot-reach-the-t14-while-it-runs-toyos.md`). -Stage 5 lands in slices, the orchestrator's cut under the owner's words "i want smoltcp out as fast as possible": netd's decisions go into `toyos-net-node` (`userland/netstack/node`), pure and host-tested and shipped in nothing, and then one change moves netd onto it and deletes smoltcp. In the tree: the node, its DHCP lease, its clients' datagram sockets, its mDNS name, its resolver, streams, listeners and the one bound on its clients' streams, listeners and datagram sockets (`userland/netstack/node/src/places.rs`); a lookup's sockets stand outside that bound, under `toyos_dns::MAX_LOOKUPS` and its rounds. Still to build on it: the datagram sockets' broadcast permission; then the move. +Stage 5 lands in slices, the orchestrator's cut under the owner's words "i want smoltcp out as fast as possible": netd's decisions go into `toyos-net-node` (`userland/netstack/node`), pure and host-tested and shipped in nothing, and then one change moves netd onto it and deletes smoltcp. In the tree: the node, its DHCP lease, its clients' datagram sockets, its mDNS name, its resolver, streams, listeners and the one bound on its clients' streams, listeners and datagram sockets (`userland/netstack/node/src/places.rs`); a lookup's sockets stand outside that bound, under `toyos_dns::MAX_LOOKUPS` and its rounds. Still to build on it: datagram senders that wait on the hop, which is [udp]'s way to [ip] through the node's `lib.rs`, `lease.rs` and `toyos-net-shard/src/lib.rs`; the datagram sockets' broadcast permission; then the move. What the node does not yet meet: @@ -40,8 +40,8 @@ What the node does not yet meet: - A refusal [udp] logs against a client's call waits in the stack until the node's next `receive`, `fire` or `link`, because only those pass over the stack's reports. Exit, at the move and not after it: the datagram calls end in that pass, or netd drains it once per turn of its loop. - The machine's name is told to the node twice, as `toyos_dhcp::HostName` in `Node::new` and as `toyos_mdns::Host` in `Node::answer_as`: the first owns its bytes and the second borrows them. Exit: one type carries the label to both; due at the move if handing `answer_as` its `&'static str` takes a leak. - The node's name is not probed for or defended (RFC 6762 §8.1, §9), as `toyos-mdns` records of itself. It is not announced again when the link comes back, which §8.3 asks and netd does today, where link-up resets its DHCP client and the address is new again: here the lease outlives the link. Exit: `toyos-mdns` does both, on the node's `link`. -- A query that does not reach its resolver is waited out for `toyos_dns::WAIT_MS` like one nobody answered, whether [udp] refused it for want of a route (`node.query-unsent`) or the network reported it back, its resolver unreachable or its port refused (`node.query-failed`): `toyos_dns::Lookup` has no word for either. Exit, before the move: `Lookup` is told and asks the next resolver at once, `a_query_udp_refuses_is_counted_holds_no_port_and_is_waited_out` finds its answer at 0 ms, and `a_query_the_network_reports_unreachable_is_counted_and_waited_out` finds its lookup ended at 3 s, [ip]'s first report of its one resolver, and not at `ROUNDS` × `WAIT_MS`. -- Lookups started together at a resolver whose link address is not yet known lose all but the newest `PENDING_PER_NEIGHBOUR` queries to [ip]'s queue for an unresolved next hop (`nb.pending-overflow`, RFC 4861 §7.2.2), and each lost one costs its lookup a `WAIT_MS`; smoltcp kept each query in its own socket until the neighbour answered. `a_burst_at_a_resolver_not_yet_resolved_leaves_ips_queue_of_queries_and_the_rest_a_wait_later` holds it as it stands. It is a regression against what ships and would ship at the move. Exit, before the move: that test finds every query of the burst on the wire once the resolver answers ARP. +- A query [ip] refuses at the transmit opportunity that would have carried it, its next hop held failed or its table full, is reported to its socket inside `Node::transmit`, which carries no lookup on: the resolver reads the report at the node's next `receive`, `fire` or `link`, which at the latest is the end of the query's wait, where a query [udp] refuses in the resolver's own call, or one reported in a frame or at a deadline, ends its wait at once. `a_query_ip_refuses_at_its_transmit_opportunity_is_read_at_the_nodes_next_wake` holds it as it stands. It is no regression: what ships waits every such query out. Until that read, an ICMP error that quotes the query's addresses and ports takes [ip]'s word's place on the socket, [udp] holding one pending error a socket, the latest: the query is then waited out and not let go, which costs the early end and never an answer. Owner: the slice in which datagram senders wait on the hop (`Node::transmit` is the node's `lib.rs`). Exit: the node carries its lookups on after the stack's transmit, or [udp] refuses in the call a datagram whose next hop [ip] holds failed, so that no report waits on a socket to be displaced, and that test finds its lookup ended at the 4 s it started. +- [ip]'s queue for a next hop it is resolving is every sender's: `PENDING_PER_NEIGHBOUR` is 16, a stopgap which the node's resolver asserts is no fewer than `toyos_dns::MAX_LOOKUPS`, so every held lookup's first query waits there whole, but a datagram a client sends to the same next hop in that window, which for everything off the link is the router, takes the oldest one's place (`nb.pending-overflow`, RFC 4861 §7.2.2), and a query lost so costs its lookup a `WAIT_MS`; smoltcp kept each datagram in its own socket until the neighbour answered. `a_clients_datagram_behind_the_burst_takes_the_oldest_querys_place_in_ips_queue` holds it as it stands. It is a regression against what ships for more than 16 datagrams at one next hop not yet resolved, and would ship at the move; and it sizes a limit of [ip]'s by a constant of the resolver's, which leaves four unresolved next hops filling `PENDING_TOTAL` where eight did. Owner: the slice in which datagram senders wait on the hop, where [udp]'s datagram calls already go through `Stack` and `Shard`. Its design: [udp]'s `serve` asks [ip] the next-hop question before it hands a datagram over, as [tcp]'s exit does with `Hop` (NUD-25, IP-D5); a socket whose oldest datagram's hop is pending keeps it in its own queue, which `TX_DATAGRAMS` and `TX_BYTES` already bound and refuse in the call, and is offered again when the hop resolves and told when it fails; [ip] then queues only its own datagrams. Exit, before the move: that test finds every query on the wire once the resolver answers ARP; `PENDING_PER_NEIGHBOUR` is `limits.ip.nud.pending_per_neighbour`'s 8 again; and the resolver's compile-time assertion, the change to `s_ip_nud_004_overflow_drops_the_oldest` and stage 3's departure line about it are deleted. - `toyos_dns::Lookup::on_datagram` still takes a reply's source address and port, which the node's connected sockets have already matched and the node hands it from its own record of the query: the two parameters are read only by netd's resolver on smoltcp. Exit, at the move: they go with that resolver. - `node.address-refused` has no test: `toyos-dhcp` accepts no address or prefix [ip] refuses, by the same `toyos-net-wire` checks in both, so the refusal cannot be reached from the wire. Exit: the client hands [ip] a type that carries the check, and the counter goes. @@ -54,6 +54,7 @@ What stage 3 departs from its specifications: - US-43 expects `ip.not-for-us` for a datagram to an unjoined group in a frame to that group's MAC; the frame filter refuses it first, as `eth.not-for-us` (ETH-23). Exit: the scenario names the frame filter. - DH-64 expects an announce request from the client on link-up with a lease held; `toyos-net-ip` announces the held address itself on link-up, by the owner's ruling to announce rather than probe again, so the client asks nothing. Exit: the scenario drops the request, or [ip] stops announcing on its own. - The readers' UDP specification lets every datagram a closed socket had accepted leave; `toyos-net-udp` holds at most one socket's queue of them together and refuses the rest `udp.tx-discarded-on-close`, because the readers' stage 4 design holds nothing in the shard without a bound. Exit: `s_udp_us_053_accepted_datagrams_outlive_close` finds `udp.tx-discarded-on-close` at 0, every datagram a closed socket accepted leaving. +- NUD-04 sends ten datagrams and finds the third to the tenth, two overflowed, under `limits.ip.nud.pending_per_neighbour`, 8; `PENDING_PER_NEIGHBOUR` is 16, so that a burst of every lookup the node holds waits whole for its resolver's link address, and `s_ip_nud_004_overflow_drops_the_oldest` sends two more than the constant: it holds the rule, the oldest dropped and counted, and no longer the scenario's numbers. NUD-05 passes unchanged only because its eight neighbours of eight datagrams are `PENDING_TOTAL`, which did not move. Exit: that of the node's residual on [ip]'s queue above, in the slice in which datagram senders wait on the hop: the constant is 8, and the test sends NUD-04's ten. - The readers' IP specification has no scenario for 169.254/16 on an interface that holds a routable address: its routing scenarios are RTE-01 to RTE-17 and its ARP reception scenarios NBR-01 to NBR-18, none names that prefix, and NBR-12's `arp.sender-off-link` is expected of a sender in no prefix at all, which its 198.51.100.9 still is. Its input scenarios are IN-01 to IN-09, of which none names 169.254 or a link-local source: IN-05, the nearest, refuses a source that is no host or an edge of a held prefix, and [ip]'s refusal of a source at an edge of 169.254/16 on every interface, `ip.invalid-source`, is an addition to that step's list. It has none either for two next hops of which one never answers: NUD-04 overflows one neighbour's queue and NUD-05 fills the interface's, and in neither does a second neighbour answer. The tests that hold the two rules therefore name their RFC section and carry no id: the `rfc_3927_…` tests in `toyos-net-ip/tests/addr.rs` and `nbr.rs`, and `rfc_4861_7_2_2_a_silent_next_hop_holds_up_no_other` in `tests/nud.rs`. Exit: the readers add a scenario for each, and each test carries its id. What stage 3 does not yet meet: diff --git a/toyos-dns/src/lib.rs b/toyos-dns/src/lib.rs index 1f46cdaa91e..1712abefd18 100644 --- a/toyos-dns/src/lib.rs +++ b/toyos-dns/src/lib.rs @@ -431,8 +431,12 @@ pub enum Failure { NoSuchName, /// The name exists and has no `A` record (RFC 2308 §2.2). NoAddress, - /// Every query went unanswered for [`WAIT_MS`]. + /// No server answered, and a query that may have reached one was given + /// its [`WAIT_MS`]. TimedOut, + /// No query reached a server ([`Lookup::on_unreached`]) and none is left + /// to send at once: there is no answer to wait for. + Unreachable, /// The answer did not fit a UDP reply (TC). Truncated, /// Every server that answered could not answer, the last with this RCODE. @@ -478,21 +482,49 @@ struct Sent { /// any query this lookup sent for the name now asked is read, so an answer /// late past its query's wait still ends the lookup; it is read only on the /// query's own port, with the query's ID, from the server the query went to. +/// +/// **A wait is for an answer that can come.** When the newest query is +/// answered with a failure, or known or said not to have reached its server, +/// the next server is asked at once: RFC 1035 §4.2.1's interval of two to +/// five seconds is between repetitions of a query that may have been lost on +/// its way, and the same section has the other servers tried first. **The +/// interval is each server's**: one that has not answered is sent no second +/// query inside [`WAIT_MS`] of its last, or of the caller's last knowledge +/// that it was not reached, whatever is said of that query and whatever +/// another server answers; a server is its address, however often the list +/// names it. Where the next server cannot be asked yet, the lookup waits +/// until it can while any answer is still read, and ends at once only with +/// none. +/// +/// **What the caller knows and what it was told are two calls.** A query the +/// caller itself knows never reached its server ([`Lookup::on_unreached`]) is +/// let go, and a lookup with no query left whose answer is read ends with +/// [`Failure::Unreachable`]. A report from the network +/// ([`Lookup::on_report`]) is anyone's word (RFC 8085 §5.2): it lets no query +/// go and ends nothing, so the answer to a query reported so is still read. #[derive(Debug)] pub struct Lookup { /// The name now asked: the one given, or the last alias followed. name: Name, /// Aliases followed so far, counted against [`MAX_ALIASES`]. aliases: usize, - servers: Vec<[u8; 4]>, + /// Each server, and when it may be asked again: [`WAIT_MS`] after its + /// last query or after it was last known not to be reached, and at once + /// when it has answered any query, an older one's late failure included. + /// An address named twice is one server: every place it has carries the + /// same time. + servers: Vec<([u8; 4], u64)>, /// Queries sent for `name`, oldest first; one answered with a server - /// failure is taken out. [`Lookup::waiting`] is this list. + /// failure, or known not to have reached its server, is taken out. + /// [`Lookup::waiting`] is this list, which holds a query while the + /// lookup lasts. sent: Vec, /// The [`Asked`] the next query gets: none is given twice in a lookup. next: u32, /// Queries sent for `name`, answered or not. asked: usize, - /// When the newest query is given up on. + /// When the newest query is given up on; or, where it is not waited for + /// and the next server may not be asked yet, that server's turn. due: u64, /// The RCODE of the last server failure, which is what a lookup that runs /// out of queries reports if any server answered at all. @@ -509,7 +541,7 @@ impl Lookup { let mut lookup = Self { name, aliases: 0, - servers: servers.to_vec(), + servers: servers.iter().map(|server| (*server, now)).collect(), sent: Vec::new(), next: 0, asked: 0, @@ -531,22 +563,41 @@ impl Lookup { self.sent.iter().map(|s| s.asked) } - /// The next query for `name`, or the end where every one has been sent. + /// The next server's query, if it may be asked at `now`; or the wait for + /// the answers still read, until it may or the last query's wait is + /// over; or the end. fn ask(&mut self, now: u64, id: impl FnOnce() -> u16) -> Step { - if self.asked == ROUNDS * self.servers.len() { - return Step::Done(Err(match self.failed { - Some(rcode) => Failure::ServerFailed(rcode), - None => Failure::TimedOut, - })); + let turn = self.asked % self.servers.len(); + let (to, free) = self.servers[turn]; + let spent = self.asked == ROUNDS * self.servers.len(); + if !spent && now >= free { + self.free(to, now + WAIT_MS); + self.asked += 1; + let asked = Asked(self.next); + self.next += 1; + let id = id(); + self.sent.push(Sent { asked, id, to }); + self.due = now + WAIT_MS; + return Step::Ask { asked, to, query: query(id, &self.name) }; + } + if self.sent.is_empty() || (spent && now >= self.due) { + // With no query left, each was answered with a failure or never + // reached its server. + let silence = if self.sent.is_empty() { Failure::Unreachable } else { Failure::TimedOut }; + return Step::Done(Err(self.failed.map_or(silence, Failure::ServerFailed))); + } + if !spent { + // The newest query is not waited for: the next server's turn is. + self.due = free; + } + Step::Wait + } + + /// `server` may next be asked at `at`. + fn free(&mut self, server: [u8; 4], at: u64) { + for (_, free) in self.servers.iter_mut().filter(|(addr, _)| *addr == server) { + *free = at; } - let to = self.servers[self.asked % self.servers.len()]; - self.asked += 1; - let asked = Asked(self.next); - self.next += 1; - let id = id(); - self.sent.push(Sent { asked, id, to }); - self.due = now + WAIT_MS; - Step::Ask { asked, to, query: query(id, &self.name) } } /// The time is `now`: the newest query has had its [`WAIT_MS`] where it @@ -558,6 +609,33 @@ impl Lookup { self.ask(now, id) } + /// The caller knows at `now`, of its own knowledge, that the query + /// `asked` did not reach its server: it was never sent. Its answer is + /// read no more. `id` draws the ID of the query this sends. + pub fn on_unreached(&mut self, asked: Asked, now: u64, id: impl FnOnce() -> u16) -> Step { + let Some(which) = self.sent.iter().position(|s| s.asked == asked) else { + return Step::Wait; + }; + let unreached = self.sent.remove(which); + self.free(unreached.to, now + WAIT_MS); + // A newer query still waits for its own answer. + if which < self.sent.len() { + return Step::Wait; + } + self.ask(now, id) + } + + /// The network said at `now` that the query `asked` did not reach its + /// server. Its answer is still read. `id` draws the ID of the query this + /// sends. + pub fn on_report(&mut self, asked: Asked, now: u64, id: impl FnOnce() -> u16) -> Step { + // A newer query still waits for its own answer. + if self.sent.last().is_none_or(|s| s.asked != asked) { + return Step::Wait; + } + self.ask(now, id) + } + /// `msg` arrived at `now` from `port` of `from`, on the port `asked` was /// sent from. `id` draws the ID of the query this sends. pub fn on_datagram( @@ -585,6 +663,9 @@ impl Lookup { Verdict::ServerFailed(rcode) => { self.failed = Some(rcode); self.sent.remove(which); + // It answered: its interval, which is for a query that may + // have been lost, is over. + self.free(from, now); // The next server is asked now, unless a newer query is still // waiting for its own answer. if which == self.sent.len() { @@ -605,6 +686,9 @@ impl Lookup { self.sent.clear(); self.asked = 0; self.failed = None; + for (_, free) in &mut self.servers { + *free = now; + } self.ask(now, id) } }, diff --git a/toyos-dns/src/tests.rs b/toyos-dns/src/tests.rs index 9dd5631c916..630ac141f3b 100644 --- a/toyos-dns/src/tests.rs +++ b/toyos-dns/src/tests.rs @@ -853,3 +853,370 @@ fn rfc5452_9_2_an_id_is_drawn_for_a_query_that_is_sent_and_for_nothing_else() { assert_eq!(lookup.on_datagram(asked(&first).0, S1, PORT, &answer, 1, undrawn), Step::Done(Ok(vec![[1, 2, 3, 4]])), "an answer"); assert!(Lookup::start(name("www.example"), &[], 0, undrawn).is_none(), "a lookup that does not start"); } + +// --- A query that did not reach its server --- +// +// RFC 1035 §4.2.1: "Queries sent using UDP may be lost, and hence a +// retransmission strategy is required", and of that strategy "The client +// should try other servers and server addresses before repeating a query to a +// specific address of a server" and "Too aggressive retransmission can easily +// slow responses for the community at large ... the minimum retransmission +// interval should be 2-5 seconds". A query the caller could not send was not +// lost: it never left. + +#[test] +fn rfc1035_4_2_1_a_query_that_reached_nobody_is_not_waited_for_and_the_next_server_is_asked_at_once() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let (q1, ..) = asked(&first); + let second = lookup.on_unreached(q1, 7, || 2); + assert_eq!(to(&second), (S2, 2), "the other server, with the next draw, before any wait"); + let (q2, ..) = asked(&second); + assert_eq!(waiting(&lookup), [q2], "no answer is read for a query that reached nobody"); + assert_eq!(lookup.due(), 7 + WAIT_MS, "the wait is the second query's"); + let late = reply(1, OK, "www.example", &[a("www.example", [6, 6, 6, 6])]); + assert_eq!(lookup.on_datagram(q1, S1, PORT, &late, 8, undrawn), Step::Wait, "an answer to the query let go"); + assert_eq!(lookup.on_unreached(q1, 8, undrawn), Step::Wait, "the same report again is no query of this lookup's"); + let ok = reply(2, OK, "www.example", &[a("www.example", [1, 2, 3, 4])]); + assert_eq!(lookup.on_datagram(q2, S2, PORT, &ok, 9, undrawn), Step::Done(Ok(vec![[1, 2, 3, 4]]))); +} + +#[test] +fn rfc1035_4_2_1_no_server_is_asked_again_at_once_a_lookup_that_reached_none_ends() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1], 0, || 1).unwrap(); + assert_eq!(lookup.on_unreached(asked(&first).0, 3_000, undrawn), Step::Done(Err(Failure::Unreachable)), "one server: nobody else to ask"); + + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let second = lookup.on_unreached(asked(&first).0, 0, || 2); + assert_eq!(to(&second), (S2, 2)); + assert_eq!(lookup.on_unreached(asked(&second).0, 0, undrawn), Step::Done(Err(Failure::Unreachable)), "two servers, each asked once and neither reached"); +} + +#[test] +fn rfc1035_4_2_1_a_server_not_reached_is_asked_again_in_its_turn_a_wait_later() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let mut sent = vec![(0, to(&lookup.on_unreached(asked(&first).0, 0, || 2)).0)]; + let mut now = 0; + let ended = loop { + now += WAIT_MS; + // The wait of the query to S2 is over: S1's turn, and it is not reached. + let step = match lookup.on_time(now, || 3) { + Step::Done(result) => break result, + step => step, + }; + assert_eq!(to(&step).0, S1); + match lookup.on_unreached(asked(&step).0, now, || 4) { + Step::Done(result) => break result, + step => sent.push((now, to(&step).0)), + } + }; + let rounds: Vec<(u64, [u8; 4])> = (0..ROUNDS as u64).map(|round| (round * WAIT_MS, S2)).collect(); + assert_eq!(sent, rounds, "S2 is asked once a round, a wait apart, S1 before it each time"); + assert_eq!((now, ended), (ROUNDS as u64 * WAIT_MS, Err(Failure::TimedOut)), "S2 was reached and waited for, so the lookup timed out"); +} + +#[test] +fn an_older_query_that_reached_nobody_asks_nobody_and_ends_nothing() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1], 0, || 1).unwrap(); + let (q1, ..) = asked(&first); + let second = lookup.on_time(WAIT_MS, || 2); + let (q2, ..) = asked(&second); + assert_eq!(lookup.on_unreached(q1, WAIT_MS + 1, undrawn), Step::Wait, "the newer query still waits for its own answer"); + assert_eq!(waiting(&lookup), [q2]); + assert_eq!(lookup.due(), 2 * WAIT_MS, "and its wait is unchanged"); + assert_eq!(lookup.on_unreached(q2, WAIT_MS + 2, undrawn), Step::Done(Err(Failure::Unreachable)), "the newest reached nobody either"); +} + +// With a server left to ask at once, too: it is the newest query's wait. +#[test] +fn an_older_query_that_reached_nobody_does_not_hurry_the_next_server() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let second = lookup.on_time(WAIT_MS, || 2); + assert_eq!(to(&second), (S2, 2)); + assert_eq!(lookup.on_unreached(asked(&first).0, WAIT_MS + 1, undrawn), Step::Wait, "the query to S2 still waits for its own answer"); + assert_eq!(waiting(&lookup), [asked(&second).0]); + assert_eq!(lookup.due(), 2 * WAIT_MS); +} + +// The newest query reached nobody, and nobody is left to ask at once: the +// lookup ends only where no older query's answer is still read, and otherwise +// waits the newest one's wait out for it. +#[test] +fn a_lookup_whose_newest_query_reached_nobody_waits_for_an_older_querys_answer() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1], 0, || 1).unwrap(); + let (q1, ..) = asked(&first); + let second = lookup.on_time(WAIT_MS, || 2); + assert_eq!(lookup.on_unreached(asked(&second).0, WAIT_MS, undrawn), Step::Wait, "the first query may yet be answered"); + assert_eq!(waiting(&lookup), [q1]); + assert_eq!(lookup.due(), 2 * WAIT_MS, "the wait is the one the query that reached nobody began"); + let ok = reply(1, OK, "www.example", &[a("www.example", [1, 2, 3, 4])]); + assert_eq!(lookup.on_datagram(q1, S1, PORT, &ok, WAIT_MS + 1, undrawn), Step::Done(Ok(vec![[1, 2, 3, 4]]))); +} + +#[test] +fn a_lookup_that_then_reached_neither_server_waits_for_an_older_querys_answer() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let (q1, ..) = asked(&first); + let second = lookup.on_time(WAIT_MS, || 2); + assert_eq!(to(&second), (S2, 2)); + let third = lookup.on_unreached(asked(&second).0, WAIT_MS, || 3); + assert_eq!(to(&third), (S1, 3), "S1 again, a wait after its first query"); + assert_eq!(lookup.on_unreached(asked(&third).0, WAIT_MS, undrawn), Step::Wait, "the first query may yet be answered"); + assert_eq!(waiting(&lookup), [q1]); + assert_eq!(lookup.due(), 2 * WAIT_MS); + assert_eq!(lookup.on_time(2 * WAIT_MS - 1, undrawn), Step::Wait); + assert_eq!(to(&lookup.on_time(2 * WAIT_MS, || 4)), (S2, 4), "S2's turn, a wait after it was last asked"); + let ok = reply(1, OK, "www.example", &[a("www.example", [1, 2, 3, 4])]); + assert_eq!(lookup.on_datagram(q1, S1, PORT, &ok, 2 * WAIT_MS + 1, undrawn), Step::Done(Ok(vec![[1, 2, 3, 4]]))); +} + +// And the last query a lookup has: the queries before it are read until its +// wait is over, as if it had left. +#[test] +fn a_lookup_whose_last_query_reached_nobody_waits_for_the_answers_still_read() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1], 0, || 1).unwrap(); + let (q1, ..) = asked(&first); + let mut last = first; + for round in 1..ROUNDS as u64 { + last = lookup.on_time(round * WAIT_MS, || 2); + } + let end = ROUNDS as u64 * WAIT_MS; + assert_eq!(lookup.on_unreached(asked(&last).0, end - WAIT_MS, undrawn), Step::Wait, "the queries before it may yet be answered"); + assert_eq!(waiting(&lookup).first(), Some(&q1)); + assert_eq!(lookup.on_time(end - 1, undrawn), Step::Wait); + assert_eq!(lookup.on_time(end, undrawn), Step::Done(Err(Failure::TimedOut)), "queries that may have reached the server were waited for"); +} + +// --- A query the network says did not reach its server --- +// +// RFC 8085 §5.2: "applications SHOULD appropriately validate the payload of +// ICMP messages to ensure these are received in response to transmitted +// traffic ... This requires context, such as local state about communication +// instances to each destination, that although readily available in +// connection-oriented transport protocols is not always maintained by +// UDP-based applications", and "any incoming ICMP messages ... SHOULD NOT +// abort the communication". A report quotes eight octets of the query: its +// ports, and not its ID. + +#[test] +fn rfc8085_5_2_a_reported_query_is_still_read_and_the_next_server_is_asked_at_once() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let (q1, ..) = asked(&first); + let second = lookup.on_report(q1, 7, || 2); + assert_eq!(to(&second), (S2, 2), "the other server, before any wait"); + let (q2, ..) = asked(&second); + assert_eq!(waiting(&lookup), [q1, q2], "the reported query's answer is still read"); + assert_eq!(lookup.due(), 7 + WAIT_MS); + assert_eq!(lookup.on_report(q1, 8, undrawn), Step::Wait, "the same report again asks nobody"); + let ok = reply(1, OK, "www.example", &[a("www.example", [1, 2, 3, 4])]); + assert_eq!(lookup.on_datagram(q1, S1, PORT, &ok, 9, undrawn), Step::Done(Ok(vec![[1, 2, 3, 4]])), "the reported query's answer"); +} + +// Every query is reported the moment it is sent, twice: each server is asked +// once a round and no sooner, and the lookup ends where one nobody answered +// ends. +#[test] +fn rfc8085_5_2_reports_alone_end_no_lookup_and_ask_no_server_twice_inside_a_wait() { + for servers in [&[S1][..], &[S1, S2]] { + let (mut lookup, first) = Lookup::start(name("www.example"), servers, 0, || 1).unwrap(); + let mut sent = vec![(0, to(&first).0)]; + let mut newest = asked(&first).0; + let mut now = 0; + let ended = loop { + match lookup.on_report(newest, now, || 2) { + Step::Ask { asked, to, .. } => { + sent.push((now, to)); + newest = asked; + continue; + } + step => assert_eq!(step, Step::Wait, "a report ended the lookup"), + } + assert_eq!(lookup.on_report(newest, now, undrawn), Step::Wait, "the same report again"); + assert_eq!(lookup.on_time(now + WAIT_MS - 1, undrawn), Step::Wait, "the newest query's wait was cut short"); + now += WAIT_MS; + match lookup.on_time(now, || 3) { + Step::Done(result) => break result, + step => { + sent.push((now, to(&step).0)); + newest = asked(&step).0; + } + } + }; + let rounds: Vec<(u64, [u8; 4])> = (0..ROUNDS as u64).flat_map(|round| servers.iter().map(move |server| (round * WAIT_MS, *server))).collect(); + assert_eq!(sent, rounds, "each server once a round, the rounds a wait apart"); + assert_eq!((now, ended), (ROUNDS as u64 * WAIT_MS, Err(Failure::TimedOut))); + assert_eq!(waiting(&lookup).len(), ROUNDS * servers.len(), "every query's answer was read to the end"); + } +} + +#[test] +fn rfc8085_5_2_a_report_of_an_older_query_asks_nobody() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let second = lookup.on_time(WAIT_MS, || 2); + assert_eq!(lookup.on_report(asked(&first).0, WAIT_MS + 1, undrawn), Step::Wait, "the newer query still waits for its own answer"); + assert_eq!(lookup.due(), 2 * WAIT_MS, "and its wait is unchanged"); + assert_eq!(waiting(&lookup), [asked(&first).0, asked(&second).0]); +} + +// What the caller knows ends what a report could not: a query reported and +// then known not to have left is let go like any other. +#[test] +fn a_reported_query_the_caller_then_knows_reached_nobody_is_let_go() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1], 0, || 1).unwrap(); + let (q1, ..) = asked(&first); + assert_eq!(lookup.on_report(q1, 1, undrawn), Step::Wait, "one server: nobody else to ask"); + assert_eq!(lookup.on_unreached(q1, 2, undrawn), Step::Done(Err(Failure::Unreachable))); + + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let (q1, ..) = asked(&first); + let second = lookup.on_report(q1, 1, || 2); + assert_eq!(lookup.on_unreached(asked(&second).0, 2, undrawn), Step::Wait, "the reported query may yet be answered"); + assert_eq!(waiting(&lookup), [q1]); + assert_eq!(lookup.on_unreached(q1, 3, undrawn), Step::Done(Err(Failure::Unreachable)), "no query is left to read an answer for"); +} + +// --- Whoever reports or fails what --- +// +// RFC 1035 §4.2.1's interval is a server's: one that has not answered is not +// sent a second query inside it, whatever is said of its first and whatever +// another server answers. And a lookup does not end on another server's +// failure while a query's answer is read and its wait is not over. + +// S1's every query is reported the moment it is sent, and S2 answers each of +// its own with a failure at once. +#[test] +fn reports_and_another_servers_failures_hurry_no_server_that_has_not_answered_and_end_no_lookup_early() { + let refused = reply(2, OK | 5, "www.example", &[]); + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let (q1, ..) = asked(&first); + let mut sent = vec![(0, S1)]; + let mut newest = q1; + let mut now = 0; + let ended = loop { + let step = lookup.on_report(newest, now, || 2); + assert_eq!(to(&step), (S2, 2), "the other server, at once"); + sent.push((now, S2)); + assert_eq!(lookup.on_datagram(asked(&step).0, S2, PORT, &refused, now, || 9), Step::Wait, "S1 has not answered and was asked this millisecond"); + assert_eq!(lookup.on_report(newest, now, || 9), Step::Wait, "the same report again"); + assert_eq!(lookup.on_time(now + WAIT_MS - 1, || 9), Step::Wait, "the wait for S1 was cut short"); + now += WAIT_MS; + match lookup.on_time(now, || 1) { + Step::Done(result) => break result, + step => { + assert_eq!(to(&step), (S1, 1)); + sent.push((now, S1)); + newest = asked(&step).0; + } + } + }; + let rounds: Vec<(u64, [u8; 4])> = (0..ROUNDS as u64).flat_map(|round| [(round * WAIT_MS, S1), (round * WAIT_MS, S2)]).collect(); + assert_eq!(sent, rounds, "each server once a round, the rounds a wait apart"); + assert_eq!((now, ended), (ROUNDS as u64 * WAIT_MS, Err(Failure::ServerFailed(5)))); + assert_eq!(waiting(&lookup).len(), ROUNDS, "every query to S1 was read to the end"); + + // And S1's answer to its first query, a millisecond before that query's + // wait is over, is the lookup's. + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let (q1, ..) = asked(&first); + let second = lookup.on_report(q1, 0, || 2); + assert_eq!(lookup.on_datagram(asked(&second).0, S2, PORT, &refused, 0, || 9), Step::Wait); + assert_eq!(waiting(&lookup), [q1]); + let ok = reply(1, OK, "www.example", &[a("www.example", [1, 2, 3, 4])]); + assert_eq!(lookup.on_datagram(q1, S1, PORT, &ok, WAIT_MS - 1, || 9), Step::Done(Ok(vec![[1, 2, 3, 4]]))); +} + +// The same of a query the caller knows did not leave: S1 is not asked again +// inside its interval because S2 failed, and with no answer read the lookup +// ends by S2's word. +#[test] +fn a_server_not_reached_is_not_asked_again_at_once_because_another_failed() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let second = lookup.on_unreached(asked(&first).0, 0, || 2); + assert_eq!(to(&second), (S2, 2)); + let refused = reply(2, OK | 5, "www.example", &[]); + assert_eq!(lookup.on_datagram(asked(&second).0, S2, PORT, &refused, 0, || 9), Step::Done(Err(Failure::ServerFailed(5))), "S1 was asked this millisecond, and no query is read"); + + // With a query to S1 still read, the lookup waits for it instead. + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let (q1, ..) = asked(&first); + let second = lookup.on_time(WAIT_MS, || 2); + let third = lookup.on_datagram(asked(&second).0, S2, PORT, &refused, WAIT_MS, || 3); + assert_eq!(to(&third), (S1, 3), "S1 again, a wait after its first query"); + let fourth = lookup.on_unreached(asked(&third).0, WAIT_MS, || 2); + assert_eq!(to(&fourth), (S2, 2), "S2 answered, and is asked again at once like any server that fails"); + assert_eq!(lookup.on_datagram(asked(&fourth).0, S2, PORT, &refused, WAIT_MS, || 9), Step::Wait, "S1 was asked this millisecond"); + assert_eq!(waiting(&lookup), [q1]); + assert_eq!(lookup.due(), 2 * WAIT_MS); +} + +// The interval is a server's and not its place in the list: a list that names +// one address twice, as a lease may, asks it once a wait, whatever is +// reported of each query, twice over, the moment it is sent. +#[test] +fn a_server_named_twice_is_one_server_and_is_asked_once_a_wait_whatever_is_reported() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S1], 0, || 1).unwrap(); + let mut newest = asked(&first).0; + let mut sent = vec![0]; + let mut now = 0; + let ended = loop { + assert_eq!(lookup.on_report(newest, now, || 9), Step::Wait, "S1 has not answered and was asked this millisecond"); + assert_eq!(lookup.due(), now + WAIT_MS); + assert_eq!(lookup.on_report(newest, now, || 9), Step::Wait, "the same report again"); + assert_eq!(lookup.on_time(now + WAIT_MS - 1, || 9), Step::Wait); + now += WAIT_MS; + match lookup.on_time(now, || 1) { + Step::Done(result) => break result, + step => { + assert_eq!(to(&step), (S1, 1)); + sent.push(now); + newest = asked(&step).0; + } + } + }; + let turns = 2 * ROUNDS as u64; + assert_eq!(sent, (0..turns).map(|turn| turn * WAIT_MS).collect::>(), "ROUNDS queries a place in the list, a wait apart"); + assert_eq!((now, ended), (turns * WAIT_MS, Err(Failure::TimedOut))); +} + +// Where the newest query is not waited for and the next server may not be +// asked yet, the lookup wakes at that server's turn, not at the end of a +// wait nobody is keeping: S1 asked at 0 and reported at 1.5 s, S2 asked then +// and reported at 1.6 s, and S1 is asked again at 2 s. +#[test] +fn a_lookup_whose_newest_query_is_not_waited_for_wakes_at_the_next_servers_turn() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let second = lookup.on_report(asked(&first).0, 1_500, || 2); + assert_eq!((to(&second), lookup.due()), ((S2, 2), 1_500 + WAIT_MS)); + assert_eq!(lookup.on_report(asked(&second).0, 1_600, || 9), Step::Wait, "S1 was asked 1.6 s ago"); + assert_eq!(lookup.due(), WAIT_MS, "S1's turn"); + assert_eq!(lookup.on_time(WAIT_MS - 1, || 9), Step::Wait); + assert_eq!(to(&lookup.on_time(WAIT_MS, || 3)), (S1, 3)); + assert_eq!(waiting(&lookup).len(), 3, "every query's answer is still read"); +} + +// What the caller knows of a server holds for an interval too: S1's query is +// known not to have left a second after S2 was asked, so when S2's wait ends +// the lookup waits on, its answer still read, until S1 may be asked. +#[test] +fn a_server_known_unreached_is_asked_again_a_wait_after_that_and_the_lookup_waits_for_its_turn() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let second = lookup.on_time(WAIT_MS, || 2); + let (q2, ..) = asked(&second); + assert_eq!(lookup.on_unreached(asked(&first).0, WAIT_MS + 1_000, || 9), Step::Wait); + assert_eq!(lookup.on_time(2 * WAIT_MS, || 9), Step::Wait, "S1 was known unreached a second ago, and S2's answer is still read"); + assert_eq!(waiting(&lookup), [q2]); + assert_eq!(lookup.due(), 2 * WAIT_MS + 1_000, "the wait is until S1 may be asked"); + assert_eq!(lookup.on_time(2 * WAIT_MS + 999, || 9), Step::Wait); + assert_eq!(to(&lookup.on_time(2 * WAIT_MS + 1_000, || 3)), (S1, 3)); +} + +#[test] +fn a_server_that_answered_with_a_failure_is_what_a_lookup_that_then_reached_nobody_reports() { + let (mut lookup, first) = Lookup::start(name("www.example"), &[S1, S2], 0, || 1).unwrap(); + let second = lookup.on_datagram(asked(&first).0, S1, PORT, &reply(1, OK | 2, "www.example", &[]), 1, || 2); + assert_eq!(to(&second), (S2, 2)); + // S2 is not reached: S1 answered, so it is asked again at once, as after + // any failure, and the count of queries nobody received starts at S2. + let third = lookup.on_unreached(asked(&second).0, 2, || 3); + assert_eq!(to(&third), (S1, 3)); + assert_eq!(lookup.on_unreached(asked(&third).0, 3, undrawn), Step::Done(Err(Failure::ServerFailed(2)))); +} diff --git a/toyos-net-ip/src/lib.rs b/toyos-net-ip/src/lib.rs index 9cbee3f7893..59be88ccf90 100644 --- a/toyos-net-ip/src/lib.rs +++ b/toyos-net-ip/src/lib.rs @@ -108,7 +108,7 @@ pub mod limits { pub const FAILED_HOLD: Duration = Duration::from_secs(20); pub const LOCKTIME: Duration = Duration::from_secs(1); pub const IDLE_LIFETIME: Duration = Duration::from_secs(600); - pub const PENDING_PER_NEIGHBOUR: usize = 8; + pub const PENDING_PER_NEIGHBOUR: usize = 16; pub const PENDING_TOTAL: usize = 64; pub const TABLE_MAX: usize = 512; } diff --git a/toyos-net-ip/tests/nud.rs b/toyos-net-ip/tests/nud.rs index a760b8811b1..b8ef95d2408 100644 --- a/toyos-net-ip/tests/nud.rs +++ b/toyos-net-ip/tests/nud.rs @@ -66,14 +66,15 @@ fn s_ip_nud_003_release_keeps_arrival_order() { #[test] fn s_ip_nud_004_overflow_drops_the_oldest() { + let per = limits::nud::PENDING_PER_NEIGHBOUR as u8; let mut h = H::fixture_i(); - for n in 1..=10u8 { - h.send(A, B, 5001, 5001, &[b'0' + n % 10]).unwrap(); + for n in 1..=per + 2 { + h.send(A, B, 5001, 5001, &[n]).unwrap(); } h.out(); h.frame(&hex(V_ARP_REPLY)); let out: Vec> = h.out().iter().map(payload).collect(); - let expected: Vec> = (3..=10u8).map(|n| vec![b'0' + n % 10]).collect(); + let expected: Vec> = (3..=per + 2).map(|n| vec![n]).collect(); assert_eq!(out, expected); assert_eq!(h.count(Counter::NbPendingOverflow), 2); } diff --git a/toyos-net-shard/tests/udp.rs b/toyos-net-shard/tests/udp.rs index a7058b15ac8..345ee1ef3cc 100644 --- a/toyos-net-shard/tests/udp.rs +++ b/toyos-net-shard/tests/udp.rs @@ -50,5 +50,5 @@ fn s_ip_icd_012_shard_a_resolution_failure_reaches_the_connected_socket() { net.advance(Duration::from_millis(2_999)); assert_eq!(net.nodes[a].shard.recv_from(socket, &mut buf), Ok(None)); net.advance(Duration::from_millis(1)); - assert_eq!(net.nodes[a].shard.recv_from(socket, &mut buf), Err(Error::Failed(SocketError::Unreachable))); + assert_eq!(net.nodes[a].shard.recv_from(socket, &mut buf), Err(Error::Failed(SocketError::NextHopFailed))); } diff --git a/toyos-net-udp/src/lib.rs b/toyos-net-udp/src/lib.rs index a5fa0f73a40..f94fecbc81e 100644 --- a/toyos-net-udp/src/lib.rs +++ b/toyos-net-udp/src/lib.rs @@ -113,11 +113,17 @@ pub enum Binding { Connected { local: Ipv4Addr, peer: Ipv4Addr, peer_port: Port }, } -/// An error the network reported against a connected socket: its next call returns it once. +/// An error against a connected socket's datagrams: its next call returns it once. Two are what +/// an ICMP message said, which anyone who knows the socket's addresses and ports can send; one +/// is [ip]'s own. #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum SocketError { + /// An ICMP error said the peer refuses the protocol or the port. Refused, - Unreachable, + /// An ICMP error said the way to the peer is administratively prohibited. + Prohibited, + /// [ip] found no link address for the next hop of a datagram it held: nothing left. + NextHopFailed, } #[derive(Clone, Copy, Debug, PartialEq, Eq)] @@ -659,7 +665,7 @@ impl Udp { }; let pending = match class { ErrorClass::Refused => SocketError::Refused, - ErrorClass::Prohibited => SocketError::Unreachable, + ErrorClass::Prohibited => SocketError::Prohibited, ErrorClass::PathMtu | ErrorClass::Soft => return self.count(Counter::IcmpErrorSoft), }; socket.pending = Some(pending); @@ -669,7 +675,7 @@ impl Udp { /// [ip] could not reach the next hop of a datagram this crate handed it. pub fn unreachable(&mut self, flow: &Flow) { if let Some(socket) = self.connected(flow) { - socket.pending = Some(SocketError::Unreachable); + socket.pending = Some(SocketError::NextHopFailed); } } diff --git a/toyos-net-udp/tests/recv.rs b/toyos-net-udp/tests/recv.rs index 6fdc378374c..73e7a44f000 100644 --- a/toyos-net-udp/tests/recv.rs +++ b/toyos-net-udp/tests/recv.rs @@ -286,7 +286,7 @@ fn s_udp_us_048_an_error_for_no_socket() { fn s_udp_us_049_hard_and_soft() { let (mut u, id) = connected_to_dns(); u.datagram(&with_code(13)); - assert_eq!(u.udp.recv(id, &mut [0; 8]), Err(Error::Failed(SocketError::Unreachable))); + assert_eq!(u.udp.recv(id, &mut [0; 8]), Err(Error::Failed(SocketError::Prohibited))); u.datagram(&with_code(1)); let time_exceeded = edit(hex(V_ICMP_PU_A_OUT), |ip| { ip[20] = 11; diff --git a/toyos-net-udp/tests/send.rs b/toyos-net-udp/tests/send.rs index b38a55357ae..b4b95f165b4 100644 --- a/toyos-net-udp/tests/send.rs +++ b/toyos-net-udp/tests/send.rs @@ -277,7 +277,7 @@ fn s_udp_us_027_a_failed_next_hop_is_reported_once() { u.udp.send(&mut u.ip, id, b"q").unwrap(); u.run(3_000); assert_eq!(u.ip_count(toyos_net_ip::Counter::NbPendingDropped), 1); - assert_eq!(u.udp.recv(id, &mut [0; 64]), Err(Error::Failed(SocketError::Unreachable))); + assert_eq!(u.udp.recv(id, &mut [0; 64]), Err(Error::Failed(SocketError::NextHopFailed))); assert_eq!(u.udp.recv(id, &mut [0; 64]), Ok(None)); } diff --git a/userland/netstack/node/src/resolve.rs b/userland/netstack/node/src/resolve.rs index d8a5cfc23dd..1dcef864a51 100644 --- a/userland/netstack/node/src/resolve.rs +++ b/userland/netstack/node/src/resolve.rs @@ -10,9 +10,21 @@ //! query other than the one whose socket it reached. //! - **A socket lives as long as its query's answer is read**: it is closed when the lookup lets //! the query go, ends, or is let go itself, and its port is free from then. -//! - **A query [udp] refuses never left**: it is counted, holds no socket, and the lookup waits -//! its wait out as for any query nobody answered. So does one the network reports back, its -//! resolver unreachable or its port refused: counted, and waited out. +//! - **A query the node knows did not reach its resolver is let go** +//! (`toyos_dns::Lookup::on_unreached`): the lookup asks its next resolver at once, or ends +//! where no query's answer is read any more. The node knows it of a query [udp] refuses in the +//! call, which never left, is counted and holds no socket; and of one [ip] reports it found no +//! next hop for, which is counted and its socket closed. +//! - **An ICMP error is a report and not knowledge** (`toyos_dns::Lookup::on_report`): [udp] +//! hands a query's socket one that quotes the socket's addresses and ports and says refused or +//! prohibited, which takes an off-path sender the query's port to forge and not its id (RFC +//! 8085 §5.2). It is counted, and the next resolver is asked at once; the query's socket stays +//! open and its answer is read, and no number of them ends a lookup. +//! - **Every lookup's first query can wait in [ip] for one next hop at once**, where no link +//! address is known yet: [ip] holds `PENDING_PER_NEIGHBOUR` datagrams for a next hop it is +//! resolving and keeps the newest (RFC 4861 §7.2.2), which is no fewer than the lookups held +//! here. A datagram another socket sends to that next hop meanwhile takes a place in the same +//! queue. //! - **A lookup asks the resolvers of the lease it started under, and ends with that lease's //! word**: when the held lease names other resolvers, or none is held. //! - **A wait is a deadline of the node's** ([`Resolver::next_deadline`]); a reply is a frame. @@ -27,13 +39,16 @@ use alloc::vec::Vec; use core::net::Ipv4Addr; use toyos_dns::{Asked, Failure, Lookup, Name, Step, MAX_LOOKUPS, PORT}; -use toyos_net_udp::{Error, SocketId}; +use toyos_net_ip::limits::nud::PENDING_PER_NEIGHBOUR; +use toyos_net_udp::{Error, SocketError, SocketId}; use toyos_net_wire::Instant; use crate::lease::Stack; use crate::name::millis; use crate::{Counter, Counters, Node}; +const _: () = assert!(PENDING_PER_NEIGHBOUR >= MAX_LOOKUPS); + /// One lookup, from [`Node::resolve`] until its answer is taken or it is let go. #[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)] pub struct LookupId(u64); @@ -105,42 +120,51 @@ fn close(stack: &mut Stack, now: Instant, socket: SocketId) { } } -/// The length of the next reply waiting at one of `queries`' sockets, now in `reply`, and the -/// query it is for. An error the network reported against a query is counted and read past. -fn waiting(queries: &[Query], stack: &mut Stack, reply: &mut [u8], counters: &mut Counters) -> Option<(Query, usize)> { +/// What waits at a query's socket. +enum Heard { + /// A datagram of this length, now in the reply buffer. + Reply(usize), + /// [ip]'s word that the query never left. + Unreached, + /// An ICMP error's word that the query was refused. + Reported, +} + +/// The first of `queries` with something at its socket, and what: a reply is now in `reply`. +fn waiting(queries: &[Query], stack: &mut Stack, reply: &mut [u8]) -> Option<(Query, Heard)> { for query in queries { - loop { - match stack.recv_from(query.socket, reply) { - Ok(Some(received)) => return Some((*query, received.len)), - Ok(None) => break, - Err(Error::Failed(_)) => counters.add(Counter::QueryFailed, 1), - Err(Error::NoSuchSocket | Error::Refused(_)) => unreachable!("a query's socket is open while it is listed, and no rule refuses a receive"), - } + match stack.recv_from(query.socket, reply) { + Ok(Some(received)) => return Some((*query, Heard::Reply(received.len))), + Ok(None) => {} + Err(Error::Failed(SocketError::NextHopFailed)) => return Some((*query, Heard::Unreached)), + Err(Error::Failed(SocketError::Refused | SocketError::Prohibited)) => return Some((*query, Heard::Reported)), + Err(Error::NoSuchSocket | Error::Refused(_)) => unreachable!("a query's socket is open while it is listed, and no rule refuses a receive"), } } None } -/// Carries out `step` for `asking`, then closes the socket of every query the lookup no longer -/// reads an answer for. Returns how the lookup ended, if it did. -fn act(asking: &mut Asking, step: Step, now: Instant, stack: &mut Stack, counters: &mut Counters, draw: &mut impl FnMut() -> u32) -> Option, Ended>> { - let done = match step { - // An any-address bind that names no port is refused only for want of a free one. - Step::Ask { asked, to, query } => match stack.bind(Ipv4Addr::UNSPECIFIED, None, &mut *draw) { - Ok((socket, _)) => { +/// Carries out `step` for `asking`, and every step the lookup answers a query [udp] refused +/// with, then closes the socket of every query the lookup no longer reads an answer for. Returns +/// how the lookup ended, if it did. +fn act(asking: &mut Asking, mut step: Step, now: Instant, stack: &mut Stack, counters: &mut Counters, draw: &mut impl FnMut() -> u32) -> Option, Ended>> { + let done = loop { + match step { + Step::Ask { asked, to, query } => { + // An any-address bind that names no port is refused only for want of a free one. + let Ok((socket, _)) = stack.bind(Ipv4Addr::UNSPECIFIED, None, &mut *draw) else { break Some(Err(Ended::NoPort)) }; let resolver = Ipv4Addr::from(to); if stack.connect(now, socket, resolver, PORT).is_ok() && stack.send_to(now, socket, resolver, PORT, &query).is_ok() { asking.queries.push(Query { asked, socket, to }); - } else { - counters.add(Counter::QueryUnsent, 1); - close(stack, now, socket); + break None; } - None + counters.add(Counter::QueryUnsent, 1); + close(stack, now, socket); + step = asking.lookup.on_unreached(asked, millis(now), || id(&mut *draw)); } - Err(_) => Some(Err(Ended::NoPort)), - }, - Step::Wait => None, - Step::Done(result) => Some(result.map_err(Ended::Failed)), + Step::Wait => break None, + Step::Done(result) => break Some(result.map_err(Ended::Failed)), + } }; let lookup = &asking.lookup; asking.queries.retain(|query| { @@ -174,16 +198,19 @@ impl Resolver { let servers: Vec<[u8; 4]> = resolvers.iter().map(Ipv4Addr::octets).collect(); let Some((lookup, step)) = Lookup::start(name, &servers, millis(now), || id(&mut *draw)) else { unreachable!("the lease names a resolver") }; let mut asking = Asking { id: LookupId(self.next), lookup, resolvers, queries: Vec::new() }; - match act(&mut asking, step, now, stack, counters, &mut *draw) { - None => { - self.next = self.next.wrapping_add(1); - let started = asking.id; - self.asking.push(asking); - Ok(started) - } - Some(Err(Ended::NoPort)) => Err(NotStarted::ResourceExhausted), - Some(other) => unreachable!("a lookup's first step is a query, not {other:?}"), + let done = act(&mut asking, step, now, stack, counters, &mut *draw); + if done == Some(Err(Ended::NoPort)) { + return Err(NotStarted::ResourceExhausted); + } + self.next = self.next.wrapping_add(1); + let started = asking.id; + match done { + None => self.asking.push(asking), + // No resolver could be sent a query: the lookup has its id and its end at once, and + // holds no socket. + Some(result) => self.ended.push(Resolved { id: started, result }), } + Ok(started) } /// Ends every lookup whose lease went or names other resolvers, reads every reply that @@ -196,11 +223,20 @@ impl Resolver { let named = stack.lease().is_some_and(|lease| lease.dns == asking.resolvers); let mut done = if named { None } else { Some(Err(Ended::LeaseChanged)) }; while done.is_none() { - let Some((query, len)) = waiting(&asking.queries, stack, reply.as_mut_slice(), counters) else { break }; - let message = reply.get(..len).unwrap_or_default(); - // The socket is connected: [udp] delivered this from port 53 of the resolver the - // query went to, or not at all. - let step = asking.lookup.on_datagram(query.asked, query.to, PORT, message, ms, || id(&mut *draw)); + let Some((query, heard)) = waiting(&asking.queries, stack, reply.as_mut_slice()) else { break }; + let step = match heard { + // The socket is connected: [udp] delivered this from port 53 of the resolver + // the query went to, or not at all. + Heard::Reply(len) => asking.lookup.on_datagram(query.asked, query.to, PORT, reply.get(..len).unwrap_or_default(), ms, || id(&mut *draw)), + Heard::Unreached => { + counters.add(Counter::QueryFailed, 1); + asking.lookup.on_unreached(query.asked, ms, || id(&mut *draw)) + } + Heard::Reported => { + counters.add(Counter::QueryFailed, 1); + asking.lookup.on_report(query.asked, ms, || id(&mut *draw)) + } + }; done = act(asking, step, now, stack, counters, &mut *draw); } if done.is_none() { @@ -231,9 +267,11 @@ impl Resolver { } impl Node { - /// Starts looking `name` up at the resolvers the held lease names, which spends two draws: - /// the first query's id, then its port. A call refused for want of a port has spent both; - /// any other refused call spends none. + /// Starts looking `name` up at the resolvers the held lease names, which spends two draws a + /// query, its id and then its port: for the first query, and for each asked in its place + /// because [udp] refused the one before. A call refused for want of a port has spent them; + /// any other refused call spends none. A lookup no resolver could be sent a query of has + /// started and ended: its end is in the next [`Self::take_resolved`]. pub fn resolve(&mut self, now: Instant, name: Name, mut draw: impl FnMut() -> u32) -> Result { self.resolver.start(now, name, &mut self.stack, &mut self.counters, &mut draw) } diff --git a/userland/netstack/node/tests/resolve.rs b/userland/netstack/node/tests/resolve.rs index 535ee604e89..0d61d790988 100644 --- a/userland/netstack/node/tests/resolve.rs +++ b/userland/netstack/node/tests/resolve.rs @@ -13,7 +13,7 @@ mod lan; use std::net::Ipv4Addr; use std::time::Duration; -use common::{arp, from_server, message, message_of, terms, xid, A, ACK, DNS, MAC, MAC_B, NAK, R}; +use common::{arp, from_server, message, message_of, sum, terms, xid, A, ACK, DNS, MAC, MAC_B, MAC_R, NAK, R}; use lan::{udp, Lan, Seen, Udp, B}; use toyos_dns::{Failure, Name, MAX_LOOKUPS, ROUNDS, WAIT_MS}; use toyos_net_node::{Counter, Ended, LookupId, NotStarted, Refused, Resolved}; @@ -27,8 +27,20 @@ const ANSWERS: Ipv4Addr = DNS; const MAC_S: [u8; 6] = [2, 0, 0, 0, 0, 0x53]; /// On the link, and nothing there answers ARP: a LAN's resolver that is down. const SILENT: Ipv4Addr = Ipv4Addr::new(192, 0, 2, 52); +/// On the link and answering ARP, with nothing on its port 53: it reports every query back. +const REFUSES: Ipv4Addr = Ipv4Addr::new(192, 0, 2, 54); +const MAC_F: [u8; 6] = [2, 0, 0, 0, 0, 0x54]; +/// On the link, answering ARP and every query with RCODE 5, refused (RFC 1035 §4.1.1). +const FAILS: Ipv4Addr = Ipv4Addr::new(192, 0, 2, 55); +const MAC_X: [u8; 6] = [2, 0, 0, 0, 0, 0x55]; /// Off the link, asked of a lease that names no router. const UNROUTED: Ipv4Addr = Ipv4Addr::new(198, 51, 100, 53); +const UNROUTED_TOO: Ipv4Addr = Ipv4Addr::new(198, 51, 100, 54); +/// RFC 792: a destination unreachable's codes. +const HOST_UNREACHABLE: u8 = 1; +const PORT_UNREACHABLE: u8 = 3; +/// RFC 1812 §5.2.7.1: communication administratively prohibited. +const PROHIBITED: u8 = 13; const ADDRESS: [u8; 4] = [203, 0, 113, 7]; /// The address every reply that must not be read carries. const FORGED: [u8; 4] = [203, 0, 113, 66]; @@ -120,6 +132,37 @@ fn gives(query: &[u8], address: [u8; 4]) -> Vec { reply(query, RESPONSE, &[(TYPE_A, address.to_vec())]) } +/// RFC 792's destination unreachable from `reporter` at `mac`: type 3 and `code`, the checksum, +/// four unused octets, then the internet header and the first 64 bits of the datagram it is +/// about. That datagram is spelled here as RFC 791 and RFC 768 lay out one from `source` to +/// `destination` carrying `length` octets: no options, DF, TTL 64, protocol 17. +fn unreachable(code: u8, (reporter, mac): (Ipv4Addr, [u8; 6]), source: (Ipv4Addr, u16), destination: (Ipv4Addr, u16), length: usize) -> Vec { + let datagram = u16::try_from(8 + length).unwrap().to_be_bytes(); + let total = u16::try_from(20 + 8 + length).unwrap().to_be_bytes(); + let mut quoted = vec![0x45, 0, total[0], total[1], 0, 0, 0x40, 0, 64, 17, 0, 0]; + quoted.extend_from_slice(&source.0.octets()); + quoted.extend_from_slice(&destination.0.octets()); + let check = sum("ed); + quoted[10..12].copy_from_slice(&check.to_be_bytes()); + quoted.extend_from_slice(&[&source.1.to_be_bytes()[..], &destination.1.to_be_bytes(), &datagram, &[0, 0]].concat()); + let mut icmp = vec![3, code, 0, 0, 0, 0, 0, 0]; + icmp.extend_from_slice("ed); + let check = sum(&icmp); + icmp[2..4].copy_from_slice(&check.to_be_bytes()); + let total = u16::try_from(20 + icmp.len()).unwrap().to_be_bytes(); + let mut ip = vec![0x45, 0, total[0], total[1], 0, 0, 0x40, 0, 64, 1, 0, 0]; + ip.extend_from_slice(&reporter.octets()); + ip.extend_from_slice(&source.0.octets()); + let check = sum(&ip); + ip[10..12].copy_from_slice(&check.to_be_bytes()); + [&MAC[..], &mac, &[0x08, 0x00], &ip, &icmp].concat() +} + +/// `code` from `reporter` about `query`, as the node sent it. +fn reports(code: u8, reporter: (Ipv4Addr, [u8; 6]), query: &Udp) -> Vec { + unreachable(code, reporter, (query.source, query.source_port), (query.destination, query.port), query.payload.len()) +} + /// What [`ANSWERS`] says for a name. enum Says { Address([u8; 4]), @@ -147,8 +190,12 @@ struct Net { zone: Vec<(&'static str, u64, Says)>, /// Frames [`ANSWERS`] has sent and the wire has not yet delivered, with the time they arrive. held: Vec<(Instant, Vec)>, - /// Every query that left, in order. + /// Every query that left for [`ANSWERS`], in order. queried: Vec, + /// Every query that left for [`REFUSES`], in order. + refused: Vec, + /// Every query that left for [`FAILS`], in order. + failed: Vec, ended: Vec, /// The last draw [`Self::resolve`] handed out: its ids are 0x8000 and up, and those the node /// draws for itself from `lan`'s count are below it. @@ -176,6 +223,8 @@ impl Net { zone: Vec::new(), held: Vec::new(), queried: Vec::new(), + refused: Vec::new(), + failed: Vec::new(), ended: Vec::new(), draws: 0x7c00_8000, } @@ -219,6 +268,18 @@ impl Net { fn far_end(&mut self, at: Instant, seen: &Seen) { match seen { Seen::Arp { request: true, target, .. } if *target == ANSWERS => self.lan.deliver(&arp(MAC, false, MAC_S, ANSWERS, A)), + Seen::Arp { request: true, target, .. } if *target == REFUSES => self.lan.deliver(&arp(MAC, false, MAC_F, REFUSES, A)), + Seen::Arp { request: true, target, .. } if *target == FAILS => self.lan.deliver(&arp(MAC, false, MAC_X, FAILS, A)), + Seen::Udp(udp) if udp.port == 53 && udp.destination == FAILS => { + assert_eq!(udp.to, MAC_X, "a query left for a server the wire cannot reach"); + self.held.push((self.lan.now, lan::udp(MAC, MAC_X, (FAILS, 53), (A, udp.source_port), &reply(&udp.payload, RESPONSE | 5, &[])))); + self.failed.push(Query { at, udp: udp.clone(), id: u16::from_be_bytes([udp.payload[0], udp.payload[1]]), name: asked_name(&udp.payload) }); + } + Seen::Udp(udp) if udp.port == 53 && udp.destination == REFUSES => { + assert_eq!(udp.to, MAC_F, "a query left for a server the wire cannot reach"); + self.held.push((self.lan.now, reports(PORT_UNREACHABLE, (REFUSES, MAC_F), udp))); + self.refused.push(Query { at, udp: udp.clone(), id: u16::from_be_bytes([udp.payload[0], udp.payload[1]]), name: asked_name(&udp.payload) }); + } Seen::Udp(udp) if udp.port == 53 => { assert_eq!((udp.to, udp.destination), (MAC_S, ANSWERS), "a query left for a server the wire cannot reach"); let query = Query { at, udp: udp.clone(), id: u16::from_be_bytes([udp.payload[0], udp.payload[1]]), name: asked_name(&udp.payload) }; @@ -426,24 +487,189 @@ fn nothing_but_its_answer_ends_a_lookup() { assert_eq!(net.lan.node.take_resolved(), [Resolved { id, result: Ok(vec![ADDRESS]) }], "the answer, after all of them"); } -// RFC 1035 §4.2.1: a resolver that does not answer is given up on after a wait and the next is -// asked. No route leads to the first: [udp] refuses its query's socket that peer (`udp.no-route`), -// and the query is counted, holds no port, and is waited out like one nobody answered. +// RFC 1035 §4.2.1: "Queries sent using UDP may be lost, and hence a retransmission strategy is +// required", and "The client should try other servers and server addresses before repeating a +// query to a specific address of a server." A query [udp] refuses was not lost: it never left. +// No route leads to the first resolver, so [udp] refuses its query's socket that peer +// (`udp.no-route`): the query is counted, holds no port, and the second resolver is asked in the +// same call, with the next two draws, and answers at the millisecond the lookup started. #[test] -fn a_query_udp_refuses_is_counted_holds_no_port_and_is_waited_out() { +fn a_query_udp_refuses_is_counted_holds_no_port_and_the_next_resolver_is_asked_at_once() { let mut net = Net::leased(&[UNROUTED, ANSWERS], None); net.zone.push(("www.example", 0, Says::Address(ADDRESS))); let refused = net.lan.counted(Rule::NoRoute); - let id = net.lan.node.resolve(net.lan.now, name("www.example"), sequence([0x8001, 9])).expect("a lookup whose first query cannot leave"); + let id = net.lan.node.resolve(net.lan.now, name("www.example"), sequence([0x8001, 9, 0x8002, 10])).expect("a lookup whose first query cannot leave"); assert_eq!(net.lan.counted(Rule::NoRoute), refused + 1, "the premise: [udp] refused the query for want of a route"); assert_eq!(net.lan.node.counters().get(Counter::QueryUnsent), 1); assert!(!net.port_held(49_152 + 9), "the refused query's socket"); + assert!(net.port_held(49_152 + 10), "the second query's socket"); let ended = net.run(id, 25_000); assert_eq!(ended, Some(Ok(vec![ADDRESS])), "the second resolver heard {:?}", net.queried); - assert_eq!(net.ms(), WAIT_MS); + assert_eq!(net.ms(), 0, "the answer waited on a query that never left"); + let left: Vec<(u16, u16)> = net.queried.iter().map(|query| (query.id, query.udp.source_port)).collect(); + assert_eq!(left, [(0x8002, 49_152 + 10)]); assert_eq!(net.lan.node.counters().get(Counter::QueryUnsent), 1); } +// And where no route leads to any resolver the lease names, no query leaves and none is waited +// for: the lookup has started and ended in the one call, each resolver asked once (RFC 1035 +// §4.2.1 repeats a query after an interval, and there is no answer to give one to), and holds +// neither a port nor, once its end is taken, a place. +#[test] +fn a_lookup_udp_refuses_every_resolver_of_ends_in_the_call_that_started_it() { + let mut net = Net::leased(&[UNROUTED, UNROUTED_TOO], None); + let refused = net.lan.counted(Rule::NoRoute); + let id = net.lan.node.resolve(net.lan.now, name("www.example"), sequence([0x8001, 9, 0x8002, 10])).expect("started, and ended"); + assert_eq!(net.lan.counted(Rule::NoRoute), refused + 2, "the premise: [udp] refused both queries for want of a route"); + assert_eq!(net.lan.node.counters().get(Counter::QueryUnsent), 2); + assert_eq!(net.lan.node.take_resolved(), [Resolved { id, result: Err(Ended::Failed(Failure::Unreachable)) }]); + assert!(!net.port_held(49_152 + 9) && !net.port_held(49_152 + 10), "the refused queries' sockets"); + net.pass(); + assert!(net.queried.is_empty() && net.refused.is_empty(), "a query left"); + let next = net.lan.node.resolve(net.lan.now, name("other.example"), sequence([0x8003, 11, 0x8004, 12])).expect("the next lookup"); + assert_ne!(next, id, "an ended lookup's id is not given again"); +} + +// RFC 792: "If, in the destination host, the IP module cannot deliver the datagram because the +// indicated protocol module or process port is not active, the destination host may send a +// destination unreachable message to the source host." RFC 1122 §4.1.3.3: "UDP MUST pass to the +// application layer all ICMP error messages that it receives from the IP layer." The first +// resolver answers ARP and has nothing on port 53: its report of the query is counted, and the +// second resolver is asked at the millisecond the report arrived and answers in it. +#[test] +fn a_query_its_resolver_refuses_by_icmp_is_counted_and_the_next_resolver_is_asked_at_once() { + let mut net = Net::leased(&[REFUSES, ANSWERS], Some(R)); + net.zone.push(("www.example", 0, Says::Address(ADDRESS))); + let delivered = net.lan.counted(Rule::IcmpErrorDelivered); + let id = net.resolve("www.example").unwrap(); + let ended = net.run(id, 25_000); + assert_eq!(ended, Some(Ok(vec![ADDRESS])), "the second resolver heard {:?}", net.queried); + assert_eq!(net.ms(), 0, "the answer waited on a query its resolver had refused"); + assert_eq!(net.lan.counted(Rule::IcmpErrorDelivered), delivered + 1, "the premise: the report reached the query's socket"); + assert_eq!((net.refused.len(), net.queried.len()), (1, 1), "one query to each resolver"); + assert_eq!(net.lan.node.counters().get(Counter::QueryFailed), 1); + assert_eq!(net.lan.node.counters().get(Counter::QueryUnsent), 0); +} + +// RFC 8085 §5.2 has an ICMP message "not abort the communication", and RFC 1035 §4.2.1 has no +// server asked again inside its interval. The lease's one resolver reports every query back: each +// report is counted, none ends the lookup or hurries its next query, and the lookup runs its +// rounds and ends timed out, as under a resolver that says nothing. +#[test] +fn reports_alone_end_no_lookup_and_ask_no_resolver_twice_inside_a_wait() { + let mut net = Net::leased(&[REFUSES], Some(R)); + let delivered = net.lan.counted(Rule::IcmpErrorDelivered); + let id = net.resolve("www.example").unwrap(); + assert_eq!(net.run(id, 10 * WAIT_MS), Some(Err(Ended::Failed(Failure::TimedOut)))); + let rounds = u64::try_from(ROUNDS).unwrap(); + assert_eq!(net.ms(), rounds * WAIT_MS); + let left: Vec = net.refused.iter().map(|query| net.ms_of(query.at)).collect(); + assert_eq!(left, (0..rounds).map(|round| round * WAIT_MS).collect::>()); + assert_eq!(net.lan.counted(Rule::IcmpErrorDelivered), delivered + rounds, "the premise: every report reached its query's socket"); + assert_eq!(net.lan.node.counters().get(Counter::QueryFailed), rounds); +} + +// And a lease may name one resolver twice (RFC 2132 §3.8 lists addresses and forbids no repeat): +// it is one resolver, with one interval. Every query is reported back, and one is on the wire a +// wait, `ROUNDS` for each time the lease names it. +#[test] +fn a_resolver_the_lease_names_twice_is_asked_once_a_wait_whatever_it_reports() { + let mut net = Net::leased(&[REFUSES, REFUSES], Some(R)); + let id = net.resolve("www.example").unwrap(); + assert_eq!(net.run(id, 20 * WAIT_MS), Some(Err(Ended::Failed(Failure::TimedOut)))); + let turns = 2 * u64::try_from(ROUNDS).unwrap(); + let left: Vec = net.refused.iter().map(|query| net.ms_of(query.at)).collect(); + assert_eq!(left, (0..turns).map(|turn| turn * WAIT_MS).collect::>()); + assert_eq!(net.ms(), turns * WAIT_MS); + assert_eq!(net.lan.node.counters().get(Counter::QueryFailed), turns, "the premise: every report reached its query's socket"); +} + +// A report has crossed a trust boundary: anyone can send an ICMP message. RFC 1122 §4.1.3.3 has +// the application "demultiplex these messages when they arrive", which [udp] does by the quoted +// datagram's addresses and ports against the connected socket's, and §3.2.2.1 has a destination +// unreachable of code 0, 1 or 5 be "only a hint, not proof, that the specified destination is +// unreachable". A report that quotes another port, another resolver or another port of the +// resolver's, and one of a router that says host unreachable about the query itself, each reach +// [udp], which is the premise, and none ends the query's wait: no second query leaves before +// it, and the answer is still read. +#[test] +fn a_report_that_is_not_about_the_query_or_is_a_hint_ends_no_wait() { + let mut net = Net::leased(&[ANSWERS], Some(R)); + let id = net.resolve("www.example").unwrap(); + net.pass(); + let asked = net.queried[0].clone(); + let about = |source_port: u16, destination: (Ipv4Addr, u16)| unreachable(PORT_UNREACHABLE, (ANSWERS, MAC_S), (A, source_port), destination, asked.udp.payload.len()); + let forged = [ + ("another port of the node's", about(asked.udp.source_port ^ 1, (ANSWERS, 53)), Rule::IcmpErrorNoSocket), + ("another resolver", about(asked.udp.source_port, (REFUSES, 53)), Rule::IcmpErrorNoSocket), + ("another port of the resolver's", about(asked.udp.source_port, (ANSWERS, 5_353)), Rule::IcmpErrorNoSocket), + ("a router's hint", reports(HOST_UNREACHABLE, (R, MAC_R), &asked.udp), Rule::IcmpErrorSoft), + ]; + for (what, report, rule) in &forged { + let counted = net.lan.counted(*rule); + net.lan.deliver(report); + assert_eq!(net.lan.node.take_resolved(), NONE, "a report about {what} ended the lookup"); + assert_eq!(net.lan.node.counters().get(Counter::QueryFailed), 0, "a report about {what} was taken for the query\x27s"); + assert_eq!(net.lan.counted(*rule), counted + 1, "the premise: the report about {what} reached [udp]"); + } + net.until(WAIT_MS - 1); + assert_eq!(net.queried.len(), 1, "a second query left before the first one's wait ended"); + net.resolver_says(asked.udp.source_port, &gives(&asked.udp.payload, ADDRESS)); + assert_eq!(net.lan.node.take_resolved(), [Resolved { id, result: Ok(vec![ADDRESS]) }]); +} + +// RFC 8085 §5.2: "applications SHOULD appropriately validate the payload of ICMP messages to +// ensure these are received in response to transmitted traffic", and such a message "SHOULD NOT +// abort the communication": what a report quotes, an off-path sender can guess. A port +// unreachable and a prohibition in the resolver's name that quote the query's own addresses and +// ports each reach the query's socket, which is the premise, and end nothing: each is counted, +// the socket is still the query's, and the resolver's answer behind them ends the lookup. +#[test] +fn a_forged_report_of_the_querys_own_addresses_and_ports_ends_nothing_and_the_answer_behind_it_is_taken() { + let mut net = Net::leased(&[ANSWERS], Some(R)); + let id = net.resolve("www.example").unwrap(); + net.pass(); + let asked = net.queried[0].clone(); + for (reported, code) in [(1, PORT_UNREACHABLE), (2, PROHIBITED)] { + let delivered = net.lan.counted(Rule::IcmpErrorDelivered); + net.lan.deliver(&reports(code, (ANSWERS, MAC_S), &asked.udp)); + assert_eq!(net.lan.counted(Rule::IcmpErrorDelivered), delivered + 1, "the premise: the report of code {code} reached the query's socket"); + assert_eq!(net.lan.node.take_resolved(), NONE, "a report of code {code} from the wire ended the lookup"); + assert_eq!(net.lan.node.counters().get(Counter::QueryFailed), reported); + assert!(net.port_held(asked.udp.source_port), "the reported query's socket, after code {code}"); + } + net.pass(); + assert_eq!(net.queried.len(), 1, "one resolver: a report asked it again inside its wait"); + net.resolver_says(asked.udp.source_port, &gives(&asked.udp.payload, ADDRESS)); + assert_eq!(net.lan.node.take_resolved(), [Resolved { id, result: Ok(vec![ADDRESS]) }]); +} + +// And with a second resolver that answers every query with a failure, which a report of each +// query to the first would chain into a round of queries with no wait between them: every query +// that leaves for the answering resolver is reported in its name the moment it is on the wire, +// until none follows. One has left for each resolver, the lookup has not ended, and the answer +// the first resolver gives a millisecond before its query's wait is over ends it. RFC 1035 +// §4.2.1: the interval is the resolver's, whoever says what of its query. +#[test] +fn forged_reports_and_another_resolvers_failures_end_nothing_and_the_answer_behind_them_is_taken() { + let mut net = Net::leased(&[ANSWERS, FAILS], Some(R)); + net.zone.push(("www.example", WAIT_MS - 1, Says::Address(ADDRESS))); + let id = net.resolve("www.example").unwrap(); + let mut reported = 0; + for _ in 0..=2 * ROUNDS { + net.pass(); + let Some(query) = net.queried.get(reported).cloned() else { break }; + net.lan.deliver(&reports(PORT_UNREACHABLE, (ANSWERS, MAC_S), &query.udp)); + reported += 1; + } + assert_eq!(net.lan.node.counters().get(Counter::QueryFailed), u64::try_from(reported).unwrap(), "the premise: every report reached its query's socket"); + assert_eq!((net.queried.len(), net.failed.len(), net.ms()), (1, 1, 0), "one query to each resolver, the second's at the report of the first"); + assert_eq!(net.lan.node.take_resolved(), NONE, "reports and another resolver's failure ended the lookup"); + assert_eq!(net.run(id, 10 * WAIT_MS), Some(Ok(vec![ADDRESS])), "the resolvers heard {:?} and {:?}", net.queried, net.failed); + assert_eq!(net.ms(), WAIT_MS - 1); + assert_eq!((net.queried.len(), net.failed.len()), (1, 1)); +} + // RFC 1034 §5.3.3: an alias with no address is asked again at its end, every query afresh. The // first resolver answers only after six queries have left, three of them to it, and with an alias // alone: the queries for the old name are let go, so the two other answers to them, arriving @@ -581,26 +807,49 @@ fn a_query_with_no_port_to_leave_from_ends_its_lookup_by_name() { assert_eq!(left, [EPHEMERAL_FIRST + (EPHEMERAL_COUNT - 1)]); } -// RFC 4861 §7.2.2: [ip] holds a bounded queue for a next hop it is resolving, and keeps the -// newest. Lookups started together at a resolver whose link address is not yet known hand [ip] -// one query each: those past its queue never leave, and each is asked again when its wait ends. -// The track records this against the node (`issues/toyos-has-its-own-network-stack.md`). +// RFC 4861 §7.2.2: "While waiting for address resolution to complete, the sender MUST, for each +// neighbor, retain a small queue of packets waiting for address resolution to complete. ... Once +// address resolution completes, the node transmits any queued packets." Every lookup the node +// holds is started at a resolver whose link address is not yet known, and hands [ip] one query: +// each waits in that queue, and each is on the wire, in the order the lookups were started, once +// the resolver answers ARP. #[test] -fn a_burst_at_a_resolver_not_yet_resolved_leaves_ips_queue_of_queries_and_the_rest_a_wait_later() { +fn a_burst_at_a_resolver_not_yet_resolved_is_on_the_wire_once_the_resolver_answers_arp() { let mut net = Net::leased(&[ANSWERS], Some(R)); let names: Vec = (0..MAX_LOOKUPS).map(|n| format!("n{n}.example")).collect(); + let ip = |net: &Net| net.lan.node.shard().ip().neighbour(net.lan.node.shard().iface(), ANSWERS).map(toyos_net_ip::Nud::mac); + assert_eq!(ip(&net), None, "the premise: [ip] holds no entry for the resolver"); for asked in &names { net.resolve(asked).unwrap(); } + assert!(net.queried.is_empty(), "the premise: nothing has left"); net.pass(); - let queue = toyos_net_ip::limits::nud::PENDING_PER_NEIGHBOUR; - let mut left: Vec<&str> = net.queried.iter().map(|query| query.name.as_str()).collect(); - let mut newest: Vec<&str> = names[MAX_LOOKUPS - queue..].iter().map(String::as_str).collect(); - left.sort_unstable(); - newest.sort_unstable(); - assert_eq!(left, newest, "the newest {queue} queries left once the resolver answered ARP"); + assert_eq!(ip(&net), Some(Some(toyos_net_wire::ethernet::MacAddr(MAC_S))), "the premise: the resolver answered ARP"); + let left: Vec<&str> = net.queried.iter().map(|query| query.name.as_str()).collect(); + assert_eq!(left, names, "every query of the burst, once"); + assert_eq!(net.ms(), 0); + assert_eq!(net.lan.node.shard().ip().counters().get(toyos_net_ip::Counter::NbPendingOverflow), 0); +} + +// The queue is the next hop's and not the resolver's: RFC 4861 §7.2.2, "When a queue overflows, +// the new arrival SHOULD replace the oldest entry." One datagram a client sends to the resolver's +// address behind the burst takes the oldest query's place, and that lookup asks again when its +// wait ends. The track records this against [udp] (`issues/toyos-has-its-own-network-stack.md`). +#[test] +fn a_clients_datagram_behind_the_burst_takes_the_oldest_querys_place_in_ips_queue() { + let mut net = Net::leased(&[ANSWERS], Some(R)); + let names: Vec = (0..MAX_LOOKUPS).map(|n| format!("n{n}.example")).collect(); + for asked in &names { + net.resolve(asked).unwrap(); + } + let (client, _) = net.lan.node.udp_bind(ANY, None, || 0x3000).unwrap(); + net.lan.node.udp_send_to(net.lan.now, client, ANSWERS, 9, b"behind the burst").unwrap(); + net.pass(); + assert_eq!(net.lan.node.shard().ip().counters().get(toyos_net_ip::Counter::NbPendingOverflow), 1, "the premise: [ip]'s queue for the resolver overflowed by one"); + let left: Vec<&str> = net.queried.iter().map(|query| query.name.as_str()).collect(); + assert_eq!(left, names[1..], "every query but the oldest"); net.until(WAIT_MS); - assert_eq!(net.queried.len(), queue + MAX_LOOKUPS, "and every lookup's second query, a wait later"); + assert_eq!(net.queried.iter().filter(|query| query.name == names[0]).map(|query| net.ms_of(query.at)).collect::>(), [WAIT_MS], "the oldest lookup's second query, a wait later"); } // RFC 1035 §4.2.1: each resolver is asked ROUNDS times, WAIT_MS apart. A wait is a deadline of @@ -676,24 +925,44 @@ fn a_recorded_reply_of_a_public_resolver_answers_the_nodes_query() { assert_eq!(net.lan.node.take_resolved(), [Resolved { id, result: Ok(vec![[8, 8, 4, 4], [8, 8, 8, 8]]) }]); } -// RFC 1122 §3.3.1.1 and §4.1.3.3: a datagram whose next hop never answers ARP is reported to the -// socket it left from. Each of a lookup's queries to such a resolver is counted as the network -// reported it, and waited out like one nobody answered. +// RFC 1122 §2.3.2.1 and RFC 4861 §7.2.2 have address resolution give up after its requests, and +// RFC 1122 §4.1.3.3 has the socket a datagram left from told. The lease's one resolver never +// answers ARP: [ip] gives up at its third request's end, 3 s in, and reports the two queries it +// held, the first and the one asked when the first one's wait ended. Each is counted, and the +// lookup ends there by name, at [ip]'s first word of its resolver, with no third query: RFC 1035 +// §4.2.1 repeats a query after an interval, to a server that may have missed it. This is [ip]'s +// own knowledge and no datagram's word, so it ends the lookup where an ICMP error does not. #[test] -fn a_query_the_network_reports_unreachable_is_counted_and_waited_out() { +fn a_query_the_network_reports_unreachable_is_counted_and_ends_a_lookup_with_no_other_resolver() { let mut net = Net::leased(&[SILENT], Some(R)); let id = net.resolve("www.example").unwrap(); - let rounds = u64::try_from(ROUNDS).unwrap(); - assert_eq!(net.run(id, 10 * WAIT_MS), Some(Err(Ended::Failed(Failure::TimedOut)))); - assert_eq!(net.ms(), rounds * WAIT_MS); + assert_eq!(net.run(id, 10 * WAIT_MS), Some(Err(Ended::Failed(Failure::Unreachable)))); + assert_eq!(net.ms(), 3_000, "the lookup outlived [ip]'s report of its one resolver"); assert!(net.queried.is_empty(), "the premise: no query reached the wire: {:?}", net.queried); - // The first two queries wait in [ip] until the neighbour fails, at three of its requests; - // the third leaves after that and is refused at once, because [ip] holds a failed neighbour - // down for longer than the second it would take to be asked again. - let refused_at_once = net.lan.node.shard().ip().counters().get(toyos_net_ip::Counter::NbFailedRefused); - assert_eq!(refused_at_once, 1, "the premise: [ip] refused one query on a failed neighbour"); - assert_eq!(net.lan.node.counters().get(Counter::QueryFailed), rounds); + let ip = net.lan.node.shard().ip().counters(); + assert_eq!((ip.get(toyos_net_ip::Counter::NbFailed), ip.get(toyos_net_ip::Counter::NbPendingDropped)), (1, 2), "the premise: [ip] gave the resolver up once, holding two queries"); + assert_eq!(net.lan.node.counters().get(Counter::QueryFailed), 2); assert_eq!(net.lan.node.counters().get(Counter::QueryUnsent), 0); + let requests: Vec = net.lan.sent.iter().filter(|(_, seen)| matches!(seen, Seen::Arp { request: true, target, .. } if *target == SILENT)).map(|(at, _)| net.ms_of(*at)).collect(); + assert_eq!(requests, [0, 1_000, 2_000], "the premise: [ip] asked for the resolver three times, a second apart"); +} + +// [ip] holds a next hop it gave up on as failed for a while, and refuses a datagram to it at the +// transmit opportunity that would have carried it, where no call of the resolver's is in +// progress: the node reads the report at its next wake, which is the end of the query's wait. +// The track records this against the node (`issues/toyos-has-its-own-network-stack.md`). +#[test] +fn a_query_ip_refuses_at_its_transmit_opportunity_is_read_at_the_nodes_next_wake() { + let mut net = Net::leased(&[SILENT], Some(R)); + let first = net.resolve("www.example").unwrap(); + assert_eq!(net.run(first, 10 * WAIT_MS), Some(Err(Ended::Failed(Failure::Unreachable)))); + net.until(4_000); + let id = net.resolve("other.example").unwrap(); + net.pass(); + assert_eq!(net.lan.node.shard().ip().counters().get(toyos_net_ip::Counter::NbFailedRefused), 1, "the premise: [ip] refused the query on a failed neighbour, at 4 s"); + assert_eq!(net.run(id, 10 * WAIT_MS), Some(Err(Ended::Failed(Failure::Unreachable)))); + assert_eq!(net.ms(), 4_000 + WAIT_MS); + assert_eq!(net.lan.node.counters().get(Counter::QueryFailed), 3); } /// A lookup in flight as the lease's first renewal leaves: the net, the lookup and the renewal's diff --git a/userland/netstack/src/main.rs b/userland/netstack/src/main.rs index b1a522882aa..bb763e574e1 100644 --- a/userland/netstack/src/main.rs +++ b/userland/netstack/src/main.rs @@ -1293,6 +1293,9 @@ impl Netstack { // whether the name or only its address is missing. Err(Ended::Failed(Failure::NoSuchName | Failure::NoAddress)) => answer_lookup(&client, &[]), Err(Ended::Failed(Failure::TimedOut)) => client.error(ERR_TIMED_OUT), + Err(Ended::Failed(Failure::Unreachable)) => { + unreachable!("netstack: no lookup here is told a query did not reach its server, and {name}'s ended so") + } Err(Ended::Failed(why @ (Failure::Truncated | Failure::ServerFailed(_) | Failure::TooManyAliases))) => { say!("netstack: a lookup of {name} ended without an answer: {why:?}"); client.error(ERR_OTHER);