Repository navigation
libc binds a datagram socket at its first send, keeps a TCP_NODELAY for connect and for the connections a listener accepts, holds an IPv4 address as its octets, and truncates one to its caller's buffer - #787
Conversation
…or connect, and holds an address as its octets Three defects of libc's sockets, each filed by #783, each with a guest C case on tests/netcase that fails without its fix. A sendto on a datagram socket never bound wrote the datagram to handle 0 and asked netstack to send from socket 0. It now binds the socket to a port netstack chooses, through the one function bind uses, so a SO_BROADCAST set before the socket existed in netstack is handed over on either path. A TCP_NODELAY set on a stream socket before connect answered 0 and was stored nowhere. It is kept in the socket's entry and connect hands it to netstack once the connection exists; a refusal there closes the connection and fails the connect. On a datagram socket the option is EINVAL, as the host answers, where it used to reach netstack as a stream's request with a datagram socket's id. sockaddr_in's address was read with to_be_bytes and written with from_be_bytes, so an address libc made itself came back right and one built with htonl or inet_pton was reversed. inaddr.rs holds the struct with its port and address as bytes: no integer of either exists to be read in the machine's order, and every site that read or wrote one goes through it. The texts move there with it and are host-tested against the host C library's. inet_addr reads POSIX's four forms with ISO C's number forms, where it read only four decimal octets; inet_pton takes numbers of one to three digits, where it took any count; inet_ntop needs only the room its text takes, where it refused every buffer under 16 bytes; and each answers EAFNOSUPPORT or ENOSPC where POSIX names it. netinet/in.h gains in_addr_t, in_port_t and INADDR_BROADCAST, and TCP_NODELAY moves to netinet/tcp.h, where POSIX has it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RvnWQFcMuGqTHYhvSnTe8A
Every case names its peer by an address, so with addresses read in host order the datagram case sent to another host and waited on its answer until the hang ceiling, and the red was not the address case's own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RvnWQFcMuGqTHYhvSnTe8A
The source gate refuses a public IPv4 address in a tracked file, and three texts of the inet_addr differential were one: `cargo run -- --ci host` was red on them, one step of 78. They are RFC 5737's now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RvnWQFcMuGqTHYhvSnTe8A
setsockopt and getsockopt take #783's rule: the kept option by sockopt::kept, the value through switch and answer, a null pointer EFAULT. This stage's two changes sit on it: TCP_NODELAY on a datagram socket is EINVAL ahead of either, and a stream's is kept unless the socket is a connection. The never-bound issue stays deleted. Its modified side added the guest measurement of the defect, sendto answering -1 with EIO, which the fix this branch carries ends; the track's sentence that cited it is rewritten on #783's version of the line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RvnWQFcMuGqTHYhvSnTe8A
#783's host test of the option rule read the number from netinet/in.h, which this stage moved to the header POSIX has it in: the merge compiled and the test was red, one step of 78 in `cargo run -- --ci host`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RvnWQFcMuGqTHYhvSnTe8A
|
Evidence for the body at Mutations, each m1-sendto-binds-nothing.patch--- a/userland/libc/src/socket.rs
+++ b/userland/libc/src/socket.rs
@@ -424,14 +424,6 @@ pub unsafe extern "C" fn sendto(
Some(v) => v,
None => { set_errno(EINVAL); return -1; }
};
- // POSIX: a socket not yet bound is bound at its first send, to a
- // port the stack chooses.
- if entry.netstack_id == 0 {
- if let Err(e) = bind_datagram(entry, [0; 4], 0) {
- set_errno(net_err_to_errno(e));
- return -1;
- }
- }
// Write data to tx pipe
let data = core::slice::from_raw_parts(buf, len);
if let Err(_) = syscall::write(RawHandle(entry.tx_fd as u32), data) {m2-nodelay-dropped-before-connect.patch--- a/userland/libc/src/socket.rs
+++ b/userland/libc/src/socket.rs
@@ -208,14 +208,6 @@ pub unsafe extern "C" fn connect(fd: i32, addr: *const Sockaddr, addrlen: Sockle
Ok(c) => c,
Err(e) => { set_errno(net_err_to_errno(e)); return -1; }
};
- if entry.nodelay {
- if let Err(e) = toyos::net::tcp_set_option(conn.socket_id, OPT_NODELAY, 1) {
- // `conn`'s pipe ends close where it drops.
- let _ = toyos::net::tcp_close(conn.socket_id);
- set_errno(net_err_to_errno(e));
- return -1;
- }
- }
entry.netstack_id = conn.socket_id.0;
entry.local_port = conn.local_port;
entry.remote_addr = ip;
@@ -586,16 +578,13 @@ pub unsafe extern "C" fn setsockopt(
Err(refusal) => { set_errno(option_errno(refusal)); return -1; }
};
match option {
- Kept::NoDelay => {
- // netstack holds a connection from `connect` or `accept`; a
- // listener's id names none.
- if entry.connected {
- if let Err(e) = toyos::net::tcp_set_option(TcpSocketId(entry.netstack_id), OPT_NODELAY, on as u32) {
- set_errno(net_err_to_errno(e));
- return -1;
- }
+ Kept::NoDelay if entry.netstack_id != 0 => {
+ if let Err(e) = toyos::net::tcp_set_option(TcpSocketId(entry.netstack_id), OPT_NODELAY, on as u32) {
+ set_errno(net_err_to_errno(e));
+ return -1;
}
entry.nodelay = on;
}
+ Kept::NoDelay => {}
Kept::Broadcast => {
// Only a datagram is ever sent to a broadcast address, and netstack
// holds a datagram socket from its `bind`.m3-address-in-host-order.patch--- a/userland/libc/src/inaddr.rs
+++ b/userland/libc/src/inaddr.rs
@@ -20,12 +20,13 @@ pub(crate) struct SockaddrIn {
impl SockaddrIn {
pub(crate) fn new(ip: [u8; 4], port: u16) -> Self {
- Self { sin_family: AF_INET as u16, sin_port: port.to_be_bytes(), sin_addr: ip, sin_zero: [0; 8] }
+ Self { sin_family: AF_INET as u16, sin_port: port.to_be_bytes(), sin_addr: u32::from_be_bytes(ip).to_ne_bytes(), sin_zero: [0; 8] }
}
/// The address and port, or `None` for another family's.
pub(crate) fn endpoint(&self) -> Option<([u8; 4], u16)> {
- (self.sin_family == AF_INET as u16).then(|| (self.sin_addr, u16::from_be_bytes(self.sin_port)))
+ (self.sin_family == AF_INET as u16)
+ .then(|| (u32::from_ne_bytes(self.sin_addr).to_be_bytes(), u16::from_be_bytes(self.sin_port)))
}
}
diff --git a/userland/libc/src/socket.rs b/userland/libc/src/socket.rs
--- a/userland/libc/src/socket.rs
+++ b/userland/libc/src/socket.rs
@@ -849,5 +849,5 @@ pub unsafe extern "C" fn ntohl(netlong: u32) -> u32 {
#[no_mangle]
pub unsafe extern "C" fn inet_addr(cp: *const u8) -> u32 {
let text = core::slice::from_raw_parts(cp, super::string::strlen(cp));
- inaddr::numbers_and_dots(text).map_or(u32::MAX, u32::from_ne_bytes)
+ inaddr::numbers_and_dots(text).map_or(u32::MAX, u32::from_be_bytes)
}m4-connect-hands-nothing-over.patch--- a/userland/libc/src/socket.rs
+++ b/userland/libc/src/socket.rs
@@ -208,14 +208,6 @@ pub unsafe extern "C" fn connect(fd: i32, addr: *const Sockaddr, addrlen: Sockle
Ok(c) => c,
Err(e) => { set_errno(net_err_to_errno(e)); return -1; }
};
- if entry.nodelay {
- if let Err(e) = toyos::net::tcp_set_option(conn.socket_id, OPT_NODELAY, 1) {
- // `conn`'s pipe ends close where it drops.
- let _ = toyos::net::tcp_close(conn.socket_id);
- set_errno(net_err_to_errno(e));
- return -1;
- }
- }
entry.netstack_id = conn.socket_id.0;
entry.local_port = conn.local_port;
entry.remote_addr = ip;m5-bind-hands-no-broadcast-over.patch--- a/userland/libc/src/socket.rs
+++ b/userland/libc/src/socket.rs
@@ -135,13 +135,6 @@ unsafe fn fill_sockaddr(addr: *mut Sockaddr, addrlen: *mut SocklenT, ip: [u8; 4]
/// `SO_BROADCAST` set before it existed there.
fn bind_datagram(entry: &mut SocketEntry, ip: [u8; 4], port: u16) -> Result<(), NetError> {
let bound = toyos::net::udp_bind(ip, port)?;
- if entry.broadcast {
- if let Err(e) = toyos::net::udp_set_option(bound.socket_id, OPT_BROADCAST, 1) {
- // `bound`'s pipe ends close where it drops.
- let _ = toyos::net::udp_close(bound.socket_id);
- return Err(e);
- }
- }
entry.netstack_id = bound.socket_id.0;
entry.local_port = bound.bound_port;
entry.bound = true;The host peer ( peer.c/* The host run's peer, not in the tree: a TCP listener that holds what it
accepts and a UDP socket that answers each datagram with itself, both on
the loopback address. Prints "<tcp port> <udp port>" and serves until
killed. */
#include <arpa/inet.h>
#include <netinet/in.h>
#include <stdio.h>
#include <string.h>
#include <sys/select.h>
#include <sys/socket.h>
#include <unistd.h>
static unsigned bind_loopback(int fd) {
struct sockaddr_in me;
socklen_t len = sizeof me;
memset(&me, 0, sizeof me);
me.sin_family = AF_INET;
me.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
if (bind(fd, (struct sockaddr *)&me, sizeof me) != 0) return 0;
getsockname(fd, (struct sockaddr *)&me, &len);
return ntohs(me.sin_port);
}
int main(void) {
int l = socket(AF_INET, SOCK_STREAM, 0), u = socket(AF_INET, SOCK_DGRAM, 0);
unsigned tcp = bind_loopback(l), udp = bind_loopback(u);
listen(l, 16);
printf("%u %u\n", tcp, udp);
fflush(stdout);
for (;;) {
fd_set r;
FD_ZERO(&r);
FD_SET(l, &r);
FD_SET(u, &r);
if (select((l > u ? l : u) + 1, &r, 0, 0, 0) < 0) return 1;
if (FD_ISSET(l, &r)) accept(l, 0, 0);
if (FD_ISSET(u, &r)) {
char buf[64];
struct sockaddr_in from;
socklen_t len = sizeof from;
ssize_t n = recvfrom(u, buf, sizeof buf, 0, (struct sockaddr *)&from, &len);
if (n >= 0) sendto(u, buf, (size_t)n, 0, (struct sockaddr *)&from, len);
}
}
}The three cases on the host, round 1, step host-cases (the sources are logThe edges on the host, round 3, step host-edges: edges.c/* What the host's C library answers at the edges of the calls libc declares. */
#include <arpa/inet.h>
#include <errno.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
static void pton(const char *s) {
unsigned char o[4] = {0xaa, 0xaa, 0xaa, 0xaa};
errno = 0;
int r = inet_pton(AF_INET, s, o);
printf("inet_pton(\"%s\") = %d errno=%d bytes=%02x %02x %02x %02x\n", s, r, errno, o[0], o[1], o[2], o[3]);
}
static void addr(const char *s) {
in_addr_t a = inet_addr(s);
unsigned char o[4];
memcpy(o, &a, 4);
printf("inet_addr(\"%s\") bytes=%02x %02x %02x %02x\n", s, o[0], o[1], o[2], o[3]);
}
static void ntop(socklen_t size) {
unsigned char o[4] = {192, 0, 2, 1};
char buf[32];
memset(buf, '#', sizeof buf);
buf[31] = 0;
errno = 0;
const char *r = inet_ntop(AF_INET, o, buf, size);
printf("inet_ntop(192.0.2.1, size %u) = %s errno=%d (ENOSPC=%d)\n", (unsigned)size, r ? r : "NULL", errno, ENOSPC);
}
int main(void) {
const char *p[] = {"10.0.2.2", "192.0.2.4", "255.255.255.255", "0.0.0.0", "0192.0.2.4", "192.0.2.04", "00192.0.2.4", "000192.0.2.4",
"192.0.2.00", "192.0.2.0", "256.1.1.1", "192.0.2", "192.0.2.4.5", "192.0.2.4.", ".192.0.2.4", "1..2.3", "", " 192.0.2.4",
"192.0.2.4 ", "192.0.2.4x", "0x192.0.2.4", "192.0.2.-4", "+192.0.2.4", "192.0.2.4\n", "127.1"};
for (unsigned i = 0; i < sizeof p / sizeof *p; i++) pton(p[i]);
const char *a[] = {"10.0.2.2", "192.0.2.4", "255.255.255.255", "0.0.0.0", "127.1", "127.0.1", "1.2.65535", "1.2.65536", "1.16777215",
"1.16777216", "4294967295", "4294967296", "16909060", "0x7f.1", "0x7f000001", "0X7F.0.0.1", "010.0.0.1", "08.0.0.1",
"09", "0", "00", "0x", "0x.1", "192.0.2.4 ", "192.0.2.4 x", "192.0.2.4\n", " 192.0.2.4", "192.0.2.4x", "256.0.0.1", "1.256.0.1",
"1.2.256.1", "192.0.2.256", "192.0.2.4.5", "192.0.2.4.", "1.", ".1", "1..2", "", "-1", "+1", "192.0.2.0x10", "0377.1",
"0400.1", "0xff.1", "0x100.1", "1.0x10000", "a", "0xg"};
for (unsigned i = 0; i < sizeof a / sizeof *a; i++) addr(a[i]);
for (socklen_t n = 8; n <= 11; n++) ntop(n);
ntop(16);
char buf[64];
unsigned char big[4] = {255, 255, 255, 255};
printf("inet_ntop(255.255.255.255, 16) = %s\n", inet_ntop(AF_INET, big, buf, 16));
errno = 0;
printf("inet_ntop(255.255.255.255, 15) = %p errno=%d\n", (void *)inet_ntop(AF_INET, big, buf, 15), errno);
errno = 0;
printf("inet_pton(af 99) = %d errno=%d (EAFNOSUPPORT=%d)\n", inet_pton(99, "192.0.2.4", buf), errno, EAFNOSUPPORT);
errno = 0;
printf("inet_ntop(af 99) = %p errno=%d\n", (void *)inet_ntop(99, big, buf, 64), errno);
printf("INADDR_LOOPBACK=%08x INADDR_BROADCAST=%08x INADDR_ANY=%08x INADDR_NONE=%08x INET_ADDRSTRLEN=%d\n",
(unsigned)INADDR_LOOPBACK, (unsigned)INADDR_BROADCAST, (unsigned)INADDR_ANY, (unsigned)INADDR_NONE, INET_ADDRSTRLEN);
printf("sizeof in_addr_t=%zu in_port_t=%zu\n", sizeof(in_addr_t), sizeof(in_port_t));
/* Does a connection a listener accepts carry the listener's TCP_NODELAY? */
int l = socket(AF_INET, SOCK_STREAM, 0), on = 1;
struct sockaddr_in me;
memset(&me, 0, sizeof me);
me.sin_family = AF_INET;
me.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
socklen_t len = sizeof me;
printf("listener set TCP_NODELAY before bind: %d\n", setsockopt(l, IPPROTO_TCP, TCP_NODELAY, &on, sizeof on));
bind(l, (struct sockaddr *)&me, sizeof me);
listen(l, 1);
getsockname(l, (struct sockaddr *)&me, &len);
int c = socket(AF_INET, SOCK_STREAM, 0);
printf("connect: %d\n", connect(c, (struct sockaddr *)&me, sizeof me));
int acc = accept(l, 0, 0), v = -1;
len = sizeof v;
getsockopt(acc, IPPROTO_TCP, TCP_NODELAY, &v, &len);
printf("accepted socket's TCP_NODELAY: %s\n", v ? "set" : "clear");
int l2 = socket(AF_INET, SOCK_STREAM, 0);
me.sin_port = 0;
bind(l2, (struct sockaddr *)&me, sizeof me);
listen(l2, 1);
printf("listener set TCP_NODELAY after listen: %d errno=%d\n", setsockopt(l2, IPPROTO_TCP, TCP_NODELAY, &on, sizeof on), errno);
/* recvfrom and getsockname on a datagram socket never bound */
int d = socket(AF_INET, SOCK_DGRAM, 0);
len = sizeof me;
memset(&me, 0xaa, sizeof me);
printf("getsockname of an unbound datagram socket: %d port=%u\n", getsockname(d, (struct sockaddr *)&me, &len), ntohs(me.sin_port));
return 0;
}log
|
|
Review round 1 of #787 at Net lines, stage alone: 23 files, +809 −257. Production ( BLOCKER
NOTE
Re-measure on
|
Both files the stage and the merge touch, tests/toyos.rs and issues/toyos-has-its-own-network-stack.md, merged without a conflict; the stage against the merged branch is the same 23 files and the same hunks as against 8bceea0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RvnWQFcMuGqTHYhvSnTe8A
…sk_windows's stalls are folded into it Two guests of one run on this branch at 68cf1f8 went silent at the same step in the same second, virt_el1_smp and virt_mask_windows, each straight after the spawn record of test_rs_counters_read. virt_mask_windows's wait is named for the boot's last word because that is what it waits on after unmap_touch; the silence is the read's. So its own file is folded in, with its first occurrence and the one on wt/toyos-move-abi at 94fd25e, neither of which kept a console of the guest. Nothing cited the folded file. Each number was read off that run's log, none of which is in the tree. The issue gains the owner the tracker asks of it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RvnWQFcMuGqTHYhvSnTe8A
…ule and the address calls' refusals on the host tier, a sockaddr truncated to its caller's buffer A listener's TCP_NODELAY is refused again, as it was before this stage: netstack holds a listener from bind and answers its id not connected, and nothing is kept for it. The stage had turned that into a kept value no accepted connection carried. nodelay_kept.c gains the listener's lines. The issue says what is left (a value set before bind) and names the ABI stage that gives a listener its option as its exit. A TCP option asked of a datagram socket is sockopt.rs's rule now, with Linux's two answers by reading (ENOPROTOOPT to a setter, EOPNOTSUPP to a getter, before a value or a length is read), a row of the host differential, and Darwin's measured answers in darwin(). The four guest lines that held Darwin's EINVAL are gone. inet_pton's and inet_ntop's family refusal and inet_ntop's fit are in inaddr.rs, against the host at every buffer size from 0 to 17 for a text of each length and at four families neither has. inet_pton refuses a number with a zero before another digit, as glibc does, so no text has two values; Darwin reads it, and the differential says so. SockaddrIn is aligned as C's struct is, and answers a caller's buffer no more bytes than it holds, with the address's own length written back, as POSIX has accept, recvfrom, getpeername and getsockname truncate one: against the host's getsockname at every length from 0 to 32. That closes issues/libc-writes-a-whole-sockaddr-into-a-shorter-buffer.md, which nothing cited. The two netcase tests boot through one helper and share one holding server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RvnWQFcMuGqTHYhvSnTe8A
|
Round 6 evidence at
|
|
Answer to the round-1 review (#787 (comment)), at The base. #783 has not landed. This head carries #783 at BLOCKER
NOTE
The re-measure list
One red on the way: round 6's first clippy at |
|
Review round 2 of #787 at Net lines, stage alone: 26 files, +1200 −357. Production ( While this was read #783 landed: Round 1's BLOCKERs
The round's other changes
The stall recordsThe fold is justified by the evidence; it does not merge two defects under one name.
Every number of the new records is in the logs: 75.99 and 63.37, 1.52x, 366 s and 808 s, 35 of 37, both BLOCKERNone open. NOTE
The head that landsIt is this head with
A test of that list absent from the log is a red.
LAND AFTER NAMED CHANGES |
… case reads it on a connection the host dials in Round 2 of #787's review, its five NOTEs. accept: with a listener's set after bind refused, the value a listener holds is frozen at bind and is the option it held when any connection to it began, which is the host's rule and the own stack's. A program that set it before bind was told yes, read 1 on the listener and got connections without it. accept now sends a kept one for the connection it was answered and stores it in the accepted socket's entry, as connect does; a refusal there closes the connection and fails the accept with the request's errno, recorded in the reset issue's sentence. The test: nodelay_kept.c gains two listeners, one that held the option at bind and one that did not, and reads the option on the socket accept returns from each. Nothing dialled into a guest before: the harness now asks QEMU's monitor to forward two loopback ports of its choosing (hostfwd_add, read back from info usernet) and dials both when the case says its listeners wait. The same source on the host, Darwin 27.0.0, against a loopback peer that dials in: the first 25 lines are the guest's, and the three on the refused set are the recorded difference. What the test reads is libc's entry, as getsockopt answers from it: accept storing the value and sending netstack nothing stays green, which nothing-reads-whether-a-streams-tcp-nodelay-reached-netstack now says of accept beside connect. Issues. a-connection-libc-accepts-does-not-carry-its-listeners-tcp-nodelay is deleted: what remained of it, a set after bind refused until the pipe ABI gives a listener its option, is what a-listening-sockets-tcp-nodelay-is-refused-and-an-accepted-streams-is-answered-from-a-guess says and exits on; that file takes libc's half as it now stands and loses a message name the tree no longer has, and the track's listener-defects line names it. Filed: getsockname and getpeername answering a null buffer or a null length 0, with Darwin's answers measured. The counters record: the stall at 94fd25e is ten minutes after the two at 68cf1f8, not before them, and its compiler clause names the LLVM fault's file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RvnWQFcMuGqTHYhvSnTe8A
|
Answer to the round-2 review (#787 (comment)), at The merge. The five NOTEs1. diff --git a/userland/libc/src/socket.rs b/userland/libc/src/socket.rs
index 600916d99..4cbc660ee 100644
--- a/userland/libc/src/socket.rs
+++ b/userland/libc/src/socket.rs
@@ -64,7 +64,7 @@ struct SocketEntry {
// What netstack holds for the socket, which `getsockopt` answers from. A
// socket netstack does not hold yet keeps each for the call that makes it
// there to hand over: `broadcast` for a datagram socket's bind, `nodelay`
- // for a stream's `connect`.
+ // for a stream's `connect` and for each connection a listener accepts.
nodelay: bool,
broadcast: bool,
}
@@ -290,6 +290,9 @@ pub unsafe extern "C" fn accept(
};
let listener_id = TcpSocketId(entry.netstack_id);
let notify_fd = entry.notify_fd;
+ // A set after `bind` is refused, so this is the option the listener held
+ // when any connection to it began.
+ let nodelay = entry.nodelay;
// Block until a connection arrives (read 1 byte from notify pipe)
let mut notify_byte = [0u8; 1];
@@ -299,6 +302,14 @@ pub unsafe extern "C" fn accept(
Ok(a) => a,
Err(e) => { set_errno(net_err_to_errno(e)); return -1; }
};
+ if nodelay {
+ if let Err(e) = toyos::net::tcp_set_option(accepted.socket_id, OPT_NODELAY, 1) {
+ // `accepted`'s pipe ends close where it drops.
+ let _ = toyos::net::tcp_close(accepted.socket_id);
+ set_errno(net_err_to_errno(e));
+ return -1;
+ }
+ }
if !addr.is_null() && !addrlen.is_null() {
fill_sockaddr(addr, addrlen, accepted.remote_addr, accepted.remote_port);
@@ -315,7 +326,7 @@ pub unsafe extern "C" fn accept(
rx_fd: accepted.rx.into_raw().0 as i32,
tx_fd: accepted.tx.into_raw().0 as i32,
notify_fd: 0,
- nodelay: false,
+ nodelay,
broadcast: false,
};
let new_fd = alloc_socket(new_entry);A hand-over refused because the peer already reset closes the connection and fails the Tested, with a change inside
The issue. 2. The track names that file in its listener-defects line, with what its exit waits on: the pipe ABI giving a listener its option, which the node has ( 3. Filed, not fixed: 4. The counters record. "Two earlier stalls" is "Two other stalls", and the 5. The body no longer says #783 has not landed, and its paragraph on Gates at
|
| gate | exit | |
|---|---|---|
cargo run -- --build-only |
0 | |
cargo run -- --ci host |
0 | "Host: 78 step(s), all green" |
cargo run -- --clippy |
0 | |
cargo test --test toyos-build -- --jobs 1 --nocapture libc_sockets |
0 | 1 passed, 1 total |
| m2, (f), (h) | guest 1 each | tree clean after each |
| (i) | guest 0 | green, recorded |
cargo test --test toyos-build -- --jobs 4 --nocapture |
0 | 37 passed, 37 total; load 33.19 to 35.33 on 14 cores, ceilings at 1.00x; no stall |
The two new patches
Each applied with git apply --check and git apply, run, restored with git apply -R.
h-accept-carries-nothing.patch:
diff --git a/userland/libc/src/socket.rs b/userland/libc/src/socket.rs
index 4cbc660ee..cd5e74534 100644
--- a/userland/libc/src/socket.rs
+++ b/userland/libc/src/socket.rs
@@ -290,9 +290,6 @@ pub unsafe extern "C" fn accept(
};
let listener_id = TcpSocketId(entry.netstack_id);
let notify_fd = entry.notify_fd;
- // A set after `bind` is refused, so this is the option the listener held
- // when any connection to it began.
- let nodelay = entry.nodelay;
// Block until a connection arrives (read 1 byte from notify pipe)
let mut notify_byte = [0u8; 1];
@@ -302,14 +299,6 @@ pub unsafe extern "C" fn accept(
Ok(a) => a,
Err(e) => { set_errno(net_err_to_errno(e)); return -1; }
};
- if nodelay {
- if let Err(e) = toyos::net::tcp_set_option(accepted.socket_id, OPT_NODELAY, 1) {
- // `accepted`'s pipe ends close where it drops.
- let _ = toyos::net::tcp_close(accepted.socket_id);
- set_errno(net_err_to_errno(e));
- return -1;
- }
- }
if !addr.is_null() && !addrlen.is_null() {
fill_sockaddr(addr, addrlen, accepted.remote_addr, accepted.remote_port);
@@ -326,7 +315,7 @@ pub unsafe extern "C" fn accept(
rx_fd: accepted.rx.into_raw().0 as i32,
tx_fd: accepted.tx.into_raw().0 as i32,
notify_fd: 0,
- nodelay,
+ nodelay: false,
broadcast: false,
};
let new_fd = alloc_socket(new_entry);i-accept-stores-and-sends-nothing.patch:
diff --git a/userland/libc/src/socket.rs b/userland/libc/src/socket.rs
index 4cbc660ee..b5b3826d1 100644
--- a/userland/libc/src/socket.rs
+++ b/userland/libc/src/socket.rs
@@ -302,14 +302,6 @@ pub unsafe extern "C" fn accept(
Ok(a) => a,
Err(e) => { set_errno(net_err_to_errno(e)); return -1; }
};
- if nodelay {
- if let Err(e) = toyos::net::tcp_set_option(accepted.socket_id, OPT_NODELAY, 1) {
- // `accepted`'s pipe ends close where it drops.
- let _ = toyos::net::tcp_close(accepted.socket_id);
- set_errno(net_err_to_errno(e));
- return -1;
- }
- }
if !addr.is_null() && !addrlen.is_null() {
fill_sockaddr(addr, addrlen, accepted.remote_addr, accepted.remote_port);The host measurements
nodelay_kept on the host, its two listeners dialled once it says they wait:
Darwin 27.0.0
$ cc -Wall -Wextra -Werror -o nodelay_kept tests/netcase/nodelay_kept.c
EXIT=0
$ ./nodelay_kept 127.0.0.1 <the peer's stream port> <a free port> <another>, each listener dialled once the case says it waits
the peer's address is one: 1
a fresh stream socket's TCP_NODELAY: 0
set before connect: 0
read before connect: 1
connect: 0
read after connect: 1
cleared on the connection: 0
read after the clear: 0
set on the connection: 0
read after the set: 1
set on a second socket: 0
and cleared before connect: 0
connect: 0
read after connect: 0
set on a third socket before bind: 0
bind it: 0
listen on it: 0
read from that listener: 1
bind a fourth socket: 0
listen on it: 0
nodelay_kept: both listeners wait for a peer
accept from the listener that held the option: 1
read from its connection: 1
accept from the listener that held none: 1
read from its connection: 0
set on the listener: 0 <-- WRONG
which is refused as not connected: 0 <-- WRONG
read from the listener: 1 <-- WRONG
nodelay_kept: 3 wrong
EXIT=1
getsockname and getpeername with null pointers, on a connected stream:
Darwin 27.0.0
CC_EXIT=0
getsockname null buffer, length 0: 0 errno 0 (-)
length after: 16
getsockname null buffer, length 16: -1 errno 14 (Bad address)
length after: 16
getsockname null length: -1 errno 14 (Bad address)
getpeername null buffer, length 0: 0 errno 0 (-)
length after: 16
getpeername null buffer, length 16: -1 errno 14 (Bad address)
length after: 16
getpeername null length: -1 errno 14 (Bad address)
accept null buffer, null length: 0 errno 0 (-)
EXIT=0
|
Review round 3 of #787 at Net lines of the branch against The merge is git's own. Round 2No BLOCKER was open. Its five NOTEs:
The hunk in
|
The file records that libc's connect and accept fail ENOTCONN, with the connection closed, when the peer resets before the TCP_NODELAY hand-over. Its exit was met by a guest test whose nodelay() answers Ok, which asks netstack nothing, so both failures would have stood with no record left. The exit now also asks that netstack answers the option request on a stream its peer reset, and that connect and accept answer as the host does, each by a test that can be red. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RvnWQFcMuGqTHYhvSnTe8A
|
CI at
|
One conflict, in the track's line on the broadcast permission. #787 changed that line's last sentence only: the C case now runs the sequence without `bind` too, held by `tests/netcase/sendto_unbound.c`, where the sentence waited on an issue #787 closes and deletes. This branch rewrote the rest of the line. The line is this branch's with #787's ending. The line after it, on a logged refusal waiting in the stack, stays deleted: #787 did not touch it, and this branch met its exit. #787's other hunk in the track, the listener's `TCP_NODELAY` issue named among the listener defects, merged by itself. No line of the merged tree names any of the four issue files #787 deleted, by path or by bare name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RvnWQFcMuGqTHYhvSnTe8A
Head
5d9c996e2:2c8a4ad58, the head every measurement below was taken at, and one commit over it that changes one file ofissues/alone, the exit condition ofa-stream-its-peer-reset-refuses-the-option-requests-a-host-answers.md(git diff 2c8a4ad58 5d9c996e2 --stat: 1 file, +4 −1;cargo test --lib sourcegateat it, exit 0). At2c8a4ad58: this stage onmainat0d6cd9a60, which carries #783, the change that filed the three defects this closes. The merge ofmain(f1b8b85e7) was git's own, no conflict: its tree iseae4f5926b06, the treegit merge-tree --write-tree 0d6cd9a60 7d82642f3writes, and againstmainit was the stage's 26 files alone, +1200 −357 (round 7, merge). At2c8a4ad58the stage is 27 files, +1304 −363, and at5d9c996e2+1308 −364,issues/+216 −122:userland/libc+301 −179, tests +791 −63,issues/+212 −121. Round 7 over round 6 is one hunk ofacceptinuserland/libc/src/socket.rs,tests/netcase/nodelay_kept.c, the case's side oftests/toyos.rs, andissues/.Three defects of libc's sockets, each fixed at its owner in
userland/libc, each with a guest C case that is red without its fix, and each issue deleted; and, by the round-1 review, a fourth: an address written past a caller's buffer.What changed, per decision
sendtoon a socket netstack does not hold wrote the datagram to handle 0 and asked netstack to send from socket 0. It now binds to a port netstack chooses first, as POSIX has it.bindand that first send go through one function,bind_datagram, so aSO_BROADCASTset before the socket existed in netstack is handed over on either path, with The pipe ABI carries a datagram socket's broadcast permission from std and libc to netstack, and the option read is gone #783's refusal handling.TCP_NODELAYset beforeconnectis kept and handed over.setsockoptstores it for a stream netstack does not hold yet;connectsends a kept one oncetcp_connecthas answered, and a refusal there closes the connection and fails theconnect.TCP_NODELAYset afterbindstays refused, and one set beforebindreaches the connections it accepts, the orchestrator's decisions on round 1's second BLOCKER and round 2's first NOTE. The guard insetsockoptisnetstack_id != 0: netstack holds a listener frombindand answers its idERR_NOT_CONNECTED, so the set is -1ENOTCONNand nothing is kept for it. That freezes a listener's value atbind, so it is exactly the option the listener held when any connection to it began, which is the host's rule and the own stack's.accepthands a kept one to netstack for the connection it was answered and stores it in the accepted socket's entry, asconnectdoes; before this a program that set it beforebindwas told yes, read 1 on the listener and got connections without it. A refusal of the hand-over, a peer that reset in between, closes the connection and fails theacceptwith the request'serrno, asconnect's path does:issues/a-stream-its-peer-reset-refuses-the-option-requests-a-host-answers.mdsays so. What is left, the set afterbind, isissues/a-listening-sockets-tcp-nodelay-is-refused-and-an-accepted-streams-is-answered-from-a-guess.md, whose exit is the pipe ABI giving a listener its option.sockopt.rs, for every name at TCP's level, ahead of the value's length and pointers. The answer implemented is Linux's, by reading and not by measurement:ENOPROTOOPTto a setter (udp_setsockoptfalls toip_setsockopt, which refuses a level that is notSOL_IP) andEOPNOTSUPPto a getter (do_ip_getsockopt's refusal of the same). Darwin answersEINVAL, andEFAULTto a setter handed a null value of one byte or more (measured, round 6, host-dgram-nodelay); that is indarwin(). CI's Linuxhostjob is the measurement, read by the orchestrator at the head that lands: a Linux red on the rowNoDelay of a datagram socketinsocket_options::an_options_value_crosses_as_the_hosts_doessends this back.userland/libc/src/inaddr.rsholdsSockaddrInwithsin_port: [u8; 2]andsin_addr: [u8; 4]: no integer of either exists to be read in the machine's order. It is#[repr(C, align(4))], C's alignment, asserted with its size.fill_sockaddr, whichaccept,recvfrom,getpeernameandgetsocknameanswer through, wrote sixteen bytes whatever*addrlenheld.SockaddrIn::answerwrites as many as*addrlenholds and writes the address's own length back, as POSIX has it and as the host'sgetsocknamedoes at every length from 0 to 32. One function;issues/libc-writes-a-whole-sockaddr-into-a-shorter-buffer.mdis deleted, and nothing cited it.ptonandntophold the family refusal andinet_ntop's fit;numbers_and_dotsisinet_addr.inet_pton, and sogetaddrinfo, refuses a number with a zero before another digit, as glibc does:inet_addrreads010as octal, and no text has two values. Darwin reads such a number as decimal with any count of digits, which the differential states (darwin_reads).netinet/in.hgainsINADDR_BROADCAST,in_addr_tandin_port_t;arpa/inet.hdeclaresinet_addrasin_addr_t;TCP_NODELAYmoves to a newnetinet/tcp.h, where POSIX has it.tests/toyos.rs's two netcase tests boot through oneboot_netcase, which owns the lease line, and share oneholding_server. Something dials into a guest, fornodelay_kept's two listeners:forwards_intoasks QEMU's monitor for a forward from a loopback port of QEMU's choosing to each guest port (hostfwd_add, the host ports read back frominfo usernet), andlibc_socketsdials both when the case says its listeners wait (run_test_hooked, on the guest's own line and no clock). Measured before it was built on: QEMU 11.1.1 with no guest takeshostfwd_add net0 tcp:127.0.0.1:0-:<port>and names the port it bound. All of it is intests/toyos.rs;tests/commonis untouched.Declared functions that were wrong, fixed here
inet_addrinet_ptonEAFNOSUPPORTfor another familyinet_ntoperrnoENOSPConly when the text and its NUL do not fit, the buffer left as it was; null andEAFNOSUPPORTfor another familyaccept,recvfrom,getpeername,getsockname*addrlenbytes, 16 written backTests, and why each is on its tier
Host,
toyos-libc-copies.internet_addresses.rs, nine tests: the readers and the writer against the host'sinet_pton,inet_addrandinet_ntop;inet_ntopinto a buffer of every size from 0 to 17 for a text of each length from 7 to 15, comparing the answer, the host'serrnoand both buffers whole; four families neither libc nor the host has, through both calls;SockaddrIn's bytes, size and alignment; andSockaddrIn::answeragainst the host'sgetsocknameat every length from 0 to 32.socket_options.rs: #783's table gains the row ofTCP_NODELAYasked of a datagram socket, set and get at every length, null and not, and a null length.Guest, one machine test
libc_socketson one boot oftests/netcase, three C programs against two servers the harness starts. Why QEMU: what is asserted is what libc's calls ask of netstack and what comes back, and netstack is one binary that owns its NIC with no host build; the T14's shared boots run no netstack with a peer the harness controls. The four datagram-socket lines round 1 had there are gone to the host tier. The listener's lines are innodelay_kept.cbecause each is netstack's answer to libc's request: the refusal of a set with a listener's id, and the accepted socket's option, 1 from a listener that held it atbindand 0 from one that did not, which needs a connection netstack accepted from a peer.socket.rshas no host build, so no host test reaches either, and the boot is one the case already has. What the accepted socket's read sees is libc's entry, whichgetsockoptanswers from:acceptcarrying nothing is red (h), andacceptstoring the value and sending netstack nothing is green (i), underissues/nothing-reads-whether-a-streams-tcp-nodelay-reached-netstack.mdbesideconnect's.Oracle: POSIX's text and one host
Each case is the same source on the host, built with
cc -Wall -Wextra -Werrorand run on the development machine, Darwin 27.0.0, against a loopback peer (round 6, host-cases;nodelay_keptagain at this head's source with a peer that dials both listeners in, round 7, host-case-worktree).addr_orderandsendto_unbound: the guest's output is byte for byte the host's (diffexit 0, 19 and 17 lines).nodelay_kept: its first 25 lines are the host's, the two accepted sockets' reads among them, and the three on a listener's set afterbindare not, which is the recorded difference and no accident:Linux is unread: no program here was run on it, and no Linux container runtime was running on the development machine. The differential tests run against whichever C library the host has, so CI's
hostjob is their first reading against glibc and against Linux'ssetsockopt,getsockoptandgetsockname. What rests on my reading until then: the datagram row's two words and that Linux says them before it reads a length; that glibc'sinet_ptonrefuses exactly a zero before another digit; thatgetsocknamecopiesmin(len, 16)and writes 16 back. The tests to read by name in that job's log are the review's list, withinternet_addresses::a_buffer_too_short_for_the_text_is_refused_as_the_host_refuses_it,internet_addresses::another_family_is_refused_as_the_host_refuses_itandinternet_addresses::an_address_is_truncated_to_the_callers_buffer_as_the_host_truncates_it.Negative controls
Each a checked patch (
git apply --check), applied, run, restored withgit apply -R, tree clean after. m2, (f), (h) and (i) are at2c8a4ad58, round 7, sincesocket.rschanged under them; the rest are at7d82642f3, round 6, and nothing they mutate or run has changed since butsocket.rs'saccept. The patches are in comments below. Host iscargo test -p toyos-libc-copies; guest iscargo test --test toyos-build -- --jobs 1 --nocapture libc_sockets.sendtobinds nothingsendto_unbound, 11 lines:sendto from it: -1TCP_NODELAYdropped beforeconnect(no hand-over, a set before netstack holds the socket keeps nothing)nodelay_kept, 4 lines:read before connect: 0,read after connect: 0,read from that listener: 0,read from its connection: 0a_sockaddr_holds_its_port_and_address_in_network_orderand the truncation test;addr_order, 8 linesinet_ntop's>=to>: the NUL one byte past a buffer sized exactlya_buffer_too_short_for_the_text_is_refused_as_the_host_refuses_itinet_ntoprefuses every buffer under 16inet_ptonanswers 0 for another familyanother_family_is_refused_as_the_host_refuses_itfill_sockaddr)an_address_is_truncated_to_the_callers_buffer_as_the_host_truncates_itan_options_value_crosses_as_the_hosts_does:SetwhereRefused(92)entry.connected)nodelay_kept: the listener's three linesinet_ptonreads a zero-led numbera_number_with_a_zero_before_another_digit_is_no_dotted_quadand the differential's countacceptcarries nothing: the round's hunk revertednodelay_kept, 1 line:read from its connection: 0on the socket accepted from the listener that held the optionacceptstores the listener's value and sends netstack nothing(a), (b) and (c) are the review's, moved with the code: the fit and the family refusal are
inaddr.rs's now, so (b) and (c) mutate the refusal there, and theerrnoeach maps to is onematchinsocket.rsa reader checks. On this host (e) is red by the test's own assertion of libc's two words and not by the host's answer, which is Darwin's; the host's side of that row is Linux's to give.m4, (i) and m5 stay green, as the review decided of the first and the last: m4 (
connecthands no kept option over) and (i), its like ataccept, areissues/nothing-reads-whether-a-streams-tcp-nodelay-reached-netstack.md, whose exit names both calls, and m5 (bind_datagramhands no keptSO_BROADCASTover) is the track's broadcast line. m4 and m5 not run again: measured at19e2d33ea, round 1.Issues
Closed, each by its own exit:
libc-sends-from-a-datagram-socket-it-never-bound(m1),libc-drops-a-tcp-nodelay-set-before-connect(m2),libc-reads-an-ipv4-address-in-host-order(m3),libc-writes-a-whole-sockaddr-into-a-shorter-buffer(d).git grepof each slug at the head finds nothing.Filed:
nothing-reads-whether-a-streams-tcp-nodelay-reached-netstack,libc-getaddrinfo-ignores-its-service-and-its-hints, andlibc-getsockname-and-getpeername-answer-a-null-buffer-with-success, round 2's third NOTE, not fixed here: its three rows of the host's answers are measured on Darwin 27.0.0 (round 7, host-null-sockname) and are its exit's oracle, libc its owner.a-stream-its-peer-reset-refuses-the-option-requests-a-host-answersgains the sentence onconnect's andaccept'sENOTCONNwhen a peer resets between netstack's answer and the hand-over.a-connection-libc-accepts-does-not-carry-its-listeners-tcp-nodelay, filed in round 6, is deleted, nothing citing it: withacceptcarrying the value set beforebind, what remained of it is the set afterbind, refused until the pipe ABI gives a listener its option, whicha-listening-sockets-tcp-nodelay-is-refused-and-an-accepted-streams-is-answered-from-a-guess, onmainsince #777, already says and exits on. That file is made true of libc as it stands (the refusal'serrno, the value kept beforebindand handed over ataccept, the test that holds both and the three lines its exit turns), losesTcpGetOption, a message #783 removed, and is named in the track's listener-defects line with what its exit waits on.The stalls (
fe241fc3e).issues/a-counters-read-under-host-load-can-go-silent-for-15-s.mdgains round 5's two silent guests of one run, with their six lines, the load and the waits, and the owner the tracker asks for.issues/virt-mask-windows-went-quiet-on-a-loaded-host.mdis folded into it and deleted, nothing citing it (round 7 corrects one word of the fold, the stall at94fd25e71being ten minutes after the two at68cf1f870by the runs' logs and not before them, and namesissues/the-forks-llvm-deletes-a-loops-exit-on-a-no-wrap-flag-scalar-evolution-gives-the-wrong-value.mdwhere it spoke of the compiler):virt_mask_windows's second wait spans the counters read, and the one stall of it with a console ends on the read's spawn record. Its two other stalls, atb9430ed61and onwt/toyos-move-abiat94fd25e71, kept no console of the guest and are recorded as that: the step each stopped in is not read from it. Nothing in its recorded evidence shows a different defect.Gates
Each is the command's own exit code, at
2c8a4ad58with a clean tree; logs are named by round and step.cargo run -- --build-only(round 7, build)cargo run -- --ci host(round 7, ci-host)toyos-libc-copies46 passed inside itcargo run -- --clippy(round 7, clippy)cargo test --test toyos-build -- --jobs 1 --nocapture libc_sockets(round 7, guest-libc-sockets)cargo test --test toyos-build -- --jobs 4 --nocapture(round 7, guest-suite)The suite ran once, on the 14-core host with other worktrees building: one-minute load 33.19 at its start and 35.33 at its end; liveness ceilings paid at 1.00x; workers 501 s building against 284 s testing.
libc_sockets,netstack_socket_churn,virt_mask_windows,virt_el1_smpandvirt_smppassed in 12, 9, 5, 11 and 10 s; no stall, so the counters record gains no occurrence, and one green run is no evidence about the read under load. CI'shost(Linux) andguest / suiteunder KVM are read by the orchestrator at the head that lands.What I am unsure of
userland/libc/src/socket.rs: libc builds only for the guest and no shape lints it.inaddr.rsandsockopt.rsare linted through the host crate.connectand atacceptis unseen (m4, (i)): the tests read libc's entry, and nothing reads netstack's.accept's hand-over fails theacceptENOTCONN: read from the code and recorded, reached by no test.🤖 Generated with Claude Code
https://claude.ai/code/session_01RvnWQFcMuGqTHYhvSnTe8A