Skip to content

TCP/IP + Basic async runtime. - #6

Merged
supernomad merged 137 commits into
mainfrom
net-stack
Mar 17, 2026
Merged

supernomad merged 137 commits into
mainfrom
net-stack

Conversation

@supernomad

Copy link
Copy Markdown
Contributor

This implements a partial but robust network stack including:

  • Ethernet
  • Neighbor handling (Arp + Ndp)
  • PMTU
  • IPv4/v6
  • ICMPv4/v6
  • Udp
  • Tcp

@codecov

codecov Bot commented Mar 10, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.77836% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.60%. Comparing base (f99bc4b) to head (4b763e0).
⚠️ Report is 137 commits behind head on main.

Files with missing lines Patch % Lines
src/net/fragment/reader.rs 97.68% 13 Missing ⚠️
src/net/fragment/writer.rs 97.38% 8 Missing and 3 partials ⚠️
src/net/handler/icmpv4.rs 98.64% 2 Missing and 5 partials ⚠️
src/net/checksum/common.rs 98.80% 0 Missing and 2 partials ⚠️
src/net/fragment/pkt.rs 99.22% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main       #6      +/-   ##
==========================================
+ Coverage   88.37%   89.60%   +1.23%     
==========================================
  Files          44      101      +57     
  Lines        3767    22433   +18666     
  Branches     3767    22433   +18666     
==========================================
+ Hits         3329    20101   +16772     
- Misses        375     2138    +1763     
- Partials       63      194     +131     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

neli's typed attribute deserialization misparses ethtool CHANNELS_GET
responses that mix nested (Header) and flat (count) attributes at the
same level. Switch to raw netlink socket I/O for the response and parse
NLA attributes manually.

Also treat 0xFFFFFFFF as "not applicable" (ethtool's encoding for n/a
channels, common on virtio-net/cloud VMs).
The ethtool command ID was wrong: we used 4 (LINKMODES_GET) instead of
17 (CHANNELS_GET). The attribute enum was also misordered — OtherMax and
OtherCount were placed at the end instead of their correct positions (4
and 8), shifting RxCount/TxCount/CombinedMax/CombinedCount by 2.
This was becoming a maintenance burden and at the end of the day wasn't
especially viable as a operating mode the new rt will replace.
Design spec and implementation plan for making the event loop's TX
completion drain non-blocking to fix multi-queue scaling regression
(342K vs 544K req/s, 6.4x per-core efficiency drop).
Replace the synchronous completion drain in LocalRuntime::run() with an
in-flight TX counter. Completions are collected opportunistically at the
top of each iteration rather than blocking until all arrive. This removes
the per-iteration kernel round-trip that was throttling multi-queue
throughput.

Key changes:
- Add in_flight_tx counter tracking frames in kernel TX pipeline
- Drain completions non-blocking at loop start (Phase 1)
- Send tx_return non-blocking with one retry on WouldBlock
- Extract recycle_rx_return() helper (fill queue first, overflow to free_frames)
- Fix pre-existing invariant gap: use <= to account for FragmentReader frames

New invariant: free_frames + in_flight_tx + tx_return <= expected_total
(replaces strict free_frames == expected_free_frames)

Spec: docs/superpowers/specs/2026-03-17-async-tx-completion-design.md
socket.send() only writes descriptors to the TX ring — the kernel needs
a sendto() syscall (via maybe_wake) to start DMA processing. The async
completion drain removed the synchronous completion loop which had been
implicitly providing this kick. Without it, frames sat in the TX ring
and never transmitted.
NeighborHandler is only accessed single-threaded via Rc within
LocalRuntime, but was using DashMap (a concurrent hashmap) for the
neighbor table. Perf profiling showed 22.7% of CPU in
lookup_or_resolve and another 15.4% in DashMap's atomic CAS
operations (cas8_acq/cas8_rel) — a total of ~38% CPU wasted on
unnecessary synchronization.

Replace DashMap with FxHashMap wrapped in UnsafeCell for interior
mutability (same pattern as PmtuCache and TcpHandler). This
eliminates all atomic operations, shard locking, and replaces
SipHash with FxHash for lookups.

Changes:
- handler.rs: DashMap → UnsafeCell<FxHashMap>, add table() helper
- arp.rs: &DashMap param → &mut FxHashMap
- ndp.rs: &DashMap param → &mut FxHashMap (4 functions)
- Cargo.toml: remove dashmap dependency
…ghbor lookup

Perf profiling showed lookup_or_resolve consuming 33% of CPU — called
on every TCP segment in poll_send, doing a FxHashMap probe each time
even though the destination MAC never changes for an established
connection.

Add dst_mac: Option<MacAddress> to Tcb. On first successful neighbor
resolution, the MAC is cached. All subsequent segments on that
connection use the cached value directly, bypassing the hash table
entirely.

5 call sites in transmit.rs updated with cache-first pattern:
- Data segment send loop (hot path)
- Delayed ACK fallback
- Zero-window probe
- Linger RST
- FIN transmission
The previous MAC caching commit only updated transmit.rs (poll_send),
but timers.rs has 4 additional lookup_or_resolve call sites that were
bypassing the cache. More critically, listen.rs was initializing
dst_mac: None even though the inbound path already knows the remote
MAC from the incoming Ethernet frame header.

Fixes:
- listen.rs: initialize dst_mac from incoming frame's src_mac
- timers.rs: add cache-first pattern to all 4 lookup_or_resolve sites
  (delayed ACK, keep-alive, SACK recovery, RTO retransmit)
Add mpsc channel mesh so multi-queue workers propagate ARP/NDP neighbor
updates to peer queues. Each LocalRuntime drains its receiver at the top
of the event loop before packet processing.
@supernomad
supernomad merged commit 4b763e0 into main Mar 17, 2026
5 of 6 checks passed
@supernomad
supernomad deleted the net-stack branch March 17, 2026 22:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant