Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 33 additions & 12 deletions .claude/skills/elfuse-conventions/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,10 +130,9 @@ coverage. Both fail rather than skip when their tool is missing.

`make indent` is a no-op on a clean tree, in both halves: every file clang-format
selects already formats to itself, and every file commentflow selects already
reflows to itself. That was not free. The tree's comments were wrapped by hand
before the tool existed, and the one-time reflow rewrote 142 of the 353 C and
header files, both assembly files, and 36 of the 41 shell scripts. It landed as
its own commit, under the rule the commit section states.
reflows to itself. The tree's comments were wrapped by hand before the tool
existed, and the one-time reflow landed as its own commit, under the rule the
commit section states.

The gate is what keeps it a no-op. If `make indent` ever hands you a diff in a
file you did not touch, something reintroduced hand-wrapping, or your
Expand Down Expand Up @@ -169,8 +168,8 @@ is a sequentially-consistent read-modify-write, and a site already holding the
lock that serializes it pays for ordering it does not need while saying nothing
about the ordering it does. Where a file has many such sites, name the
discipline once in helpers rather than spelling the order out at each: the
`pending_load` / `pending_or` / `pending_clear` group at the top of
`src/syscall/signal.c` is the shape.
`pending_load` / `pending_or` / `pending_clear` group in
`src/syscall/signal.h` is the shape.

Never hand an `_Atomic` object to `memcpy` or to a guest read/write helper. That
copies the object representation, which is not an atomic read of it. Load into a
Expand All @@ -192,9 +191,11 @@ states no more about the ordering than the plain operator does.

`scripts/check-atomics.py` holds the two halves a regex can settle: no
`__atomic_*` or `__sync_*`, and no C11 atomic call without its `_explicit`
form. It does not check plain-operator access to an `_Atomic` object, because
finding those needs the declarations resolved and the tree still carries a large
pre-existing set of them; that half stays a review question.
form. It reads these skill files too, under the banned-spelling half only, so
prose may quote a bare `atomic_load` but not a concrete `__atomic_*` name or an
`__ATOMIC_*` order constant; write either family with the star, as this
paragraph does. That script's module docstring carries the reasoning, and why
plain-operator access to an `_Atomic` object is left a review question.

State the order and name what it pairs with. Relaxed is right under a lock that
already serializes the access. Release and acquire are for a publish a lock-free
Expand Down Expand Up @@ -265,9 +266,13 @@ flag, or helper is named for its operation, not the manner.

Not in `src/`: attribution, dates, commented-out code, issue-tracker numbers
(barred from `docs/` and README prose too, PR#40, PR#223; they belong in a
commit trailer), or `TODO`/`FIXME`; incomplete work belongs in the commit
message or PR. Editing part of a comment re-opens all of it: re-read the
block and rewrite what no longer reads cleanly.
commit trailer), or a bare `TODO`/`FIXME`; incomplete work belongs in the
commit message or PR. `CONTRIBUTING.md` carries the marker rule, which asks a
`TODO` to say what remains and to carry an all-caps owner ahead of the word
when one applies. What `src/` writes instead is the stage or the condition the
work waits on, as a parenthesis (`grep -rn 'TODO(' src/`). Editing part of a
comment re-opens all of it: re-read the block and rewrite what no longer reads
cleanly.

Mechanics: `/* */` only in `.c`, `.h`, and `.S`, no `//`, no Doxygen tags;
multi-line blocks align on ` * `, close with `*/` on its own line, indented
Expand Down Expand Up @@ -417,3 +422,19 @@ would push one person's habit onto everybody.

Build and toolchain requirements are in `docs/testing.md`, section "Build
Requirements". They belong to a machine, not to this convention set.

## Authoritative sources

- `CONTRIBUTING.md` for C style, the formatter, and the commit-message rules;
it wins where both files speak.
- `scripts/check-atomics.py`, its module docstring, for what the atomics gate
checks and what it deliberately leaves to review.
- `scripts/check-ascii.py` for the character-set gate, which reads only `.c`,
`.h` and `.S` under `src`, `tests` and `frama-c-stubs`: the markdown half of
the em dash ban has no gate behind it and stays a review question.
- `scripts/check-skill-refs.py` for how a path, target, or section named in
these files is resolved.
- `scripts/install-git-hooks.sh` for the hooks a fresh clone installs. They
run `.ci/check-format.sh` and `.ci/check-commentflow.sh` at commit time and
the commit-log check at push time; none of the gates above is among them, so
those first fail at `make check` or in CI.
7 changes: 5 additions & 2 deletions .claude/skills/elfuse-debug/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ classification is the first bisection, and it is free:
| `CRASH_UNEXPECTED_HVC` / `CRASH_UNEXPECTED_EC` | The shim and the host dispatcher disagree about the protocol. Usually a half-landed HVC change. |
| `CRASH_HV_CHECK` | Hypervisor.framework refused a call. Host-side: a mapping or permission elfuse asked for is not one HVF allows. |
| `CRASH_ELR_ZERO` | Register state after exec is not what the host wrote. A return-to-EL0 path problem, not a loader problem. |
| `CRASH_UNEXPECTED_EXIT` | `hv_vcpu_run()` returned an exit reason the loop does not handle. Host-side: HVF or the run loop, not the guest. |
| `CRASH_TIMEOUT` | One `hv_vcpu_run()` iteration exceeded the `--timeout` watchdog. |

`CRASH_TIMEOUT` is the one that gets misread. The watchdog bounds a single run
Expand Down Expand Up @@ -198,6 +199,8 @@ so prefer them when the two disagree:
`--timeout`, `--fakeroot`, and `ELFUSE_FAKEROOT_EXEC`.
- `docs/internals.md`, section "GDB Stub" - the snapshot protocol and the
`src/debug/` split.
- The header comments in `src/core/startup-trace.h`, `src/debug/syscall-hist.h`,
and `src/debug/crashreport.h` - each states its env var's accepted values and
- The header comments in `src/core/startup-trace.h` and
`src/debug/syscall-hist.h` - each states its env var's accepted values and
what it costs when disabled.
- `src/debug/crashreport.h` for the crash-type enum and the report layout.
The report is unconditional.
38 changes: 27 additions & 11 deletions .claude/skills/elfuse-guest-abi/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,13 @@ Only `bad_exception` vectors may clobber X5, because they halt.
| #12 | System instruction trap | cache maintenance logging |
| #13 | Ptrace stop | taken after the shim restores the HVC #5 saved frame |

The register contract per HVC, including which return values each one accepts,
is the header comment in `src/core/shim.S`. That comment is the specification;
this table is an index into it.
The register contract per shim HVC, including which return values each one
accepts, is the header comment in `src/core/shim.S`. That comment is the
specification; this table is an index into it. HVC #6 is the exception: it
never reaches the shim, so the header does not list it. Its contract is the
`hvc6_handler` comment in `src/core/guest.h`, which names both routes into it;
they are dispatched from `case 6` in `src/syscall/proc.c` and from the private
pseudo-syscall in `src/syscall/syscall.c`.

Changing the protocol is never a one-file change. The shim, the host
dispatcher, the crash and debug paths that decode HVCs, and the documentation
Expand Down Expand Up @@ -208,11 +212,20 @@ Two rules survive any layout change:
## shim_data integrity

shim_data is `MEM_PERM_RW_EL1_ONLY` and holds a host-published cache the EL1
shim serves inline: identity slots (pid/ppid/uid/euid/gid/egid/tid), the
urandom-eligible fd bitmap, a urandom ring, and an attention bitmask
(`ATTN_BIT_SIGTIMER`, `ATTN_BIT_CRED`, `ATTN_BIT_TRACE`). HVC #5 is taken only
when attention is raised, the fd is not in the bitmap, or the ring needs a
refill.
shim serves inline: identity slots (pid/ppid/uid/euid/gid/egid, plus pgid
and sid), per-bucket futex waiter counts, the urandom-eligible fd bitmap, a
urandom ring, and an attention bitmask
(`ATTN_BIT_SIGTIMER`, `ATTN_BIT_CRED`, `ATTN_BIT_TRACE`, `ATTN_BIT_PTRACE`;
`src/core/shim-globals.h` is the list). This governs the inline fast paths
only: the identity calls, getpgid(0) and getsid(0), read on a urandom fd,
getrandom, a futex wait whose word moves during the EL1 spin (answered EAGAIN,
or EFAULT when the word cannot be read), and a futex wake whose bucket holds
no waiter (answered 0). gettid needs no slot: it reads CONTEXTIDR_EL1, which
the host sets to the tid per vCPU. Every other syscall forwards as HVC #5, and
so does a fast-path call the shim cannot settle: attention raised, the fd not
in the urandom bitmap, the ring short, a futex word that still matches, a
bucket with a waiter. The dispatch and bail labels in `src/core/shim.S` are
the full list.

Four things keep EL0 out of it, and a change that weakens any one of them is a
guest-readable host cache:
Expand All @@ -226,9 +239,12 @@ guest-readable host cache:
- `/proc/self/maps` reports the span as PROT_NONE.

Publishing into the cache is bracketed rather than ordered by luck:
`shim_globals_attn_or` (`__ATOMIC_SEQ_CST`) raises the attention bit before
the mutator's stores, so a weakly-ordered ARM64 reader cannot observe the
publish without the bit; the clear is `__ATOMIC_RELEASE`.
`shim_globals_attn_or` raises the attention bit before the mutator's stores
with `atomic_fetch_or_explicit(..., memory_order_seq_cst)`, so a
weakly-ordered ARM64 reader cannot observe the publish without the bit;
`shim_globals_attn_and` clears it with
`atomic_fetch_and_explicit(..., memory_order_release)`. Both are C11; the
atomics rule and its gate are `elfuse-conventions`.

## Stack construction

Expand Down
25 changes: 14 additions & 11 deletions .claude/skills/elfuse-security/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: elfuse-security
description: The guest as an attacker - where the trust boundary runs, the rules a handler on it obeys, what the gates already catch, and what is out of scope. Use when a change parses a guest-chosen length, translates a guest address, resolves a guest path, allocates on the guest's behalf, or blocks holding shared state, and when auditing a diff or writing a finding up.
description: The guest as an attacker - where the trust boundary runs, the rules a handler on it obeys, what the gates already catch, and what is out of scope. Use when a change parses a guest-chosen length, translates a guest address, resolves a guest path, walks a raw USB descriptor blob, allocates on the guest's behalf, or blocks holding shared state, and when auditing a diff or writing a finding up.
---

# Security at the guest boundary
Expand All @@ -25,7 +25,7 @@ which lanes prove it.

## Where the boundary runs

Five surfaces, ordered by what one bad value reaches:
The surfaces, ordered by what one bad value reaches:

- Syscall arguments. X0-X5 and X8 arrive from EL0 with no host filter in
front, so every wrapper reached from `src/syscall/dispatch.tbl` is on the
Expand All @@ -34,12 +34,17 @@ Five surfaces, ordered by what one bad value reaches:
translator, and the permission half is the security half.
- Formats the host parses for the guest: the ELF the loader reads, netlink
messages, FUSE frames, control messages, sigframes, sockaddrs, iovecs,
dirents. Each carries lengths, offsets, or counts the guest supplies, but
what the guest owns differs per format, so answer that per format rather
than assuming it. The ELF is read by offset, a sockaddr length arrives as a
separate syscall argument, and a sigframe is built by the host and then left
where the guest can rewrite it before `rt_sigreturn` reads it back.
dirents, and the usbdevfs URB structures (`usbdevfs_urb`,
`usbdevfs_ctrltransfer`, `usbdevfs_bulktransfer`). Each carries lengths,
offsets, or counts the guest supplies, but what the guest owns differs per
format, so answer that per format rather than assuming it. The ELF is read
by offset, a sockaddr length arrives as a separate syscall argument, and a
sigframe is built by the host and then left where the guest can rewrite it
before `rt_sigreturn` reads it back.
- Paths. Every name the guest supplies, absolute ones included.
- Raw USB descriptor blobs, walked by `src/runtime/usb-desc.c`. The one
input here the guest does not author; the header comment in
`src/runtime/usb-desc.h` says why it is untrusted anyway.
- Shared pages. The guest and the host see the same memory, so a structure
validated in guest memory and then passed on by address was not validated.

Expand Down Expand Up @@ -207,10 +212,8 @@ Under `make check`:
- `scripts/check-atomics.py` fails a C11 atomic operation written without the
`_explicit` form, and bans the compiler builtins, so the order is written at
the site. Its docstring names what it deliberately leaves out: plain-operator
access to an `_Atomic` object, which needs per-translation-unit declarations
and which the tree already carries a large set of. That half is a review
question, so it is the one memory-order case to spend budget on rather than
skip.
access to an `_Atomic` object. That half is a review question, so it is the
one memory-order case to spend budget on rather than skip.
- `scripts/check-svc-tails.py` holds every return tail to the X7 ptrace test,
bar the one exception its docstring names and allowlists.
- `scripts/check-syscall-coverage.py` is a best-effort audit of `dispatch.tbl`
Expand Down
25 changes: 20 additions & 5 deletions .claude/skills/elfuse-syscall/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: elfuse-syscall
description: Adding or changing a Linux syscall in elfuse. Covers dispatch.tbl, sc_ wrappers, the translation boundary, path and filename resolution, fd classes, lock order, and the coverage gate. Use when touching src/syscall/ or the syscall side of src/runtime/, adding a syscall number, or debugging a guest ENOSYS/EINVAL/EPERM. If the change also alters what the guest observes on return (registers, page permissions, the EL0 return path), read elfuse-guest-abi as well.
description: Adding or changing a Linux syscall in elfuse. Covers dispatch.tbl, sc_ wrappers, the translation boundary, path and filename resolution, fd classes, lock order, usbdevfs, and the coverage gate. Use when touching src/syscall/ or the syscall side of src/runtime/, working on a USB descriptor blob, adding a syscall number, or debugging a guest ENOSYS/EINVAL/EPERM. If the change also alters what the guest observes on return (registers, page permissions, the EL0 return path), read elfuse-guest-abi as well.
---

# Adding a syscall to elfuse
Expand Down Expand Up @@ -152,9 +152,9 @@ target. See the `elfuse-verify` skill.
## FDs and locks

Guest fds are not host fds. Allocate through the bitmap allocator in
`fdtable.c`; classify with the helpers in `fd.c` and `fd.h` (socket, pidfd,
eventfd, timerfd, signalfd). A class check that reads the raw fd number is wrong after
a `dup`.
`fdtable.c`. The `FD_*` type constants live in `linux-wire.h`; the class
predicates live in `fd.c`, `fd.h` and `internal.h`. A class check that reads
the raw fd number is wrong after a `dup`.

The lock order is the comment at the top of `internal.h`. Acquire in the order
it lists, and add a new lock to that comment as soon as it exists, whether or
Expand All @@ -169,7 +169,7 @@ the ordering:

## Subsystems with rules of their own

Three areas under `src/syscall/` are not ordinary domain files, and a change
Some areas under `src/syscall/` are not ordinary domain files, and a change
that treats them as such tends to compile and then deadlock or leak:

- FUSE (`fuse.c`) runs a whole filesystem transport inside the guest. Sessions
Expand All @@ -185,6 +185,21 @@ that treats them as such tends to compile and then deadlock or leak:
- Abstract Unix sockets, SCM_RIGHTS, and netlink each carry their own
serialization format over guest-supplied lengths, which is why several of
them have proof targets.
- usbdevfs (`usbdev.c`, with `src/runtime/usb-sysfs.c` and `usb-desc.c`):
`/dev/bus/usb/BBB/DDD` character devices over IOKit, asynchronous URBs, and
a synthetic `/sys/bus/usb` tree that goes through the same intercept layer
as procfs. Its locks do not nest the way the rest of the ordered list does,
and the `usbdev_table_lock` entry in `internal.h` is where that is written
down. The IOKit calls sit behind a COM seam, and `ELFUSE_USB_FIXTURE` stands
modeled devices in front of it so the fd contract runs with no hardware. A
device that answers a transfer needs both `ELFUSE_USB_FIXTURE=loopback` at
run time and a binary built with `USB_LOOPBACK_FIXTURE=1`: the env var picks
the device, the build flavor decides whether the fixture is linked in at all.
`mk/config.mk` owns the split and says why the two flavors write separate
binaries.
`scripts/gen-usbdev-ioctl-departed.py` generates
build/usbdev-ioctl-departed-vectors.h from
`tests/usbdev-ioctl-departed.tbl`: edit the table, not the header.

`docs/internals.md` has a section for each.

Expand Down
Loading
Loading