Repository navigation
A file's mtime is wall-clock time: nanoseconds since the Unix epoch, UTC - #587
Conversation
Stat::mtime is documented and stamped as nanoseconds since boot, DATA stores that number across reboots, the FAT adapter answers local Unix seconds instead, and std and libc read both as time since the epoch. A build tool comparing mtimes is misled by every reboot. Filed so the fix that follows closes it by name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The owner approved this ABI change for the self-hosting track: ninja and make decide rebuilds by mtime, and an mtime since boot made every reboot reorder the tree. Stat::mtime is now nanoseconds since the Unix epoch in UTC, off the wall clock at the write, and never a time since boot. The layout is unchanged (a u64); the meaning is the kernel's to keep: - kernel/src/clock.rs: utc_nanos is the RTC's whole-second anchor carried on by the counter, and utc_secs (SYS_CLOCK_EPOCH) is now its whole seconds, so a stamp taken after the epoch syscall answered s is at least s seconds - the two can never disagree about a second. mtime_now is what a write stamps: utc_nanos, or on a machine whose RTC never answered, a clock that starts at the epoch at boot, so one boot's stamps still order. local_of_utc and utc_of_local convert for a format that stores local time. - object/ops.rs, vfs.rs and the two self-tests stamp with mtime_now where they stamped nanos_since_boot. - fat32_adapter.rs stamps the mtime the VFS hands it (the write's instant, not the flush's) as local seconds, and answers file_mtime as UTC nanoseconds; it used to answer local seconds, a different unit from every other mount. FAT keeps two seconds of local time, which the ABI doc now says, and toyos-fat32 gains a host test that a write time drops the odd second. - bcachefs and tmpfs store the u64 as given, so DATA keeps nanoseconds across a reboot; ROOT's image stamps 0, as src/image.rs always did. - libc's struct stat gains POSIX's st_atim/st_mtim/st_ctim, with st_mtime as the macro POSIX defines; fstat put nanoseconds into st_mtime, which is seconds. - std needs no change: the fork's Metadata::modified already reads the word as nanoseconds since UNIX_EPOCH. sshd's SFTP attributes, which read modified(), now report real times. The contract is the one PR #536's fsd already meets: it reports nanoseconds since the epoch, off SYS_CLOCK_EPOCH's whole seconds. Tests: file_mtime judges /tmp's stamp between two SYS_CLOCK_EPOCH readings and requires a second write's stamp to be later (whole seconds cannot say that), on the shared boot; its write/read modes are driven by file_mtime_survives_a_reboot, which stages the RTC with -rtc base=, checks DATA's stamp against that instant, reads the same stamp off the image with the host's bcachefs reader, and boots again with the RTC a day on to read it back unchanged. Closes issues/filesystem/a-files-mtime-is-nanoseconds-since-boot.md. Files what the audit found and this does not fix: userlands-wall-clock-has-whole-seconds-and-an-mtime-has-nanoseconds, the-last-handle-to-close-stamps-the-file-with-its-own-mtime, a-fat-files-mtime-reads-finer-through-its-writer-than-after-a-reopen, std-answers-an-mtime-of-1970-for-what-it-did-not-stat. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review, round 1, at 32948daReady for review. CI BLOCKER
NOTE
REMOVE
SEND BACK |
…every stamp site observed - clock: `mtime_now` is `utc_nanos().unwrap_or(0)`. A machine whose RTC never answered has no date, so a file written there is undated (0), never 1970 plus uptime. `NANOS_PER_SEC` is private and used for every second in the module; FAT's two conversions take and give an `mtime`. - toyos-abi: `Stat::mtime` states each mount's resolution and that 0 is undated; the path citations and "never a time since boot" are gone. - std (fork ac63a08077f, `wt-toyos-mtime`): `Metadata::modified` answers `Unsupported` for 0, the kind `SYS_CLOCK_EPOCH` refuses with there. - libc: `st_nlink` is `u64`, as `nlink_t` is. The Rust `Stat` was 112 bytes against C's 120, so C read `st_blksize` as `st_size` and a stamp's nanoseconds as `st_mtime`. `202_stat_mtime` asserts the size, the length, the seconds and the nanoseconds through `fstat` and `stat`. - `file_mtime` judges a create with truncation, a create of a missing file and a truncation against the wall clock, and its `undated` mode, run by the new `file_mtime_undated` on the `rtc-dead` machine, finds a written file undated without ever asking the time. - Issues: the last-handle defect now reaches FAT; the std issue is renamed to what stays true; filed: an undated FAT file reads back as 1980, and the shared-object cache's identity is blind on an undated machine. The so-cache issue's `nanos_since_boot` claim is deleted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Round 2, head 57cc41e: NOT READY FOR REVIEW |
Recount tests/testcases/LICENSE's tinycc totals from the tree as merged (318 files, 62 not upstream, 61 written here) instead of either side's stale count. main's wall-clock boot-drift refactor (MAX_BOOT_DRIFT_SECS replaced by after_the_base, measured against the boot's own elapsed time) merged without a textual conflict but left file_mtime_survives_a_reboot referencing the deleted constant; it now uses after_the_base like main's other callers, with mtime_boot returning the elapsed Duration alongside the printed mtime. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main's #583 made the RTC UTC: `clock` keeps one anchor, BOOT_SECS, and the firmware zone, UTC_OFFSET_SECS and local_secs are gone. The mtime now comes from that same anchor. - kernel/src/clock.rs: main's `init_wall(century_reg)`, BOOT_SECS and its "reads {civil} UTC" line are kept. This branch's utc_nanos is BOOT_SECS * 10^9 plus nanos_since_boot, and utc_secs stays its whole seconds, which is the value main's utc_secs computed. mtime_now stays utc_nanos, or 0 (undated) when the RTC never answered. local_secs_of and mtime_of_local are deleted because no zone is left to convert through. NANOS_PER_SEC becomes pub for the FAT adapter. - kernel/src/fat32_adapter.rs: this branch's `stamp(mtime)` and `now() = stamp(mtime_now())` are kept, now as whole UTC seconds of the mtime. main's refusal-reason moves onto `stamp`: FAT specifies local time, and this stamps UTC because the owner ruled the hardware clock is UTC. `file_mtime` answers modified_unix * 10^9. An undated 0 clamps to FatTime::EPOCH, which is what main's now() gave with no RTC. - tests/toyos.rs: main's wall_clock_utc row, CARRIES entry and dispatch arm replace wall_clock_zone, which main deleted. This branch's file_mtime_survives_a_reboot and file_mtime_undated are kept beside them. - rust: main's pin 9c3eea441d8 (#597) is merged into the fork's wt-toyos-mtime as 90697f1401a, with no conflicts. - issues/filesystem/a-fat-files-mtime-reads-finer-...: "two seconds of local time" is no longer true, so it now reads "two-second units". issues/kernel/sys-clock-realtime-is-now-a-format-of-sys-clock-epoch.md is unchanged. Both handlers still read clock::utc_secs, and its value did not change. This branch adds no caller of clock_realtime. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review, round 3, at 389f572CI Net size is +582/−35. Production is +67/−31: kernel +42/−21, toyos-abi +5/−1, libc +19/−8, and the rust pin 1/1. Tests are +380/−2 and issues +135/−3. Round-1 BLOCKERs
BLOCKER
NOTE
REMOVE
SEND BACK 🤖 Generated with Claude Code |
Review round 3 on #587: - fat32_adapter.rs: update_metadata stamps now(), the flush's own instant, as main did, not the flushing handle's mtime. The last handle to close can be a reader opened before the last write, so its mtime would store the file as older than its contents. `create` still stamps the mtime the VFS hands it. Stat::mtime's doc says FAT's two seconds are the flush's, and the last-handle issue's paragraph saying the defect reached FAT goes. - file_mtime's shared run writes /log/file-mtime, reopens it and requires its stamp within (before-1)*1e9 ..= after*1e9 and on a whole second, so dropping the adapter's seconds-to-nanoseconds conversion reds it. - the-so-caches-identity-is-blind-on-an-undated-machine.md merges into the same-size-rewrite section of the-so-caches-refusals-are-narrower-than-its- reach.md, whose false fat32_adapter.rs:849 clause goes. - std-calls-undated-what-it-did-not-stat.md also records that a directory's stat and a symlink's lstat answer mtime 0, so modified() on them is Err(Unsupported). - a-fat-files-mtime-reads-finer...md loses "at or below it" and "the one a later reader sees is the older", false now FAT stamps at the flush. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U6SVYFkdvV2t38KzNrESxs
|
Mutation patches for round 5, head m132-ops.patch — `kernel/src/object/ops.rs:132`: a create of a missing file stamps `nanos_since_boot()`. Test:
|
Review, round 4, at f7cadedCI Net size is +586/−38. Production is +67/−28: kernel +42/−18, toyos-abi +5/−1, libc +19/−8, and the rust pin 1/1. Tests are +397/−2, counting the Round-3 BLOCKERs
Round-3 NOTEs: the so-cache issue files are merged, and the dir/symlink refusal is recorded in Negative controlRunning it at
At BLOCKERNone. NOTE
REMOVE
LAND AFTER NAMED CHANGES 🤖 Generated with Claude Code |
…dition checkably Delete the doc line on `now()`. Rewrite the exit condition of the FAT mtime issue so it names the check, and record there that the flush's own instant (`now()` vs the handle's `_mtime`) is untested. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U6SVYFkdvV2t38KzNrESxs
Brings #587 (a file's mtime is UTC nanoseconds since the Unix epoch), #603, #614 and #615. Conflicts, every hunk of both sides: - kernel/src/fat32_adapter.rs (modify/delete): the deletion stands. #587's hunks there are `stamp(mtime)` as whole UTC seconds, `now()` as `stamp(mtime_now())`, `create` stamping the VFS's mtime, the flush stamping its own instant, and `file_mtime` answering seconds times 10^9. FAT stamping is fsd's on this branch: its `lstat`, `list` and `node_meta` already answer seconds times 10^9, its level stamps the flush's own instant, and the next commit gives it the nanosecond clock to divide. - kernel/src/vfs.rs: this branch deleted `open_backing_identified`'s flush of a dirty file, so #587's `mtime_now` there goes with it. #587's `file_mtime` doc hunk is taken. - kernel/src/object/ops.rs: the write and `ftruncate` stamps keep this branch's `file_cache::touch` and `set_size`, and stamp `mtime_now`; the two open stamps merged clean. #587's comment on dirty state in the cache is not taken: this branch's cache keeps no dirt. - kernel/src/leak_selftest.rs: this branch deleted `fat_reopen_census` with the FAT adapter it probed, so #587's `mtime_now` there goes too. - kernel/src/clock.rs (merged clean): `NANOS_PER_SEC` is private, since the FAT adapter it was made public for is gone. - issues/isolation/the-so-caches-refusals-are-narrower-than-its-reach.md: this branch deleted the FAT same-size-rewrite section, because the kernel's cache reaches no FAT mount. #587's undated paragraph is still true of `/tmp`, the one writable mount a kernel `dlopen` reaches, so the section stays for it, with #587's exit condition. - rust: c4c65e3e87a on ToyOSOrg/rust wt-toyos-fsd merges main's pin 90697f1401a with no conflict. Issues arriving with main, against this tree: - two-shared-members-assume-a-bcachefs-home-the-t14-does-not-have.md (#615) is deleted. Its premise is the kernel's tmpfs `/home` and `NvmeBacking`, and both are gone: on the T14 `/home` is fsd's bcachefs in memory. The T14 run at 27b5641 reads `fs_large_file` exit=0 and `home_backing_revoked` exit=0 beside fsd's "this machine has no DATA partition; ... are in memory" line. - the-last-handle-to-close-stamps-the-file-with-its-own-mtime.md is deleted. Its mechanism is the kernel write-back's last-close flush, which this branch deletes. `/tmp`'s mtime is the file's (`file_cache::touch`), and fsd's DATA keeps one mtime per node. What stays true of a handle's mtime is filed as a-kernel-files-fstat-answers-its-handles-mtime.md. - a-fat-files-mtime-reads-finer-through-its-writer-than-after-a-reopen.md is deleted. Its exit condition's second arm holds here: the kernel mounts no FAT volume a process writes. fsd's FAT reads the entry's stamp through a handle, which differs from a reopen's the other way. That is filed as a-fat-files-mtime-through-its-handle-is-its-last-level.md. - std-calls-undated-what-it-did-not-stat.md: the sentence that std reads an mtime only off SYS_FSTAT is deleted, since a served file's comes off fsd's stat. - tests/common/wallclock.rs: `mtime_boot`'s tmpfs guard looked for the kernel's "are a tmpfs", which nothing prints here. It reads fsd's `storage::IN_MEMORY` line instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U6SVYFkdvV2t38KzNrESxs
#587's contract: a file's mtime is nanoseconds since the Unix epoch, UTC, DATA keeps the nanosecond, and 0 is undated. fsd stamped DATA with `clock_epoch()` times 10^9, whole seconds, so two writes to `/home` inside one second carried one mtime. It also read the clock a second time for FAT (`utc_secs`), with the same undated rule in another unit. - `fsd::volume::now_nanos` is the one reader. It derives the kernel's anchor exactly: `SYS_CLOCK_EPOCH` is `BOOT_SECS` plus the whole seconds of the counter at the call, and the process's clock page is the same counter on the kernel's formula, so a call bracketed by two readings inside one second names `BOOT_SECS`. A bracket that straddles a second is asked again, and three that straddle panic by name. The anchor is taken once: the kernel sets it once, before the first process. A refused clock is undated, 0. - FAT's volume takes the same clock and divides it, as main's `stamp(mtime_now())` did. `utc_secs` is deleted. - `file_mtime` judges `/home` too: two writes, the second strictly later, and not both whole seconds. `file_mtime undated` writes `/home` beside `/tmp` and requires both undated. - The host test `the_anchor_is_the_kernels` holds the derivation to the kernel's arithmetic, a straddled bracket included. - userlands-wall-clock-...md: the sentence that a file server can stamp only whole seconds is deleted; fsd stamps nanoseconds now. - home_overwrite_zero.rs: the comment that `fs::metadata` is the kernel's `file_cache::size` is deleted; `/home` is fsd's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U6SVYFkdvV2t38KzNrESxs
The owner approved this ABI change for the self-hosting track. ToyOS will build LLVM on itself, and ninja and make decide what to rebuild by mtime. So a file's modification time is now wall-clock time, in nanoseconds, and a reboot does not change it.
The contract
toyos_abi::syscall::Stat::mtimeis nanoseconds since the Unix epoch, UTC. It is taken off the wall clock at the write, to the resolution its mount keeps:/tmp(tmpfs)/boot,/log)src/image.rsstamps 0, for a reproducible imageThe source is #583's UTC anchor. Since #583 the RTC keeps UTC and
clockholds a single anchor,BOOT_SECS.utc_nanosisBOOT_SECS·10⁹ + nanos_since_boot().SYS_CLOCK_EPOCHis its whole seconds, and a write stamps it, so a stamp and the epoch syscall cannot disagree about a second.0 is undated. A machine whose RTC never answered has no date, so a file written there is stamped 0, never 1970 plus uptime. std's
Metadata::modifiedanswers 0 withErrorKind::Unsupported. That is the kindSYS_CLOCK_EPOCH's refusal maps to on that machine, whereSystemTime::nowpanics. libc reports 0 as 0.Accuracy is the RTC's second, and resolution is the counter's. The anchor truncates the RTC's second, as
SYS_CLOCK_EPOCHalready did.The layout of
Statis unchanged (oneu64), so no syscall changes. The sysroot key moves, because libc'sstruct statand the std fork change.What changed, per decision
kernel/src/clock.rsutc_nanosis the anchor carried on by the counter.utc_secs(SYS_CLOCK_EPOCH) isutc_nanos / 10⁹. That is the value main'sutc_secscomputed, since⌊(B·10⁹ + n)/10⁹⌋ = B + ⌊n/10⁹⌋.mtime_nowis what a write stamps:utc_nanos, or 0 with no RTC.NANOS_PER_SECispub, for the FAT adapter.object/ops.rs,vfs.rs,revoke_selftest.rs,leak_selftest.rsstamp withmtime_nowwherever they stampednanos_since_boot. The sites are: create with truncation, create of a missing file, write,ftruncate, and the flushopen_backing_identifiedmakes.fat32_adapter.rsstamp(mtime)is the whole UTC seconds of the mtime. FAT specifies local time, and this stamps UTC because the owner ruled the hardware clock is UTC.createstamps themtimethe VFS hands it. Before, it stampednow()and ignored the argument.update_metadatastamps the flush's own instant, as on main, and ignores the flushing handle'smtime: the last handle to close can be a reader that opened before the last write, and itsmtimewould store the file as older than its contents.file_mtimeanswersmodified_unix·10⁹, the same unit as every other mount.now()isstamp(mtime_now()), for the flush, reconcile and mkdir. An undated 0 clamps toFatTime::EPOCH, which is what main'snow()gave with no RTC.Metadata::modifiedanswersUnsupportedfor 0. Otherwise it already read the word as nanoseconds sinceUNIX_EPOCH, so it is correct now and was wrong before. The change is fork commitac63a08077fonToyOSOrg/rustbranchwt-toyos-mtime. It is pinned as90697f1401a, which merges main's pin9c3eea441d8(toolchain: LLVM, with clang and LLD, is built once per key and every compiler links it #597) with no conflicts.struct statgets POSIX'sst_atim/st_mtim/st_ctim(struct timespec), andst_mtimebecomes the macro POSIX defines.fstatused to put nanoseconds intost_mtime, which is seconds.st_nlinkisu64, asnlink_t(unsigned long) is. The RustStatwas 112 bytes against C's 120. A C caller readst_blksize(0) asst_sizeand the stamp'stv_nsecasst_mtime, andfstatleft C's last 8 bytes unwritten. That mismatch predates this branch; nothing in the tree calledstatfrom C.Tests
202_stat_mtime(C, Fast): writes 17 bytes to/tmp, thenfstats andstats the file. It printssizeof(struct stat)(120),st_size(17), whetherst_mtimelies between twotime(NULL)readings around the write, and whetherst_mtim.tv_nsecis below a second.file_mtime(Rust, Fast, shared boot) judges/tmp. Each stamp must lie between twoSYS_CLOCK_EPOCHreadings taken around what made it:before·10⁹ ≤ mtime < (after+1)·10⁹. It covers four stamp sites:File::createwith no write (create with truncation);OpenOptions::new().write(true).create(true)on a path first shown to be missing (create of a missing file);set_len(1)andsync_all(ftruncate), which must stamp strictly later than its create.Then
/log(FAT): it writes, closes and reopens/log/file-mtime, and requires(before−1)·10⁹ ≤ stamp ≤ after·10⁹and a whole second, withafterread after the reopen'sstat.file_mtime_undated(Nightly): boots withrtc-deadand checks that the kernel refused the clock by name. It then runsfile_mtime undated, which writes/tmp/file-mtime-undatedwithout asking the time and requiresmodified()to fail withUnsupported.file_mtime_survives_a_reboot(Nightly), two boots of one DATA image:2033-03-07T09:14:25with-rtc base=and writes/home/file-mtime.bin. The printed stamp must lie withinafter_the_baseof that instant.bcachefsreader.toyos-fat32a_write_time_keeps_the_even_second(host) tests the conversion behind the FAT row: a write time drops the odd second thatFatTimekeeps only for creation times.main's
wall_clock_utcchecks the FAT stamp of the log against the staged RTC. The FAT stamp is nowstamp(mtime_now()), so that test also covers this branch's FAT path.Gates
At
f7caded3, the merge oforigin/mainat4de5ecdb. Host only: I ran no QEMU.cargo run -- --ci hostcargo run -- --build-onlycargo test --test toyos-build -- --listFast 202_stat_mtime,Fast file_mtime,Nightly file_mtime_survives_a_reboot,Nightly file_mtime_undated)High-risk: ABI and filesystems
cargo run -- --build-only, reverted it, and showed the tree clean afterwards. Each must turn its test red:nlink-u32.patch(libc back to the pre-branchst_nlink: u32) →202_stat_mtime:st_sizereads 0 andst_mtimereads a nanosecond count.site-114.patch(create with truncation back tonanos_since_boot()) →file_mtime.undated.patch(unwrap_or(0)→unwrap_or_else(nanos_since_boot)) →file_mtime_undated:modified()answers 1970 plus uptime.m132-ops,m821-ops(create of a missing file, andftruncate, back tonanos_since_boot()) →file_mtime.m926-fat-seconds(the FAT adapter'sfile_mtimeanswers seconds unconverted) →file_mtime's/logarm. These three are in the PR comment of round 5; each built atf7caded3(exit 0).git diff HEAD origin/mainoverkernel/src,toyos-abi/src,userland/libc,tests/testcases/LICENSEand202_stat_mtime.{c,expect}. That reverts the whole implementation onto7e151819and keeps every other test.202_stat_mtimeis dropped because main's header has nost_mtim, so the test cannot compile there. The std pin is kept, since it only changes what 0 means. At389f572cthe script applied it (12 files), built it (EXIT=0), reverted it and showed the tree clean.file_mtimegoes red on its first assertion, since a stamp since boot is about 10⁹·uptime, far belowbefore·10⁹.file_mtime_survives_a_rebootgoes red on the oracle: a drift of about −1.99·10⁹ s.-rtc base=, the same oracle thewall_clock_*tests use. For the C layout, clang's ownsizeof/offsetofover the header.What I am unsure of
BackingId(size plus mtime) no longer sees a same-size rewrite on any mount there. Before,nanos_since_bootkept two writes apart. Recorded in the same-size-rewrite section ofissues/isolation/the-so-caches-refusals-are-narrower-than-its-reach.md.FatTime::from_unix_secsclamps 0 to 1980-01-01, which reads back as a date. Filed:issues/filesystem/an-undated-file-on-fat-reads-back-as-1980.md.modified()refuses a directory and a symlink.statof a directory andlstatof a symlink answer mtime 0, which std now reports asUnsupportedwhere it answeredOkbefore. Recorded inissues/filesystem/std-calls-undated-what-it-did-not-stat.md.SystemTime::nowand libc'sclock_gettimedo, so a just-written file can read as up to a second in the future against the program's own "now". Filed:issues/filesystem/userlands-wall-clock-has-whole-seconds-and-an-mtime-has-nanoseconds.md.Closes
issues/filesystem/a-files-mtime-is-nanoseconds-since-boot.md. Also filed, found by the audit and not fixed here:issues/filesystem/the-last-handle-to-close-stamps-the-file-with-its-own-mtime.mdissues/filesystem/a-fat-files-mtime-reads-finer-through-its-writer-than-after-a-reopen.mdissues/filesystem/std-calls-undated-what-it-did-not-stat.md:DirEntry::metadataanswers 0 for a file the kernel has a stamp for, a directory'sstatand a symlink'slstatanswer 0, andset_timesreturnsOkhaving set nothing.🤖 Generated with Claude Code
https://claude.ai/code/session_01U6SVYFkdvV2t38KzNrESxs