feat(cache): use HTTP 404 for remote cache fetch misses

CHANGELOG.md

@@ -2,7 +2,7 @@
 
 - **Fixed** An invalid glob in `--filter` no longer shows its error message twice ([#763](https://github.com/voidzero-dev/vite-task/pull/763)).
 - **Changed** The detailed summary from `vp run --verbose` and `vp run --last-details` now shows each underlying cause of an error on its own line ([#761](https://github.com/voidzero-dev/vite-task/pull/761)).
-- **Added** Remote caching. Configure an endpoint with the workspace's `cache: { remote: { url } }` or `VP_REMOTE_CACHE_URL`, and choose access with `--remote-cache=off|read|read-write` or `VP_REMOTE_CACHE`. The default is `read` with an endpoint and `off` without one. After a local cache miss, `vp run` looks the task up in the remote cache and, on a hit, restores its outputs and caches it locally. The task output and the run summary show which hits came from the remote cache. A failed read is just a cache miss, with the failure as its reason. In `read-write` mode, `vp run` also uploads the results of successful, cacheable tasks after caching them locally. A failed upload doesn't fail the task; the run summary shows a warning instead. Ctrl-C, or a failing task, stops remote cache requests right away, and a task still being looked up doesn't start. Tasks can opt out with `cache: { remote: false }`. Requests use the proxy environment variables or, on macOS and Windows, the system proxy settings ([#727](https://github.com/voidzero-dev/vite-task/pull/727), [#755](https://github.com/voidzero-dev/vite-task/pull/755), [#756](https://github.com/voidzero-dev/vite-task/pull/756), [#757](https://github.com/voidzero-dev/vite-task/pull/757), [#764](https://github.com/voidzero-dev/vite-task/pull/764), [#771](https://github.com/voidzero-dev/vite-task/pull/771)).
+- **Added** Remote caching. Configure an endpoint with the workspace's `cache: { remote: { url } }` or `VP_REMOTE_CACHE_URL`, and choose access with `--remote-cache=off|read|read-write` or `VP_REMOTE_CACHE`. The default is `read` with an endpoint and `off` without one. After a local cache miss, `vp run` looks the task up in the remote cache and, on a hit, restores its outputs and caches it locally. The task output and the run summary show which hits came from the remote cache. A failed read is just a cache miss, with the failure as its reason. In `read-write` mode, `vp run` also uploads the results of successful, cacheable tasks after caching them locally. A failed upload doesn't fail the task; the run summary shows a warning instead. Ctrl-C, or a failing task, stops remote cache requests right away, and a task still being looked up doesn't start. Tasks can opt out with `cache: { remote: false }`. Requests use the proxy environment variables or, on macOS and Windows, the system proxy settings ([#727](https://github.com/voidzero-dev/vite-task/pull/727), [#755](https://github.com/voidzero-dev/vite-task/pull/755), [#756](https://github.com/voidzero-dev/vite-task/pull/756), [#757](https://github.com/voidzero-dev/vite-task/pull/757), [#764](https://github.com/voidzero-dev/vite-task/pull/764), [#771](https://github.com/voidzero-dev/vite-task/pull/771), [#772](https://github.com/voidzero-dev/vite-task/pull/772)).
 - **Fixed** On Windows, environment variable names used by `vp run` now match regardless of ASCII letter case. Assignments in task commands override earlier assignments and inherited variables spelled differently, and `FORCE_COLOR`, `VP_RUN_CONCURRENCY_LIMIT`, and variables requested through `@voidzero-dev/vite-task-client` are found under any spelling ([#747](https://github.com/voidzero-dev/vite-task/pull/747)).
 - **Changed** A task's cache settings now go inside `cache`, e.g. `cache: { env: ["NODE_ENV"], input: ["src/**"] }`; `cache: true` is the same as `cache: {}`. `env`, `untrackedEnv`, `input`, and `output` are no longer supported at the top level of a task ([#749](https://github.com/voidzero-dev/vite-task/pull/749)).
 - **Fixed** Cached tasks on macOS no longer intermittently fail with exit 2 and `oils I/O error (main): No such process` when a fast command finishes before the shell gets scheduled. The bundled shell that runs task commands is updated to Oils 0.38.0, which fixes this race ([#702](https://github.com/voidzero-dev/vite-task/issues/702), [#703](https://github.com/voidzero-dev/vite-task/pull/703)).

crates/vt/src/session/cache/remote.rs

@@ -97,32 +97,32 @@ pub(super) struct Restore {
 
 /// Turn the result of a fetch into an entry to restore or a miss. `validate`
 /// checks an exact entry against the current execution. A fallback's miss
-/// reason compares its key with `cache_key`. A failed fetch, an entry that
-/// doesn't decode or whose blob doesn't match its value, or a validation error
-/// is a read failure.
+/// reason compares its key with `cache_key`, and no match is a miss without
+/// an entry. A failed fetch, an entry that doesn't decode or whose blob
+/// doesn't match its value, or a validation error is a read failure.
 #[expect(
     clippy::result_large_err,
     reason = "`CacheMiss` is intentionally large, and a lookup returns it once"
 )]
 pub(super) fn resolve(
-    fetched: Result<Fetched, ReadError>,
+    fetched: Result<Option<Fetched>, ReadError>,
     cache_key: &CacheEntryKey,
     validate: impl FnOnce(&CacheEntryValue) -> anyhow::Result<Option<FingerprintMismatch>>,
 ) -> Result<Restore, CacheMiss> {
     let (value, blob_id) = match fetched.map_err(ReadError::into_miss)? {
-        Fetched::Exact { value, blob_id } => {
+        Some(Fetched::Exact { value, blob_id }) => {
             let value: CacheEntryValue = deserialize_cache(&value)
                 .map_err(|err| ReadError::CorruptValue(err).into_miss())?;
             if value.output_archive.is_some() != blob_id.is_some() {
                 return Err(ReadError::MismatchedBlob.into_miss());
             }
             (value, blob_id)
         }
-        Fetched::Fallback { key } => {
+        Some(Fetched::Fallback { key }) => {
             let key = decode_key(&key).map_err(ReadError::into_miss)?;
             return Err(CacheMiss::FingerprintMismatch(key.into_mismatch(cache_key)));
         }
-        Fetched::NotFound => return Err(CacheMiss::NotFound),
+        None => return Err(CacheMiss::NotFound),
     };
     match validate(&value) {
         Ok(None) => Ok(Restore { value, blob_id }),
@@ -150,15 +150,15 @@ impl RemoteClients {
     }
 
     /// Fetch the entry stored under `cache_key`, falling back to the entry
-    /// last stored for `execution_cache_key`. Stops when `cancel_token` is
-    /// cancelled.
+    /// last stored for `execution_cache_key`. Returns `None` if neither key
+    /// matched. Stops when `cancel_token` is cancelled.
     pub(super) async fn fetch(
         &self,
         endpoint: &Arc<str>,
         cache_key: &CacheEntryKey,
         execution_cache_key: &ExecutionCacheKey,
         cancel_token: &CancellationToken,
-    ) -> Result<Fetched, ReadError> {
+    ) -> Result<Option<Fetched>, ReadError> {
         let client = self.client(endpoint).map_err(ReadError::Fetch)?;
         let key = encode_key(cache_key)?;
         let secondary_key = encode_key(execution_cache_key)?;
@@ -430,7 +430,8 @@ mod tests {
     fn exact_entry_that_validates_is_restored() {
         let key = cache_key(ResolvedGlobConfig::default_auto());
         let restore =
-            resolve(Ok(exact(&cache_value())), &key, validate_against(BTreeMap::new())).unwrap();
+            resolve(Ok(Some(exact(&cache_value()))), &key, validate_against(BTreeMap::new()))
+                .unwrap();
         assert_eq!(restore.blob_id.as_deref(), Some("1"));
         assert_eq!(restore.value.std_outputs[0].content, b"built\n");
         assert_eq!(restore.value.duration, Duration::from_millis(5));
@@ -440,7 +441,7 @@ mod tests {
     fn exact_entry_that_fails_validation_is_a_mismatch() {
         let current_inputs = BTreeMap::from([(RelativePathBuf::new("src/a.txt").unwrap(), 1)]);
         let miss = resolve(
-            Ok(exact(&cache_value())),
+            Ok(Some(exact(&cache_value()))),
             &cache_key(ResolvedGlobConfig::default_auto()),
             validate_against(current_inputs),
         )
@@ -466,7 +467,7 @@ mod tests {
             .tracked_env_queries
             .insert(TrackedEnvQuery::Glob(Str::from("PROBE_[")), BTreeMap::new());
         let miss = resolve(
-            Ok(exact(&value)),
+            Ok(Some(exact(&value))),
             &cache_key(ResolvedGlobConfig::default_auto()),
             validate_against(BTreeMap::new()),
         )
@@ -480,7 +481,7 @@ mod tests {
         let mut input_config = ResolvedGlobConfig::default_auto();
         input_config.positive_globs.insert(Str::from("src/**"));
         let miss = resolve(
-            Ok(Fetched::Fallback { key: stored_key }),
+            Ok(Some(Fetched::Fallback { key: stored_key })),
             &cache_key(input_config),
             not_validated,
         )
@@ -492,9 +493,9 @@ mod tests {
     }
 
     #[test]
-    fn not_found_is_a_miss_without_an_entry() {
+    fn no_match_is_a_miss_without_an_entry() {
         let key = cache_key(ResolvedGlobConfig::default_auto());
-        let miss = resolve(Ok(Fetched::NotFound), &key, not_validated).unwrap_err();
+        let miss = resolve(Ok(None), &key, not_validated).unwrap_err();
         assert!(matches!(miss, CacheMiss::NotFound), "{miss:?}");
     }
 
@@ -512,7 +513,7 @@ mod tests {
         let mut trailing = serialize_cache(&cache_value()).unwrap();
         trailing.push(0);
         for value in [b"not a cache value".to_vec(), trailing] {
-            let fetched = Ok(Fetched::Exact { value, blob_id: None });
+            let fetched = Ok(Some(Fetched::Exact { value, blob_id: None }));
             let miss = resolve(fetched, &key, not_validated).unwrap_err();
             assert_eq!(read_failure(miss), "remote cache value is corrupt");
         }
@@ -523,7 +524,8 @@ mod tests {
         let key = cache_key(ResolvedGlobConfig::default_auto());
         let without_archive = CacheEntryValue { output_archive: None, ..cache_value() };
         for (value, blob_id) in [(cache_value(), None), (without_archive, Some(Str::from("1")))] {
-            let fetched = Ok(Fetched::Exact { value: serialize_cache(&value).unwrap(), blob_id });
+            let value = serialize_cache(&value).unwrap();
+            let fetched = Ok(Some(Fetched::Exact { value, blob_id }));
             let miss = resolve(fetched, &key, not_validated).unwrap_err();
             assert_eq!(read_failure(miss), "remote cache entry's blob doesn't match its value");
         }
@@ -542,7 +544,7 @@ mod tests {
         let mut garbage = encode_header().unwrap();
         garbage.extend(b"not a cache key");
         for stored_key in [b"not a cache key".to_vec(), other_header, garbage] {
-            let fetched = Ok(Fetched::Fallback { key: stored_key });
+            let fetched = Ok(Some(Fetched::Fallback { key: stored_key }));
             let miss = resolve(fetched, &key, not_validated).unwrap_err();
             assert_eq!(read_failure(miss), "remote cache key is corrupt");
         }

crates/vt_bin/tests/e2e_snapshots/fixtures/remote_cache/snapshots/corrupt_archive.md

@@ -5,7 +5,7 @@
 ```
 $ vtt write-file dist/output.txt built
 
-[remote-cache] POST /fetch 200 not_found
+[remote-cache] POST /fetch 404
 [remote-cache] POST /store 200
 ```
 

crates/vt_bin/tests/e2e_snapshots/fixtures/remote_cache/snapshots/fallback.md

@@ -5,7 +5,7 @@
 ```
 $ vtt write-file dist/output.txt built
 
-[remote-cache] POST /fetch 200 not_found
+[remote-cache] POST /fetch 404
 [remote-cache] POST /store 200
 ```
 

crates/vt_bin/tests/e2e_snapshots/fixtures/remote_cache/snapshots/local_and_remote_hits.md

@@ -5,7 +5,7 @@
 ```
 $ vtt write-file dist/output.txt built
 
-[remote-cache] POST /fetch 200 not_found
+[remote-cache] POST /fetch 404
 [remote-cache] POST /store 200
 ```
 

crates/vt_bin/tests/e2e_snapshots/fixtures/remote_cache/snapshots/read.md

@@ -5,7 +5,7 @@
 ```
 $ vtt write-file dist/output.txt built
 
-[remote-cache] POST /fetch 200 not_found
+[remote-cache] POST /fetch 404
 [remote-cache] POST /store 200
 ```
 

crates/vt_bin/tests/e2e_snapshots/fixtures/remote_cache/snapshots/read_write.md

@@ -7,7 +7,7 @@ The fetch finds no entry. The new execution is uploaded with one store request.
 ```
 $ vtt write-file dist/output.txt built
 
-[remote-cache] POST /fetch 200 not_found
+[remote-cache] POST /fetch 404
 [remote-cache] POST /store 200
 ```
 

crates/vt_bin/tests/e2e_snapshots/fixtures/remote_cache/snapshots/restore.md

@@ -5,7 +5,7 @@
 ```
 $ vtt write-file dist/output.txt built
 
-[remote-cache] POST /fetch 200 not_found
+[remote-cache] POST /fetch 404
 [remote-cache] POST /store 200
 ```
 

crates/vt_bin/tests/e2e_snapshots/fixtures/remote_cache_backend/snapshots/entries_and_associations.md

@@ -3,8 +3,8 @@
 ## `remote-cache-server cbor-http POST /fetch --cbor '{"key": '\''A'\'', "secondary_key": '\''S'\''}'`
 
 ```
-{"status": 200, "content_type": "application/cbor", "body": {"kind": "not_found"}}
-[remote-cache] POST /fetch 200 not_found
+{"status": 404, "content_type": "text/plain; charset=utf-8", "body": "Not found"}
+[remote-cache] POST /fetch 404
 ```
 
 ## `remote-cache-server cbor-http POST /store --form-cbor 'metadata={"key": '\''A'\'', "secondary_key": '\''S'\'', "value": '\''VA'\''}'`

crates/vt_bin/tests/e2e_snapshots/fixtures/remote_cache_backend/snapshots/invalid_requests.md

@@ -133,6 +133,6 @@ Failed stores did not replace the value or blob.
 Failed stores did not publish a secondary association.
 
 ```
-{"status": 200, "content_type": "application/cbor", "body": {"kind": "not_found"}}
-[remote-cache] POST /fetch 200 not_found
+{"status": 404, "content_type": "text/plain; charset=utf-8", "body": "Not found"}
+[remote-cache] POST /fetch 404
 ```

crates/vt_remote_cache/README.md

@@ -4,14 +4,14 @@ Client for the [remote cache server API](https://github.com/voidzero-dev/vite-ta
 
 `Client::new` takes the configured endpoint, which can include a namespace path, such as `https://cache.example.com/projects/my-project`. Each operation appends its route to that path, so a store goes to `https://cache.example.com/projects/my-project/store`. Endpoints that aren't HTTP or HTTPS URLs are rejected.
 
-`Client::fetch` sends a key and a secondary key as a CBOR map of byte strings. It decodes the response into `Fetched`: an exact match with its value and blob ID, a fallback match with the key it's stored under, or no match. An exact match must include `blob_id`, which is null when there's no blob. The fallback's value and blob ID aren't decoded.
+`Client::fetch` sends a key and a secondary key as a CBOR map of byte strings. A 404 response means neither key matched, so `fetch` returns `None` without decoding the body. The body of a 200 response is a CBOR map tagged by `kind`, and `Fetched` decodes only two kinds: `exact`, an exact match with its value and blob ID, and `fallback`, a fallback match with the key it's stored under. Any other kind is a malformed response. An exact match must include `blob_id`, which is null when there's no blob. The fallback's value and blob ID aren't decoded.
 
 `Client::download` gets a blob by its ID. After a 200 response, it returns a `Download`, from which the caller reads the blob's chunks as they arrive.
 
 `Client::store` sends a multipart request: a CBOR `metadata` part with the key, secondary key, and value as byte strings, and an optional `blob` part streamed from a file. The response body isn't decoded.
 
-Only HTTP 200 counts as success for every operation. Redirects aren't followed, so a redirect fails like any other status.
+Only HTTP 200 counts as success for every operation, except that a fetch also accepts 404 as no match. Redirects aren't followed, so a redirect fails like any other status.
 
 reqwest configures TLS. It uses the process's default rustls crypto provider, which the client installs as ring unless one is already installed, and verifies certificates with the operating system's verifier. Requests go through the proxy set in `HTTPS_PROXY`, `HTTP_PROXY`, or `ALL_PROXY`, except for hosts in `NO_PROXY`. Without those variables, the system proxy settings are used on macOS and Windows. Connections time out after 10 seconds. Reads time out after 60 seconds, and until the response headers arrive, that limit also covers sending the request.
 
-`Error` names the kind of failure: an invalid endpoint, a client that couldn't be created, a blob file that couldn't be read, a network error (including timeouts and responses that end early), a status other than 200, or a malformed fetch response. Its messages contain no OS-specific details, so they can be shown to users as is. The details are in the source: the underlying error, the parse error for an endpoint that isn't a URL, or the message in an error response's body. Network errors leave out the request URL, since the endpoint may contain credentials, such as a token in its query.
+`Error` names the kind of failure: an invalid endpoint, a client that couldn't be created, a blob file that couldn't be read, a network error (including timeouts and responses that end early), a status other than 200 (or 404, for a fetch), or a malformed fetch response. Its messages contain no OS-specific details, so they can be shown to users as is. The details are in the source: the underlying error, the parse error for an endpoint that isn't a URL, or the message in an error response's body. Network errors leave out the request URL, since the endpoint may contain credentials, such as a token in its query.