feat(core): make enum its own field kind

.changeset/enum-field-kind.md

@@ -0,0 +1,7 @@
+---
+'@haverstack/core': minor
+'@haverstack/conformance-fixtures': minor
+'@haverstack/adapter-conformance': minor
+---
+
+`enum` is a field kind: `{ kind: 'enum', values: [...] }` replaces `{ kind: 'string', enum: [...] }`. `defineType()` refuses the `enum` key on any field. `isCompatible()` now checks enums: a required enum accepts only an enum candidate whose values it lists all of, and a required `string` or `text` accepts an enum. An enum becoming a plain `string` is additive; a `string` becoming an enum needs a version bump. A typed read types an enum field as its listed values or any other string (`StoredContentOf`, `UnlistedValue`) and no longer throws on an unlisted value, since an enum can widen in place; typed writes still take only the listed values (`ContentOf`, `PatchOf`). `migration()` types both sides as `StoredContentOf`, so a step carries an unlisted value through. A typed `subscribe()` delivers a record that `migrateAll()` moves off the handle's version without its `record`, instead of throwing in the dispatcher. Conformance fixtures pin a 422 for a value outside the list and a 409 for removing values. The adapter conformance suite checks that an enum field sorts by value.

AGENTS.md

@@ -68,6 +68,7 @@ This is about _our_ history. Genuinely foreign input — a third-party server's
 ## Architecture rules
 
 - **`Stack` is the invariant layer; adapters are storage engines.** Validation, `_config` protection, ID rules, migration policy, and permission checks live in `packages/core` so every adapter inherits them. Don't reimplement an invariant inside an adapter. (The one documented exception: the `_config` query exclusion must live in each adapter's own query predicate.)
+- **Validate where data enters; trust it inside.** Public API calls and wire requests are checked once. A stored Type or Record from an adapter is trusted to match its type: don't guard for a hand-edited store (a missing `values` list, an unknown `kind`). That includes `adapter-api`: its server is held to the wire contract the way a storage engine is held to the adapter contract. See [CONTRIBUTING.md § Architecture conventions](./CONTRIBUTING.md#architecture-conventions).
 - **Package prefixes declare the contract**: `adapter-*` = full `StackAdapter`, `record-adapter-*` = `StackRecordAdapter`, `blob-adapter-*` = `StackBlobAdapter`.
 - **Optional capabilities are optional methods**, checked for truthiness at the call site with a documented fallback — never a boolean in `capabilities`.
 - **Shared SQLite logic goes in `@haverstack/sqlite-shared`**, not duplicated across adapters. It is `private` and bundled into its consumers at build time, so it is never a `dependencies` entry and adding an export to it does not widen any package's public API.

CONTRIBUTING.md

@@ -233,6 +233,8 @@ Note that this is about _our_ history, not about inputs we genuinely receive. Ha
 
 **`Stack` is the invariant layer; adapters are storage engines.** Validation, `_config` protection, ID rules, migration policy, and permission logic live in `packages/core` so every adapter inherits them. An adapter that reimplements an invariant is a bug waiting to diverge. The deliberate exception is documented in the spec: the `_config` query exclusion must live in each adapter's own query predicate, because post-filtering in `Stack` would under-fill a page.
 
+**Validate where data enters; trust it inside.** Input is checked once, at the boundary it crosses: a public API call such as `defineType()` or `create()`, or a wire request a server decodes. Past that point the types are true, and code reads a stored Type or Record as its type says. Data an adapter returns is inside: a store that holds something core never wrote was edited by hand, and guarding for that in every reader costs more than it buys. Each guard for malformed stored data also forces a policy choice (does a missing list allow nothing? does an unknown kind sort as text?), and those choices drift apart between call sites. That holds for `adapter-api` too: the server behind it is held to the wire contract, which `@haverstack/conformance-fixtures` pins, the way a storage engine is held to the adapter contract.
+
 **Package naming declares the adapter contract**, per [`docs/spec/adapters.md`](./docs/spec/adapters.md):
 
 | Prefix             | Implements           |

packages/adapter-conformance/src/record.ts

@@ -1028,6 +1028,29 @@ export function runRecordAdapterConformance(options: RecordAdapterConformanceOpt
             1, 2, 3,
           ]);
         });
+
+        test('orders an enum field by value, not by its position in values', async () => {
+          await adapter.saveType(
+            conformanceType({
+              schema: {
+                title: { kind: 'string' },
+                status: { kind: 'enum', values: ['want', 'reading', 'finished'] },
+              },
+            }),
+          );
+          for (const status of ['want', 'finished', 'reading']) {
+            await adapter.createRecord(makeRecord({ content: { title: status, status } }));
+          }
+
+          const result = await adapter.queryRecords({
+            sort: { contentField: 'status', direction: 'asc' },
+          });
+          expect(result.records.map((r) => (r.content as { status: string }).status)).toEqual([
+            'finished',
+            'reading',
+            'want',
+          ]);
+        });
       });
     }
 

packages/conformance-fixtures/src/index.ts

@@ -2674,6 +2674,30 @@ export const errorResponseFixtures: ConformanceFixture<unknown, WireError>[] = [
       },
     },
   },
+  {
+    name: 'error-validation-enum-value-not-listed',
+    description:
+      'PATCH content setting an enum field to a value its `values` list does not name returns ' +
+      '422 with code "validation", the detail naming the allowed values. An enum value is a ' +
+      "string, so the type check passes and only the list refuses it. Assumes the record's " +
+      'type declares { status: { kind: "enum", values: ["want", "reading", "finished"] } }.',
+    method: 'PATCH',
+    path: '/records/1hk153x00001',
+    requestBody: { contentPatch: { status: 'abandoned' } },
+    responseStatus: 422,
+    responseBody: {
+      error: {
+        code: 'validation',
+        message: 'Content validation failed',
+        details: [
+          {
+            path: 'status',
+            message: 'Expected one of "want", "reading", "finished", got "abandoned"',
+          },
+        ],
+      },
+    },
+  },
   {
     name: 'error-validation-failed-restore',
     description:
@@ -3532,6 +3556,40 @@ export const errorResponseFixtures: ConformanceFixture<unknown, WireError>[] = [
       },
     },
   },
+  {
+    name: 'error-schema-drift-enum-values-removed',
+    description:
+      'POST /types redefining an enum field with fewer values narrows what the field accepts, ' +
+      'so it is not additive and returns 409 with code "schema_drift", naming the removed ' +
+      'values. Adding values, or an enum becoming a plain string, accepts strictly more and is ' +
+      'additive. Assumes "com.example/book@1" is already stored with { status: { kind: "enum", ' +
+      'values: ["want", "reading", "finished"] } }.',
+    method: 'POST',
+    path: '/types',
+    requestBody: {
+      id: 'com.example/book@1',
+      baseId: 'com.example/book',
+      version: 1,
+      name: 'Book',
+      schema: { status: { kind: 'enum', values: ['want', 'reading'] } },
+      schemaHash: '5d1e0c9b8a7f6e5d4c3b2a1908f7e6d5c4b3a291807f6e5d4c3b2a1908f7e6d5',
+      createdAt: '2024-01-01T00:00:00.000Z',
+    },
+    responseStatus: 409,
+    responseBody: {
+      error: {
+        code: 'schema_drift',
+        message:
+          'Schema drift detected for type "com.example/book@1": the stored schema and the new ' +
+          'definition differ beyond additive evolution (new optional fields only). Bump the ' +
+          'version instead of redefining "com.example/book@1" in place.',
+        schemaDrift: {
+          typeId: 'com.example/book@1',
+          violations: [{ path: 'status', message: 'enum values removed: "finished"' }],
+        },
+      },
+    },
+  },
   {
     name: 'error-bad-request-id-invalid-charset',
     description:

packages/core/src/index.ts

@@ -44,6 +44,8 @@ export type {
   TypeHandle,
   Migration,
   ContentOf,
+  StoredContentOf,
+  UnlistedValue,
   PatchOf,
   TypedRecord,
   TypedQuery,
@@ -101,6 +103,7 @@ export type {
   TypeSchema,
   FieldDef,
   ScalarFieldDef,
+  EnumFieldDef,
   ArrayFieldDef,
   ObjectFieldDef,
   ScalarFieldKind,

packages/core/src/schema.ts

@@ -11,6 +11,7 @@
 
 import type { TypeSchema, FieldDef, ScalarFieldKind } from './types.js';
 import type { ReadonlyTypeSchema } from './type-handle.js';
+import { formatEnumValues } from './validate.js';
 
 // -------------------------------------------------------
 // Canonical schema serialization
@@ -58,11 +59,11 @@ const canonicalizeFieldDef = (def: FieldDef): unknown => {
     };
   }
 
-  // Scalar. `enum` is sorted so reordering the values is not a change.
+  // Scalar. An enum's `values` are sorted so reordering them is not a change.
   return {
-    ...(def.kind === 'string' && def.enum && { enum: [...def.enum].sort() }),
     kind: def.kind,
     ...(def.required !== undefined && { required: def.required }),
+    ...(def.kind === 'enum' && { values: [...def.values].sort() }),
   };
 };
 
@@ -97,21 +98,23 @@ export const hashSchema = async (schema: TypeSchema): Promise<string> => {
 
 /**
  * Kinds acceptable in a candidate field, per required kind. `string` and
- * `text` are mutually acceptable for reading; everything else requires an
- * exact match. See docs/spec/data-model.md § Type compatibility.
+ * `text` are mutually acceptable for reading, and either accepts an enum,
+ * whose values are strings; everything else requires an exact match. A
+ * required enum takes the subset rule in isFieldCompatible instead. See
+ * docs/spec/data-model.md § Type compatibility.
  */
-const READ_COMPATIBLE: Record<ScalarFieldKind, ScalarFieldKind[]> = {
-  string: ['string', 'text'],
-  text: ['text', 'string'],
+const READ_COMPATIBLE: Record<Exclude<ScalarFieldKind, 'enum'>, ScalarFieldKind[]> = {
+  string: ['string', 'text', 'enum'],
+  text: ['text', 'string', 'enum'],
   number: ['number'],
   boolean: ['boolean'],
   date: ['date'],
   'record-ref': ['record-ref'],
   'file-ref': ['file-ref'],
 };
 
-// Candidate schemas can come from another app's Type definition (the
-// untrusted side of duck-typed consumption), so recursion into array
+// Candidate schemas can come from another app's Type definition, whose
+// shape validation bounds its depth, so recursion into array
 // items / object properties is depth-bounded — matches MAX_VALIDATION_DEPTH
 // in validate.ts. Past the limit we can't verify compatibility, so we
 // fail closed (treat as incompatible) rather than risk a stack overflow.
@@ -148,6 +151,12 @@ const isFieldCompatible = (candidate: FieldDef, required: FieldDef, depth: numbe
     );
   }
   if (candidate.kind === 'array' || candidate.kind === 'object') return false;
+  // A consumer expecting an enum can handle only the values it lists.
+  if (required.kind === 'enum') {
+    if (candidate.kind !== 'enum') return false;
+    const allowed = new Set(required.values);
+    return candidate.values.every((v) => allowed.has(v));
+  }
   return READ_COMPATIBLE[required.kind].includes(candidate.kind);
 };
 
@@ -181,7 +190,8 @@ export const isCompatible = (
 // -------------------------------------------------------
 //
 // Deliberately distinct from isCompatible() above: read compatibility and
-// evolution legality are different relations that disagree on text/string.
+// evolution legality are different relations that disagree on text/string
+// and on enum.
 // See docs/spec/data-model.md § Type compatibility.
 
 export type SchemaDriftViolation = {
@@ -207,7 +217,10 @@ const diffField = (
     violations.push({ path, message: 'exceeds max nesting depth; cannot verify additive change' });
     return;
   }
-  if (stored.kind !== candidate.kind) {
+  // An enum becoming a string accepts strictly more, so it is the one kind
+  // change that is additive.
+  const widensEnum = stored.kind === 'enum' && candidate.kind === 'string';
+  if (stored.kind !== candidate.kind && !widensEnum) {
     violations.push({
       path,
       message: `kind changed from "${stored.kind}" to "${candidate.kind}"`,
@@ -220,20 +233,12 @@ const diffField = (
       message: `required changed from ${!!stored.required} to ${!!candidate.required}`,
     });
   }
-  // An enum narrows what a string field accepts when it is added or loses
-  // values, and widens it when it is removed or gains them.
-  if (stored.kind === 'string' && candidate.kind === 'string') {
-    if (!stored.enum && candidate.enum) {
-      violations.push({ path, message: 'enum added to an existing string field' });
-    } else if (stored.enum && candidate.enum) {
-      const kept = new Set(candidate.enum);
-      const removed = stored.enum.filter((v) => !kept.has(v));
-      if (removed.length > 0) {
-        violations.push({
-          path,
-          message: `enum values removed: ${removed.map((v) => JSON.stringify(v)).join(', ')}`,
-        });
-      }
+  // Losing a value narrows what an enum accepts; gaining one widens it.
+  if (stored.kind === 'enum' && candidate.kind === 'enum') {
+    const kept = new Set(candidate.values);
+    const removed = stored.values.filter((v) => !kept.has(v));
+    if (removed.length > 0) {
+      violations.push({ path, message: `enum values removed: ${formatEnumValues(removed)}` });
     }
   }
   // Opening a declared container, or closing an open one, changes which

packages/core/src/sort.ts

@@ -54,7 +54,8 @@ const ZONELESS_DATE_TIME_RE = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?$/;
  *
  * `date` normalizes to epoch milliseconds so a date field orders the way
  * `createdAt` does instead of by ISO string collation, and `boolean` to
- * 0/1 so false precedes true.
+ * 0/1 so false precedes true. An enum orders as text: its declaration
+ * order is not part of the schema.
  */
 export function contentSortEntry(kind: ScalarFieldKind, value: unknown): SortEntry | null {
   switch (kind) {
@@ -69,7 +70,11 @@ export function contentSortEntry(kind: ScalarFieldKind, value: unknown): SortEnt
       const ms = Date.parse(ZONELESS_DATE_TIME_RE.test(value) ? `${value}Z` : value);
       return Number.isNaN(ms) ? null : { kind: 'num', num: ms };
     }
-    default:
+    case 'string':
+    case 'text':
+    case 'record-ref':
+    case 'file-ref':
+    case 'enum':
       return typeof value === 'string'
         ? { kind: 'text', text: value, key: contentSortKey(value) }
         : null;

packages/core/src/stack.ts

@@ -441,8 +441,8 @@ export interface StackClient {
   ): Promise<StackRecord & { content: T }>;
   /**
    * Typed read: the record at `presentAt: 'latest'`, checked to be exactly
-   * the handle's Type with every enum field holding a value the handle
-   * lists, else it throws. Live records only — a tombstone reads as `null`,
+   * the handle's Type, else it throws. An enum field may hold a value the
+   * handle does not list. Live records only — a tombstone reads as `null`,
    * and there is no `includeDeleted`.
    * See docs/spec/data-model.md § Type handles.
    */