feat(data): schema type-constructors + sideloaded service schemas driving createLazy

[krisnye]

Description

Teaches Schema to describe data-adjacent types (reactive values, async values, streams, callables), so a service's whole surface can be described by one plain, serializable Schema. Service schemas are sideloaded — published beside the service and passed explicitly — and now drive AsyncDataService.createLazy.

This started as a bespoke ServiceDescriptor (states/actions/services buckets) and collapsed, over the course of the work, into "a service is just an object Schema" once Schema.ToType learned to express these types. The bespoke descriptor, its ToService transform, and the Service.descriptor instance slot were all removed.

1. Schema type-constructors (schema.ts, to-type.ts)

Four new, nestable type constructors, mapped by Schema.ToType:

Schema	ToType
{ type: "observe", value: S }	Observe<ToType<S>>
{ type: "promise", value: S }	Promise<ToType<S>>
{ type: "generator", value: S }	AsyncGenerator<ToType<S>>
{ type: "function", signature: { parameters: S[], returns?: S } }	(...args) => ToType<returns> (absent returns ⇒ void; absent signature ⇒ () => void; absent value ⇒ any)

They compose and nest (e.g. an AsyncGenerator<Data> parameter inside an object argument), which the plain-Data-only argument model could not express. Like the existing blob/typed-buffer types, they are confined by usage — they belong in service schemas, not ECS component / typed-buffer schemas (there is deliberately no DataSchema split: Schema already produces non-immutable / non-Data types such as TypedBuffer, and consumers already handle-or-throw per context).

The function constructor's members (parameters, returns, and the invocation-policy external) are grouped under a nested signature so they live only on function schemas rather than on every Schema node.

signature.external?: { agent?: boolean; link?: boolean } is optional invocation-policy metadata — who may invoke the function from an untrusted channel — with deliberately opposite default polarity: link (deeplink/URL, least trusted) is a default-deny whitelist (invocable only when link === true); agent (acting on the user's behalf) is a default-allow blacklist (invocable unless agent === false). It is metadata only: Schema.ToType ignores it (a function differing only in external derives the same signature), and it does not interact with the schema validators or createLazy. resolveExternalInvocation(schema) (reached as Schema.resolveExternalInvocation) is the single source of truth for resolving that polarity — call sites must not re-derive it.

2. Sideloaded service schemas (service.ts, is-valid-with-*-schema.ts)

• A "service descriptor" is now just a Schema; ToService is just Schema.ToType.
• Schemas are authored beside the service (MyService.schema via the namespace pattern) and passed explicitly. The base Service also carries an optional schema?: Schema slot so a factory may attach it to the instance for runtime introspection — excluded from IsValid like serviceName (both IsValidProperty variants now skip base-Service metadata keys when recursing into nested objects, so validating a nested Service & {…} doesn't recurse into the self-referential Schema).
• Two opt-in validators (each in its own file), taking the schema as a sideloaded type arg:
  • IsValidWithPartialSchema<T, S> — service members INCLUDE everything S describes (subset).
  • IsValidWithCompleteSchema<T, S> — service members are EXACTLY what S describes.

3. createLazy driven by a schema (create-lazy.ts)

createLazy now takes { load, schema, preload? }. Each member's runtime wrapper is derived from its schema (observe, or function classified by returns). The gate is schema-match only — EquivalentTypes<Schema.ToType<S>, members>, the actual correctness condition for building the wrappers — and is deliberately decoupled from IsValid: a service that legitimately returns non-Data (e.g. a fetch(): Promise<Response> port) can still be lazily chunk-loaded, while async-data-service conformance stays a separate definition-site concern. A LazyMemberSchema constraint rejects unsupported members (nested-object, malformed returns) at compile time rather than at runtime, so the type-level gate and the runtime dispatch cannot disagree.

Drive-by fixes

• Schema.ToType blob mapping. { type: "blob" } resolved to any (its test passed only vacuously); it now resolves to Blob, guarded by a regression test.
• Bump only publishable packages. pnpm run bump now runs scripts/bump.mjs, which bumps the root anchor + every publishable package (private !== true) and skips private sample/app packages, keyed off the same private field that governs pnpm -r publish. Removes the per-PR churn of ~9 unpublished package.jsons.
• Lazy generator correctness. The lazy AsyncGenerator wrapper could resurrect after termination (return()/throw() before the first next() didn't latch); it now latches a terminal state and supports [Symbol.asyncDispose].

Test evidence

• pnpm run lint — clean. pnpm run typecheck (all caches cleared, whole monorepo) — exit 0, 0 errors. check:workspace — OK.
• Type-level (colocated True/False/EquivalentTypes + @ts-expect-error): every constructor incl. the nested-AsyncGenerator-in-object-arg driver; the Blob-as-Blob-not-any regression guard; partial/complete schema matching; createLazy inference + the compile-time rejection of nested-object / malformed-returns members.
• Runtime (create-lazy.test.ts, vitest): 46 passed — all wrapper kinds, the generator terminal-state / async-dispose red→green tests, and the lazy instance exposing its schema before load; full src/service suite 120 passed.
• Load-bearing checks verified by deliberate breaks (produced TS2344); @ts-expect-error directives all consumed.
• Post-review: a fable subagent reviewed the diff; its two MAJOR findings (gate/runtime disagreement on unrecognized returns and on nested-object members) are fixed and covered by new compile-time tests, along with the minor/nit findings.

Related PRs

None.

Jira ticket

N/A — exploratory addition to @adobe/data.

Checklist

• Code compiles and type-checks (pnpm run typecheck — exit 0).
• Lint passes (pnpm run lint — clean); pnpm run check:workspace — OK.
• Tests added/updated — colocated compile-time tests + create-lazy.test.ts (44 passing, incl. red→green generator tests).
• Version bumped (publishable packages → v0.10.8; 0.10.7 was already published, so the continued work targets 0.10.8; private samples intentionally not bumped, see scripts/bump.mjs).
• Docs/comments — JSDoc on new types; README.md + create-lazy.md migrated to the schema API; removed a false IsDataService backwards-compat note.
• Backwards compatible where it can be — new schema constructors are additive; createLazy's input changed from properties to schema (pre-1.0), all callers migrated.

[kunalkindra]

suggestion: Building on the note I sent yesterday — this is the spot where dynamic schemas would land. Today schema (here, and parameters on ActionDescriptor) is static, but for agent tools the valid inputs often depend on live state: which models a user is entitled to, or options that change once a model is picked. Could the descriptor let a schema be either static or dynamic — e.g. also accepting an Observe<Schema> (Schema is Data, so it still round-trips) — so both the static shape and the runtime-narrowed one are first-class? Happy to be the first to exercise it on our side.

[krisnye]

concerning the dynamic schema. The services only expose static typed api endpoints. Those static endpoints can provide observability and invocation of dynamic things, naturally, but they can't expose statically typed constraints on each.

The service descriptor is merely a runtime description of the strongly typed service interface. So it also cannot provide dynamic schema.

That said, you can still support dynamic interactions, just like how you can today across the strongly typed service interfaces.

[krisnye]

addressed offline. We don't need yet, but may want a way to indicate expected max duration of a promise or async generator.