Skip to content

Transloadit Viewer: Storage and template images with CLI onboarding - #500

Open
kvz wants to merge 95 commits into
mainfrom
img-onboard
Open

kvz wants to merge 95 commits into
mainfrom
img-onboard

Conversation

@kvz

@kvz kvz commented Sep 12, 2026

Copy link
Copy Markdown
Member

Why: a small, trustworthy boundary for Storage consumers

Applications should not translate raw upload JSON into ad hoc image-only records or use a mutable path as an immutable reference. This follow-up provides server-fetched completed-Assembly receipts for every media kind with provenance, typed native move/delete helpers, and short-lived exact-version original URLs for inline delivery or downloads.

The app still owns authentication, asset ownership and idempotent registration; Gallery/Collections/application databases are not introduced. Downloads stream from the CDN, not through a full in-memory app Blob. Review fixes also cover one-snapshot logout, shared catalog/declaration ownership, safe scaffold rollback and malformed legacy receipt isolation.

Verification at fb2cc40: all applicable GitHub checks pass, including Node 20/22/24, E2E, fast/full verification, build/release dry run and the packed Next fixture. Local packed Next/Chromium/WebKit coverage passed 126 checks. The real owned-devdock/R2 private consumer proof passes 27 checks. Content dogfoods four archives built from this exact commit.

Not published or merged. API2 contracts must deploy first; production CDN/cache, app authorization and backup/restore remain explicit release gates. The existing Refs #270 section below is retained; this does not claim the scaffold or Console-login Playwright follow-ups are complete. Earlier sections below are historical checkpoints.

September 17: why Storage references now pin actual versions

A saved path is not durable identity. This follow-through with API2 #9057 makes completed store results and native catalog reads share Workspace, asset ID, version ID, final path, size, MIME, checksums and authoritative version geometry. Path imports remain mutable; asset-only imports follow the logical asset; asset+version selects exact retained bytes.

  • Viewer catalogs require both IDs and Workspace. The default Storage Built-ins take the asset ID as input and the real version ID as v; signed redirects seal the same identity in capability payload v2. Authorize Storage by stable identity, not a stale saved path. Custom HTTP/S3 Templates keep their existing path input contract.
  • Typed listStoredAssets / getStoredAsset and CLI storage ls / storage receipts sync consume bounded signed native catalog pages, replacing S3 List+HEAD. Deadlines, cancellation, atomic replacement and locks remain. Recovery reads public policy without publishing; placeholders survive only for the same asset/version.
  • Catalog-level apiOrigin binds even an empty published catalog; receipt provenance is retained too. Store/sync/publish/unpublish and credentialed or explicit-endpoint scaffolding refuse mixed API environments. A custom CDN does not prove API identity. Unbound legacy catalogs require recovery into a new file and review. Offline optional scaffolding preserves metadata without credentials, remote requests or rebinding.
  • All three canonical Robot schema copies are byte-identical. Step validation rejects conflicting/missing selectors, orphaned versions and recursive ID imports while preserving string booleans, interpolation and Zod 3/4 help descriptions. Native filenames remain distinct from stricter Viewer paths.
  • Council and the independent security follow-up drove red-first fixes for provenance, selector/schema parity, errors and field bounds. API2 also prevents bearer-authenticated image responses from entering shared caches.

Current exact-head verification: b1737f4a6208d23711241eb442490b3b6452c164 is green in CI run 35226151498: all ten applicable jobs, including both Verify jobs, Node 20/22/24, E2E, release dry run and the packed Next browser fixture. Three reporting-only jobs are skipped. Local root yarn check passes with 671 Node tests (one existing skip), 355 Viewer tests, 63 utils tests and the remaining package/script checks. The immediately preceding implementation passed all 126 local packed Chromium/WebKit checks (58/58/10); the final metadata-only change is also covered by the exact-head hosted fixture. Content #5973 vendors four normally packed archives from this exact commit, with SHA-256 provenance.

Breaking changes are limited to this unpublished Storage/Viewer experiment: recover old path-only receipts; rebuild applications for capability-v2 URLs; deploy the API2 Built-ins/catalog API before releasing packages and updating consumers. Pins require retained versions and current access, not unlimited retention. No npm publication, main merge or production operation occurred. Current cross-repo plan: Storage reference contract.

September 17 DX follow-up

Why: the independent reader test found inconsistent responsive defaults and misleading CLI field selection. This follow-up closes those gaps without adding a source registry or a Next image-byte proxy.

  • Every high-level Image source is constrained/responsive by default; explicit layout="none" remains the opt-out.
  • templates/assemblies list --fields now projects both text and JSON correctly. General CLI Quick Start leads with auth login; manual/CI keys remain documented.
  • Council corrections: share exact public-preview limits; retain compatible signed delivery for previously private capabilities that exceed public bounds after publication, always requiring application authorization before signing. Document this authorizer contract explicitly.
  • Local verification: root check (347 Viewer, 63 utils, 641 Node tests with one existing skip), 126 packed Chromium/WebKit browser cases, real local width-only public HEAD 200 image/webp, and a live eight-row CLI projection probe.
  • Content #5973 vendors all four packs from c07ad0d. API2 #9057 repairs saved-signup mail failure and keeps pending users unauthenticated.

Exact-head CI is green at c07ad0d: run 35196137023, all ten applicable checks passed. No npm publication, production deployment, or merge into main is part of this follow-up.

Evidence/checklist: docs/prompts/2026-09-17-viewer-dx-fixes.md.

Why

One native, responsive image component should work with Transloadit Storage and customers'
existing HTTP origins or S3 buckets. Storage is the integrated source, not a prerequisite for
Smart CDN delivery. Rename the unpublished package to @transloadit/viewer now, while preserving
the image-only implementation and direct CDN byte delivery.

import { Image } from '@transloadit/viewer/next'

<Image storage src="website/hero.jpg" alt="A canal house" width={960} preload />
<Image workspace="my-shop" template="product-images"
  src={{ path: 'chairs/oak.jpg', width: 1200, height: 800 }} alt="An oak chair" width={480} />

What

  • Separate workspace and template props; choose exactly one of storage or template.
    Storage inherits its catalog workspace; explicit or project-default conflicts fail closed.
    Custom Templates need no Storage upload/catalog and never inherit its public-prefix policy.
  • Reuse the native renderer and existing signing/capability engine. Private authorizers receive
    { workspace, template, path, request } plus asset_id/version_id for Storage. Redirect capabilities bind that complete source;
    HTTP-style escaped traversal/query/fragment inputs are rejected before signing or authorization.
    Custom Templates must pin their HTTP origin or S3 bucket/prefix and require signatures.
  • createImages remains the explicit server factory for metadata, separate keys and delivery
    policy. Image bytes are never proxied or optimized by Next. No new media renderers or alias registry.
  • CLI snippets, declarations, package exports, root docs and packed consumer use the renamed API.
    Compatible HTTP/S3 recipes and pre-release migration notes are in the reference. A dev diagnostic
    catches Templates that ignore requested formats without exposing signed URL queries.
  • Preserve the full onboarding work consolidated from Preserve image HTML attributes and enforce loading contracts #497Document durable private image delivery and bounded revocation #499: browser login; atomic typed
    catalogs; storage store --public / --hashed; stored receipt recovery; public-prefix policy;
    responsive layouts, art direction, preload, private authorization and native format selection.
  • Keep the no-init default path, optional scaffolding, ThumbHash/alpha behavior, bounded decoding,
    cache-key defaults and receipt/overwrite safeguards from the earlier rounds.

Refs #270

Implements browser/code auth login, credentials in ~/.transloadit, --no-browser and
--stdin fallbacks. Still open: npm create transloadit-scaffold and Playwright E2E of the actual
Console login/approval page. Refs, not Fixes.

Verification — current local implementation

  • yarn verify:full passes (339 image tests, 636 Node tests, plus the other workspaces).
    Final root yarn check passes too, including the missing-catalog regression: 340 image tests.
  • Packed Next fixture: 126 Chromium/WebKit checks pass: 58 with cacheComponents enabled,
    58 omitted, and 10 development checks. Both builds pass. No retries or skips.
  • Real package source-selection proof: native decode for Storage, HTTP and S3; full-identity
    authorization; 307 redirect with no image bytes; tampered Template/workspace and logout denied.
  • Separate live API2 devdock probes return decoded 100×100 WebP with alpha for both restricted
    HTTP and S3 Templates. Temporary local Templates were deleted; no production configuration changed.
    Existing legacy Templates ignored requested formats, validating the new diagnostic/recipe need.
  • Actual Content page dogfood at 1440px and 390px: native AVIF decode, correct boxes/preload,
    no console errors. Browser-only routing sends the proof asset to local API2; this is not a
    production CDN proof. Content integration is in transloadit/content#5973.
  • Two council passes plus independent security review. Valid findings fixed red-first; final
    security verdict PASS. The last council P3 (missing-catalog guidance) is fixed with a regression test.

Earlier hosted verification (before the DX follow-up above): f1b2d349e2c5be03eea3114e0a9890433d6e8362,
run 35143668326, all ten applicable
checks passed, including both Verify jobs, Node 20/22/24, E2E, release dry run and the packed Next
browser fixture. Three existing reporting-only checks are skipped. No merge was performed.
The living source-model plan records detailed local evidence and boundaries.

Content integration is green too: transloadit/content#5973 at 72d6106459,
run 35156427968, all 15 applicable
jobs passed. Four exact f1b2d349e2 archives install as plain Git blobs, including the corrected
Lighthouse sparse checkout. Full local check passes 12,135 tests; hosted units pass 12,136.
Fixture setup, 3,399 browser cases and a 4,565-page crawl pass; one existing locale-exit browser
case needed its automatic retry and remains a documented, unreproduced flake. The exact generated
website is Ready and its image-markup/preload/catalog assertions pass.

Lighthouse successfully measures six routes, but remains in observation mode: performance
70–88, SEO 69, 11 target violations. These scores are not relabeled as passing budgets or proof of
5×100. Actual native image dogfood above still uses the explicitly documented local API2 routing.

Release boundaries and known limits

  • No merge or publication. Viewer remains private at 0.0.0; reserve/publish the package before
    CLI-generated customer imports ship. Coordinate transloadit/api2#9057 / transloadit/content#5973
    availability first.
  • Public Storage is static/unsigned; custom Templates use signed delivery. Without an image-request
    authorizer, the application must authorize private sources before rendering signed direct URLs.
  • Production native-catalog availability, registry installation, production hosted-image/standalone proof and real
    Bunny cache/cost measurements remain rollout gates. The former Content Vercel OOM
    (transloadit/content#5999) was fixed upstream by transloadit/content#6000; the updated Content
    branch includes it and its generated Vercel deployment is verified Ready. That does not establish
    production delivery of the local-only lab asset.
  • The pre-existing generic getSmartCdnImageCandidates public-height issue is deferred; Image
    uses its own limit-enforcing renderer. Edge runtime is unsupported.
  • New Storage Built-ins require the actual retained version ID in v; old explicitly versioned path Built-ins remain cache-tag based. Pins require a live accessible asset and retained version, not unlimited retention. Hashed replay trusts a matching committed receipt from the verified API origin, not a remote-object probe. Fresh native recovery cannot reconstruct absent ThumbHash/alpha.
  • This package currently renders images only; Video, Audio, Document and automatic
    Viewer selection are separate follow-ups, not exports promised by this PR.

Quickstart
· Reference
· Source-model plan

@kvz kvz changed the title Complete the Storage seed-to-image recipe and SDK catalog Complete Storage images in Next.js: native props, layout, lifetime and onboarding Sep 12, 2026
@kvz
kvz changed the base branch from img-lifetime to main September 12, 2026 08:45
kvz added 15 commits September 12, 2026 12:05
Pin storage-preview 0.0.2 and bind per-format backgrounds through direct URLs and authorized redirects. Keep JPEG opaque and make its color explicit.

Sync the canonical recursive import schema from API2 #9057 and document the confirmed cache-key and two-key contracts. Cover alpha and fallback pixels in the packed browser fixture; record the separate live canary DNS limitation.
Address the five final round-3 council findings: retain receipt-file errors, print safe post-write recovery details, clarify invalid fixed sources and subpixel crops, and remove the stale import-extension prerequisite.

Reproduce all findings red-first, then verify affected packages, legacy parity, the full workspace and all 48 packed browser cases before pushing.
@kvz kvz changed the title Storage images for Next.js: public CDN delivery, private authorization and CLI onboarding Transloadit Viewer: Storage and template images with CLI onboarding Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant