Repository navigation
feat: add support for D1 Sessions API to mitigate stale data returned from replicas #813
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
3ef4674
aed1b71
ee898fe
c79bb85
3e9a82f
53b07fc
d64fbee
fd6ce1b
a461d57
23866c6
c302ed7
ae21985
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -43,7 +43,15 @@ await db.batch([ | |
|
|
||
| Void enables [D1 read replication](https://developers.cloudflare.com/d1/best-practices/read-replication/) when it provisions a database. To enable it on an existing database, run `void deploy --platform cloudflare --provision`. Queries through `db` from `void/db` use a D1 session for each Void request, so reads can run near the request and later queries in that request see at least the same data. Framework handlers without Void's request scope start each database operation on the primary. `createDb` starts a replica-capable session for the binding you pass it. | ||
|
|
||
| Replicas can lag between requests. Authentication checks use the primary so a lagging replica can't accept a revoked session. For other reads that need the latest primary data, create a client inside the request: | ||
| ::: warning | ||
| Replicas can lag between requests. [Carry a bookmark](#cross-request-bookmarks) to preserve previously observed database state, or use a [primary-first session](#reading-from-the-primary) when you need the latest primary data. | ||
| ::: | ||
|
|
||
| For raw D1 calls or another query library, use `withSession()` to read from replicas. A bare D1 binding reads from the primary. | ||
|
|
||
| ### Avoid stale reads with a primary-first session {#reading-from-the-primary} | ||
|
|
||
| Authentication checks use the primary so a lagging replica can't accept a revoked session. For other reads that need the latest primary data, create a client inside the request: | ||
|
|
||
| ```ts | ||
| import { users } from '@schema'; | ||
|
|
@@ -59,7 +67,221 @@ export const GET = defineHandler(async () => { | |
|
|
||
| With `session: 'first-primary'`, the first query reads from the primary and later queries see at least the same data. The default, `'first-unconstrained'`, allows the first query to use a replica. | ||
|
|
||
| For raw D1 calls or another query library, use `withSession()` to read from replicas. A bare D1 binding reads from the primary. | ||
| ### See your writes in subsequent requests {#cross-request-bookmarks} | ||
|
|
||
| To make a read after a save see that save in a later request, carry a [D1 bookmark](https://developers.cloudflare.com/d1/best-practices/read-replication/#start-a-session-from-previous-context-bookmark) between requests. A bookmark allows D1 to use a replica that has caught up to at least the previously observed database state. | ||
|
|
||
| ::: warning | ||
| Bookmarks do not affect cached responses. Disable ISR for pages that require fresh reads (`export const revalidate = 0` in the page's `.server.ts` file), and do not serve these requests from shared response caches. | ||
| ::: | ||
|
|
||
| Use middleware to set the default `db` session before your handlers or Pages loaders run, and return its bookmark after their queries. For a browser app, a cookie carries it through redirects and navigation: | ||
|
|
||
| Local server-side requests through `void/client` share that session. If middleware runs again, repeating an already satisfied session requirement preserves the session and its database progress. For an independent primary-first read or a newer bookmark after using `db`, use `createDb` instead. | ||
|
|
||
| ::: warning | ||
| Concurrent responses can overwrite this cookie with an older bookmark. The [header-based recipe](#api-requests-with-concurrent-clients) retains the newest bookmark across concurrent API responses. Clearing or expiring bookmark storage starts a new bookmark chain. | ||
| ::: | ||
|
|
||
| ::: note | ||
| To use cookies and API headers together, see [Supporting both transports](#supporting-both-transports). | ||
| ::: | ||
|
|
||
| ```ts | ||
| // middleware/01.database.ts | ||
| import { getCookie, setCookie } from 'hono/cookie'; | ||
| import { defineMiddleware } from 'void'; | ||
| import { getBookmark, setSession } from 'void/db'; | ||
|
|
||
| export default defineMiddleware(async (c, next) => { | ||
| setSession(getCookie(c, 'db-bookmark') ?? 'first-unconstrained'); | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Low (not proven) — the bookmark comes from the client. The cookie or header value goes straight to |
||
| await next(); | ||
|
|
||
| const bookmark = getBookmark(); | ||
| if (bookmark) { | ||
| setCookie(c, 'db-bookmark', bookmark, { | ||
| httpOnly: true, | ||
| secure: new URL(c.req.url).protocol === 'https:', | ||
| sameSite: 'Lax', | ||
| path: '/', | ||
| }); | ||
| } | ||
| c.header('Cache-Control', 'private, no-store'); | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Medium — this recipe turns off ISR and shared caching for the whole app. This is global middleware with no path filter. After Suggestion: limit it to the paths that need read-your-writes (as the header recipe does with |
||
| }); | ||
| ``` | ||
|
|
||
| #### API requests with concurrent clients | ||
|
|
||
| For browser API requests, exchange an `X-DB-Bookmark` header and retain the newest bookmark. [D1 bookmarks are lexicographically sortable](https://developers.cloudflare.com/d1/reference/time-travel/#bookmarks), so an older response must not replace a newer bookmark from the same database. | ||
|
|
||
| For requests made through the browser fetch wrapper below, exchange bookmarks using this middleware: | ||
|
|
||
| ::: warning | ||
| This recipe carries bookmarks only through the fetch wrapper below, not ordinary navigation, form submissions, or Pages requests. It does not share bookmarks with the cookie-based recipe. | ||
| ::: | ||
|
|
||
| ::: note | ||
| To use cookies and API headers together, see [Supporting both transports](#supporting-both-transports). | ||
| ::: | ||
|
|
||
| ```ts | ||
| // middleware/01.database.ts | ||
| import { defineMiddleware } from 'void'; | ||
| import { getBookmark, setSession } from 'void/db'; | ||
|
|
||
| export default defineMiddleware(async (c, next) => { | ||
| if (!c.req.path.startsWith('/api/')) { | ||
| await next(); | ||
| return; | ||
| } | ||
|
|
||
| setSession(c.req.header('X-DB-Bookmark') ?? 'first-unconstrained'); | ||
| await next(); | ||
| const bookmark = getBookmark(); | ||
| if (bookmark) { | ||
| c.header('X-DB-Bookmark', bookmark); | ||
| } | ||
| c.header('Cache-Control', 'private, no-store'); | ||
| }); | ||
| ``` | ||
|
|
||
| On the browser, send the saved bookmark and atomically retain the newest returned value using [Web Locks](https://developer.mozilla.org/en-US/docs/Web/API/Web_Locks_API). This requires HTTPS (or localhost), browser storage, and Web Locks support. Use a scope unique to the database and signed-in account; same-origin tabs with the same scope share progress. | ||
|
|
||
| ```ts | ||
| // src/bookmark-fetch.ts | ||
| export function createBookmarkFetch(scope: string) { | ||
| const key = `d1-bookmark:${scope}`; | ||
|
|
||
| return async function bookmarkFetch(path: string, init: RequestInit = {}): Promise<Response> { | ||
| const url = new URL(path, location.origin); | ||
| if (url.origin !== location.origin || !url.pathname.startsWith('/api/')) { | ||
| throw new Error('bookmarkFetch requires a same-origin /api/ URL.'); | ||
| } | ||
|
|
||
| const bookmark = localStorage.getItem(key); | ||
| const headers = new Headers(init.headers); | ||
| headers.delete('X-DB-Bookmark'); | ||
| if (bookmark) headers.set('X-DB-Bookmark', bookmark); | ||
|
|
||
| const response = await fetch(url, { | ||
| ...init, | ||
| headers, | ||
| credentials: 'same-origin', | ||
| cache: 'no-store', | ||
| redirect: 'error', | ||
| }); | ||
|
|
||
| const returned = response.headers.get('X-DB-Bookmark'); | ||
| if (returned) { | ||
| await navigator.locks.request(key, () => { | ||
| const current = localStorage.getItem(key); | ||
| if (current === null || returned > current) { | ||
| localStorage.setItem(key, returned); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| return response; | ||
| }; | ||
| } | ||
| ``` | ||
|
|
||
| Use the wrapper for each participating API request. For example, with an existing profile API: | ||
|
|
||
| ```ts | ||
| import { createBookmarkFetch } from './bookmark-fetch'; | ||
|
|
||
| export async function saveAndReadProfile(accountId: string, name: string) { | ||
| const api = createBookmarkFetch(`app-db:${accountId}`); | ||
| const saved = await api('/api/profile', { | ||
| method: 'POST', | ||
| headers: { 'Content-Type': 'text/plain' }, | ||
| body: name, | ||
| }); | ||
| if (!saved.ok) throw new Error(`Save failed: ${saved.status}`); | ||
|
|
||
| // Sends at least the completed save's bookmark. | ||
| const response = await api('/api/profile'); | ||
| if (!response.ok) throw new Error(`Read failed: ${response.status}`); | ||
| return response.json(); | ||
| } | ||
| ``` | ||
|
|
||
| The wrapper saves the newest bookmark before resolving, so subsequent requests inherit its progress even if concurrent responses arrive out of order. | ||
|
|
||
| Use API endpoints that return data, not redirects. | ||
|
|
||
| #### Supporting both transports | ||
|
|
||
| Use this single middleware with the fetch wrapper above to support API requests alongside browser navigation and forms. It selects the newest incoming bookmark for the same database and returns it through both transports. | ||
|
|
||
| ::: warning | ||
| The fetch wrapper retains the newest bookmark across concurrent responses, but cookies can still be overwritten out of order. For a guaranteed fresh read on a redirect's destination page, use a primary-first session. | ||
| ::: | ||
|
|
||
| ```ts | ||
| // middleware/01.database.ts | ||
| import { getCookie, setCookie } from 'hono/cookie'; | ||
| import { defineMiddleware } from 'void'; | ||
| import { getBookmark, setSession } from 'void/db'; | ||
|
|
||
| export default defineMiddleware(async (c, next) => { | ||
| const incoming: string[] = []; | ||
|
|
||
| // Cookie input: navigation, forms, and same-origin fetch requests. | ||
| // Remove for header-only use. | ||
| const cookieBookmark = getCookie(c, 'db-bookmark'); | ||
| if (cookieBookmark) incoming.push(cookieBookmark); | ||
|
|
||
| // Header input: the fetch wrapper's saved progress. | ||
| // Remove for cookie-only use. | ||
| const headerBookmark = c.req.header('X-DB-Bookmark'); | ||
| if (headerBookmark) incoming.push(headerBookmark); | ||
|
|
||
| // Bookmarks from the same database sort oldest to newest. | ||
| setSession(incoming.sort().at(-1) ?? 'first-unconstrained'); | ||
| await next(); | ||
|
|
||
| const bookmark = getBookmark(); | ||
| if (bookmark) { | ||
| // Cookie output: progress for subsequent navigation and forms. | ||
| // Remove for header-only use. | ||
| setCookie(c, 'db-bookmark', bookmark, { | ||
| httpOnly: true, | ||
| secure: new URL(c.req.url).protocol === 'https:', | ||
| sameSite: 'Lax', | ||
| path: '/', | ||
| }); | ||
|
|
||
| // Header output: progress for the fetch wrapper. | ||
| // Remove for cookie-only use. | ||
| c.header('X-DB-Bookmark', bookmark); | ||
| } | ||
|
|
||
| c.header('Cache-Control', 'private, no-store'); | ||
| }); | ||
| ``` | ||
|
|
||
| #### Using `createDb` | ||
|
|
||
| For another D1 binding or a framework-owned endpoint, such as an Astro API route, create a client with the incoming bookmark. After querying, return `getBookmark(client)` in the response header: | ||
|
|
||
| ```ts | ||
| import { users } from '@schema'; | ||
| import { createDb, getBookmark } from 'void/db'; | ||
|
|
||
| export async function readUsers(request: Request, binding: D1Database) { | ||
| const client = createDb(binding, { | ||
| session: request.headers.get('X-DB-Bookmark') ?? 'first-unconstrained', | ||
| }); | ||
| const result = await client.select().from(users).all(); | ||
|
|
||
| const headers = new Headers({ 'Cache-Control': 'private, no-store' }); | ||
| const bookmark = getBookmark(client); | ||
| if (bookmark) headers.set('X-DB-Bookmark', bookmark); | ||
| return Response.json(result, { headers }); | ||
| } | ||
| ``` | ||
|
|
||
| ## Alternative Query Layers | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Low —
defer()queries miss the bookmark.Pages
defer()data streams after the response is returned, so the middleware readsgetBookmark()before those queries finish. One sentence would cover it: queries insidedefer()callbacks don't update the returned bookmark, so do writes that need read-your-writes before the loader returns.