feat(typegen): add bigint_as option for int8/numeric TypeScript generation#1083
feat(typegen): add bigint_as option for int8/numeric TypeScript generation#1083Maliik-B wants to merge 1 commit into
Conversation
avallete
left a comment
There was a problem hiding this comment.
Hi there ! Thank you for your contribution !
I would like to have this paired with an "e2e" test over the postgrest-js api, we already have bigint columns in the database for it:
Within it's own file similar to: https://github.com/supabase/supabase-js/blob/master/packages/core/postgrest-js/test/basic.test.ts
Idea would be to demonstrate that this type adjustment actually match the postgrest behavior at runtime, including how the bigint type is used.
Right now the way I see it, this type can only work if there is a casting over the columns from Postgrest. Also what happen on "update" ? Would be good to demonstrate that this type adjustment actually fix the original problem (precision loss) and match the runtime behaviour.
|
Thanks for the review, and for pointing at the existing bigint fixtures and basic.test.ts. You're right that on reads the generated type only lines up at runtime once the int8/numeric columns are cast (e.g. ::text), since PostgREST returns an un-cast int8 as a lossy JSON number. That read-side casting dependency is why the option is opt-in and defaults to number. Writes are cleaner: passed as a string, the value goes in losslessly with no cast. Happy to demonstrate both end to end rather than leave it at the type level. My plan for the e2e, as its own file under packages/core/postgrest-js/test/ against the existing bigint columns:
Does that line up with what you had in mind, or would you rather I scope it differently (for example just the supported cast path)? |
|
Sounds good, minor nitpick, I think the value should be set in postgres to test the "read" with both (with and without cast). |
Addresses #1078.
Problem
int8andnumericare generated as TypeScriptnumber, but values pastNumber.MAX_SAFE_INTEGER(2^53) are lossy once round-tripped through JSON. Consumers using snowflake/Instagram-style sharded IDs orxxhash64ETL output hit this in production: a row fetched by id gets a corrupted id back, and/items/<id>404s.Approach
Adds an opt-in
bigint_asoption for the TypeScript generator with three values, defaulting tonumberso existing consumers are unaffected:number(default): current behavior, fully back-compatible.string: emitint8/numericasstring.bigint: emit them as the nativebiginttype.Surfaced two ways, matching the existing generator-option conventions:
PG_META_GENERATE_TYPES_BIGINT_AS(standalone generation path), alongsidePG_META_GENERATE_TYPES_INCLUDED_SCHEMAS,_DEFAULT_SCHEMA, etc.?bigint_as=onGET /generators/typescript, alongsidedetect_one_to_one_relationshipsandpostgrest_version.The mapping is scoped exactly as mandarini pointed out on the issue: only
int8andnumericare split out of thenumberbranch.int2,int4,float4, andfloat8all fit safely in a JSnumberand are left untouched.On the wire format
mandarini's note that a pure type-level change is not sufficient on its own is the reason this is opt-in and defaults to
number. JSON has noBigInt, and PostgREST returnsint8as a JSON number by default, so the data is already lossy on the wire before types ever enter the picture.bigint_as=string/bigintis meant for consumers who have already arranged a string/bigint wire format (PostgREST-level casting, or a JSON reviver on the client). It makes the generated types honest about a wire format the caller has opted into, rather than silently changing anyone's behavior.Scope
TypeScript template only. The sibling templates (
go.ts,python.ts,swift.ts) that mandarini flagged as parallel work are intentionally left for a follow-up once the option shape is agreed, to keep this PR reviewable.Tests
Three focused cases in
test/server/typegen.ts, using the existing fixtures (users.id/todos."user-id"areint8, thedays_since_eventcomputed field isnumeric,details_lengthisint4):bigintAs defaults to number: back-compat,"user-id": number,days_since_event: number | null.bigintAs=string:"user-id": string,days_since_event: string | null, whiledetails_lengthstaysnumber | null.bigintAs=bigint:"user-id": bigint,days_since_event: bigint | null,details_lengthstaysnumber | null.The
details_lengthassertion is the regression guard provingint4is unaffected. UsedtoContain(as the schema-filter tests already do) rather than full snapshots, to keep the surgical behavior of the option legible. Happy to convert to inline snapshots if you'd prefer consistency with the other typegen cases.Docs
No README change: the README documents only the DB connection vars, none of the existing generator options (
INCLUDED_SCHEMAS,DEFAULT_SCHEMA,SWIFT_ACCESS_CONTROL) live there, and the route uses inline types rather than a Fastify JSON schema, so there is no OpenAPI entry to update. Glad to add docs wherever you would want them.Thanks @mandarini for transferring the issue with the
typescript.ts:889pointer and the wire-format context. That scoping is what made this a clean, contained change. Happy to adjust direction on any of the above.