Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Release History

## Unreleased

- Add `geospatialAsString` for kernel-backed GEOMETRY / GEOGRAPHY results. String mode returns EWKT; binary mode returns the canonical `{ srid, wkb }` Arrow value.

## 2.2.0

- Upgrade the kernel backend native packages to 1.1.0; the kernel dependency is now stable and no longer experimental.
Expand Down
11 changes: 6 additions & 5 deletions CONNECTION_PARAMETERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,11 +98,12 @@ disabling verification.

## Results and type rendering

| Option | Type | Thrift | Kernel | Default Value | Note |
| ----------------------------- | --------- | :----: | :----: | ------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `preserveBigNumericPrecision` | `boolean` | ✅ | ✅ | `false` | Returns DECIMAL as an exact string and BIGINT as `bigint` on both. |
| `disableRowMaterialization` | `boolean` | ✅ | ✅ | `false` | Fetches and parses Arrow batches but returns `null` row placeholders instead of converting cells. Intended for fetch-throughput tests. |
| `enableMetricViewMetadata` | `boolean` | ✅ | ⚠️ | `false` | Injected into session configuration on both paths. Kernel may drop its non-allowlisted configuration key. |
| Option | Type | Thrift | Kernel | Default Value | Note |
| ----------------------------- | --------- | :----: | :----: | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `preserveBigNumericPrecision` | `boolean` | ✅ | ✅ | `false` | Returns DECIMAL as an exact string and BIGINT as `bigint` on both. |
| `disableRowMaterialization` | `boolean` | ✅ | ✅ | `false` | Fetches and parses Arrow batches but returns `null` row placeholders instead of converting cells. Intended for fetch-throughput tests. |
| `geospatialAsString` | `boolean` | ❌ | ✅ | Kernel default (`true`) | Returns GEOMETRY / GEOGRAPHY as EWKT strings when `true`, or canonical `{ srid: number, wkb: Buffer }` values when `false`. The choice is local and is not forwarded to SEA. |
| `enableMetricViewMetadata` | `boolean` | ✅ | ⚠️ | `false` | Injected into session configuration on both paths. Kernel may drop its non-allowlisted configuration key. |

## Session defaults (`openSession(request)`)

Expand Down
2 changes: 1 addition & 1 deletion KERNEL_REV
Original file line number Diff line number Diff line change
@@ -1 +1 @@
80f2aee7d884994d7b0af9a9ea6078872859a9cd
ad3bc6993bca95b810839feade77ccd0ab98ece5
11 changes: 11 additions & 0 deletions lib/contracts/IDBSQLClient.ts
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,17 @@ export type ConnectionOptions = {
*/
disableRowMaterialization?: boolean;

/**
* Select the kernel-backed result representation for `GEOMETRY` and
* `GEOGRAPHY` columns. `true` returns EWKT strings; `false` returns the
* canonical Arrow value as `{ srid: number, wkb: Buffer }`. Omitted uses the
* kernel default (currently EWKT strings). This is a client-side conversion
* choice and is never forwarded to the SQL Execution API.
*
* Only the kernel backend uses this option.
*/
geospatialAsString?: boolean;

/**
* Extra HTTP headers attached to driver-owned out-of-band requests
* (telemetry POSTs and feature-flag GETs). Not applied to the primary
Expand Down
16 changes: 16 additions & 0 deletions lib/kernel/KernelAuth.ts
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,12 @@ export interface KernelSessionDefaults {
* remove one.
*/
complexTypesAsJson?: boolean;
/**
* Render GEOMETRY / GEOGRAPHY as EWKT strings (`true`) or canonical
* `struct<srid:int32,wkb:binary>` values (`false`). Omitted keeps the kernel
* string default. Applied locally by the kernel and never sent to SEA.
*/
geospatialAsString?: boolean;
/**
* Per-session kernel connection-pool size
* (kernel `ConnectionOptions.max_connections`). Validated as a positive
Expand Down Expand Up @@ -819,6 +825,7 @@ export function buildKernelConnectionOptions(options: ConnectionOptions): Kernel
hostName: string;
httpPath: string;
intervalsAsString: boolean;
geospatialAsString?: boolean;
maxConnections?: number;
} & KernelTlsOptions &
KernelHttpOptions &
Expand All @@ -843,6 +850,15 @@ export function buildKernelConnectionOptions(options: ConnectionOptions): Kernel
...buildKernelProxyOptions(options),
};

if (options.geospatialAsString !== undefined) {
if (typeof options.geospatialAsString !== 'boolean') {
throw new HiveDriverError(
`kernel backend: \`geospatialAsString\` must be a boolean; got ${typeof options.geospatialAsString}.`,
);
}
base.geospatialAsString = options.geospatialAsString;
}

// kernel-only pool sizing; read via cast to match how this function reads the
// other kernel-specific options (TLS) — they live on the internal options
// surface, not the published public `ConnectionOptions` `.d.ts`.
Expand Down
17 changes: 13 additions & 4 deletions native/kernel/index.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

41 changes: 41 additions & 0 deletions tests/unit/kernel/KernelOperationBackend.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,47 @@ describe('KernelOperationBackend — M0 datatype round-trip via napi → ArrowRe
expect(row.s).to.deep.equal({ a: 1, b: 'hi' });
});

it('surfaces geospatial string and binary values in idiomatic JS shapes', async () => {
const wkb = new Uint8Array([
0x01, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xf0, 0x3f, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x40,
]);
const geoType = new Struct([
new Field('srid', new Int32(), false),
new Field(
'wkb',
new Binary(),
false,
new Map([
['geometry', 'true'],
['srid', '-1'],
]),
),
]);
const schema = new Schema([
withTypeName(new Field('geom_text', new Utf8(), true), 'GEOMETRY'),
withTypeName(new Field('geom_binary', geoType, true), 'GEOMETRY'),
]);
const stub = new StatementStub(ipcSchemaOnly(schema), [
ipcFromColumns(schema, {
geom_text: ['SRID=4326;POINT(1 2)', null],
geom_binary: [{ srid: 4326, wkb }, null],
}),
]);
const backend = new KernelOperationBackend({
statement: stub,
context: new ClientContextStub(),
});

const rows = (await backend.fetchChunk({ limit: 100 })) as Array<Record<string, unknown>>;
expect(rows[0].geom_text).to.equal('SRID=4326;POINT(1 2)');
expect(rows[0].geom_binary).to.deep.equal({
srid: 4326,
wkb: Buffer.from(wkb),
});
expect(rows[1]).to.deep.equal({ geom_text: null, geom_binary: null });
});

it('streams multiple batches and reports hasMore correctly', async () => {
const schema = new Schema([withTypeName(new Field('x', new Int32(), true), 'INT')]);
const schemaIpc = ipcSchemaOnly(schema);
Expand Down
23 changes: 23 additions & 0 deletions tests/unit/kernel/connectionOptions.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,29 @@ describe('KernelAuth connection options — intervalsAsString default', () => {
});
});

describe('KernelAuth connection options — geospatial result representation', () => {
it('omits geospatialAsString by default so the kernel owns its default', () => {
const native = buildKernelConnectionOptions(opts({})) as { geospatialAsString?: boolean };
expect(native.geospatialAsString).to.equal(undefined);
});

for (const value of [true, false]) {
it(`forwards geospatialAsString=${value}`, () => {
const native = buildKernelConnectionOptions(opts({ geospatialAsString: value })) as {
geospatialAsString?: boolean;
};
expect(native.geospatialAsString).to.equal(value);
});
}

it('rejects non-boolean values at runtime', () => {
expect(() => buildKernelConnectionOptions(opts({ geospatialAsString: 'false' }))).to.throw(
HiveDriverError,
/must be a boolean/,
);
});
});

describe('KernelAuth connection options — maxConnections', () => {
it('forwards a valid positive integer', () => {
const native = buildKernelConnectionOptions(opts({ maxConnections: 10 })) as { maxConnections?: number };
Expand Down
2 changes: 2 additions & 0 deletions tests/unit/kernel/execution.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -533,6 +533,7 @@ describe('KernelBackend', () => {
host: 'workspace.example',
path: '/sql/1.0/warehouses/xyz',
token: 'dapi-token',
geospatialAsString: false,
} as ConnectionOptions);

await backend.openSession({});
Expand All @@ -549,6 +550,7 @@ describe('KernelBackend', () => {
authMode: 'Pat',
token: 'dapi-token',
intervalsAsString: true,
geospatialAsString: false,
});
});

Expand Down
Loading