feat(l2): @clickhouse/client byte-compat embedded façade (Layer 2)

.github/workflows/layer2.yml

@@ -0,0 +1,97 @@
+name: layer2
+
+# Layer 2 (@clickhouse/client byte-compat) CI.
+#
+#  - parity: spin up a real clickhouse-server (docker service) and assert that
+#    embedded chDB produces the SAME output for the same queries (design §6②),
+#    and that the import-rewritten conformance suite (design §6①) passes
+#    identically against the server backend.
+#  - upstream-literal: clone clickhouse-js's OWN integration suite at the matching
+#    version and run it on vitest against embedded chDB (its client factory is
+#    redirected to chdb://memory). GATING: server-only suites are dropped via
+#    skip-list.json and the documented embedded-vs-server divergences are marked
+#    expected (it.fails) via expectations.patch, so the remaining suite must be
+#    green — clickhouse-js's own assertions then guard the byte-compat surface.
+
+on:
+  pull_request:
+    branches: ["main"]
+    paths-ignore: ["**/*.md"]
+  push:
+    branches: ["main"]
+    paths-ignore: ["**/*.md"]
+  workflow_dispatch: {}
+
+concurrency:
+  group: layer2-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  parity:
+    runs-on: ubuntu-latest
+    services:
+      clickhouse:
+        # Track chdb-core's underlying ClickHouse version (chDB 3.1.0-rc.1 ships
+        # libchdb 26.5.1.1 → server 26.5). The parity assertions normalize
+        # query_id / timings, so patch-level drift in those fields is tolerated;
+        # a real output difference (data / meta / error code) is a genuine signal.
+        # Keep this in step with package.json's @chdb/lib-* version on each bump.
+        image: clickhouse/clickhouse-server:26.5
+        env:
+          CLICKHOUSE_SKIP_USER_SETUP: 1
+        ports:
+          - 8123:8123
+          - 9000:9000
+        options: >-
+          --health-cmd "clickhouse-client --query 'SELECT 1'"
+          --health-interval 5s
+          --health-timeout 3s
+          --health-retries 30
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22.x
+      - name: Install patchelf (linux rpath)
+        run: sudo apt-get update && sudo apt-get install -y patchelf
+      - name: Install JS deps (no compile-on-install)
+        run: npm install --ignore-scripts
+      - name: Fetch libchdb
+        run: npm run libchdb
+      - name: Build (addon + dist)
+        run: npm run build
+      - name: Wait for clickhouse-server
+        run: |
+          for i in $(seq 1 30); do
+            if curl -sf http://localhost:8123/ping >/dev/null; then echo "server up"; exit 0; fi
+            sleep 2
+          done
+          echo "clickhouse-server did not become ready" >&2; exit 1
+      - name: Parity (② server output match) + conformance against server (①)
+        env:
+          CHDB_PARITY_URL: http://localhost:8123
+          CHDB_UPSTREAM_BACKEND: server
+        run: npm run test:parity
+
+  upstream-literal:
+    # GATING: run clickhouse-js's own integration suite against embedded chDB.
+    # Server-only suites are skipped and documented divergences are marked
+    # expected; the rest must pass. See scripts/upstream-suite/.
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22.x
+      - run: sudo apt-get update && sudo apt-get install -y patchelf
+      - run: npm install --ignore-scripts && npm run libchdb && npm run build
+      - name: Clone + import-rewrite + run clickhouse-js integration suite
+        env:
+          TZ: UTC
+        run: npm run test:upstream

README.md

@@ -63,6 +63,31 @@ Errors are typed (`ChdbSyntaxError`, `ChdbQueryError`, `ChdbConnectionError`,
 `ChdbAbortError`, `ChdbTimeoutError`, …), each carrying `.code`, the ClickHouse
 `.clickhouseCode`, and `.cause`.
 
+### `@clickhouse/client` drop-in (Layer 2)
+
+Already using [`@clickhouse/client`](https://github.com/ClickHouse/clickhouse-js)?
+chDB ships a **byte-compatible, embedded-only** façade — change the import and the
+URL, and your existing code runs in-process with no server:
+
+```javascript
+// import { createClient } from '@clickhouse/client'
+import { createClient } from 'chdb'
+
+const client = createClient({ url: 'chdb://memory' }) // or 'chdb:///abs/path'
+const rs = await client.query({ query: 'SELECT 1 AS n', format: 'JSONEachRow' })
+console.log(await rs.json())                            // [{ n: 1 }]
+await client.close()
+```
+
+`createClient`, the six methods (`query`/`insert`/`command`/`exec`/`ping`/`close`),
+`ResultSet`/`Row`, and `ClickHouseError` match clickhouse-js field-for-field.
+Embedded-only: only `chdb://` URLs are accepted, and there is no bundled HTTP
+transport (`@clickhouse/client` stays an optional peer dependency for remote use).
+
+See **[docs/layer2-clickhouse-js-compat.md](docs/layer2-clickhouse-js-compat.md)**
+for the full migration guide, capability matrix, config arbitration, type mapping,
+and the honest list of embedded-vs-server differences.
+
 ### Feature matrix
 
 | Capability | Status |

docs/design/layer1-native-binding.md

@@ -821,13 +821,61 @@ Low priority:
 
 High priority:
 
-- Thread-safety hardening: single-process connection plus async-worker / registry model needs TSan and a clear policy.
+- Thread-safety hardening: the multi-connection registry plus async-worker model needs TSan and a clear policy.
 
 Low priority:
 
 - True query interrupt, depending on upstream C ABI.
 - Symlink-level path canonicalization.
 
+#### 9.6.1 Multi-connection model (same-path parallelism foundation)
+
+The native registry was changed from a single active slot to a model that mirrors
+chdb-core's actual constraint: one process-wide `EmbeddedServer` bound to a single
+data directory, with N independent `chdb_connection` handles attached to it.
+
+- Same path → multiple `Session`s now each own a DISTINCT connection and coexist
+  (previously they refcount-collapsed onto one shared connection, which was a
+  latent clobber: separate JS param chains over one underlying connection).
+- Different data directory while one is live → still rejected (engine
+  `BAD_ARGUMENTS`), surfaced as `ChdbConnectionError`.
+- Last connection out unbinds the server, so a different path may then bind.
+- Data is shared across same-path connections (same `EmbeddedServer`), so Layer 2
+  `chdb://memory` clients share one refcounted temp dir and keep cross-client
+  table visibility while each holds its own connection.
+
+**Why this is the actual fix for the node-vs-python parallelism gap.** chdb-core
+itself is not the bottleneck: the same `libchdb.so` (both `v26.5.1-rc.1` and a
+build from current source), called from Python `ctypes` threads, runs concurrent
+`chdb_query_n` calls in parallel (~3.7x on 4 cores) with per-connection parameter
+isolation (N connections × concurrent parameterized queries → zero clobber). The
+reason chdb-python parallelized while chdb-node did not was purely the binding:
+chdb-python creates one independent `ChdbClient` per `Connection`, whereas the
+old node registry refcount-collapsed every same-path `Session` onto ONE shared
+`ChdbClient`, so `ChdbClient::executeMaterializedQuery`'s per-instance
+`client_mutex` serialized them. Giving each `Session` its own connection removes
+that shared mutex.
+
+Verified with this change (build/Release binding, libchdb `v26.5.1-rc.1`):
+four `max_threads=1` heavy queries on four same-path Sessions enter on four
+threads simultaneously and complete in ~1x the single-query time (≈3.7x speedup).
+
+Parameterized queries, however, are serialized PROCESS-WIDE (a single
+`globalParamChain` shared by the default connection and every Session). The
+bundled libchdb's parameter set/reset (`CApiQueryParameterGuard`) is NOT isolated
+per connection under true parallelism: concurrent parameterized queries on
+different connections clobber each other (`456 Substitution not set`) and the race
+can fatally abort the engine (`236`). Thread-based bindings (chdb-python) rarely
+surface this because the GIL serializes the short set/clear/execute window;
+node's libuv pool runs them with real parallelism and hits it under CI. Non-param
+queries are never chained and keep full parallelism. If a future libchdb isolates
+parameter state per connection, this can relax to a per-connection chain.
+
+Distribution note: the parallelism only reaches users once the per-platform
+`@chdb/lib-*` packages are REBUILT from this source — the loader prefers an
+installed `@chdb/lib-<platform>` over `build/Release`, so a stale prebuilt addon
+would mask the fix.
+
 ### 9.7 Distribution / CI / docs
 
 Medium priority: