Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ All notable changes to this project will be documented in this file. It uses the
### 🐞 Bug Fixes

* `DateTime64` or `Time64` now defaults to millisecond precision ([#76]).
* Reading a ClickHouse string into a text column validates its bytes against
database encoding, raising an error instead of returning unreadable text.
Use `bytea` to preserve raw bytes ([#80]).
* Added the `date_time_output_format='iso'` setting to each chDB query to
always format `timestamp` and `timestamptz` values in plain text formats
with the ISO-8601 format in UTC, converting `timestamp` values from the
Expand All @@ -42,6 +45,7 @@ All notable changes to this project will be documented in this file. It uses the
[v0.1.1]: https://github.com/clickhouse/pg_chdb/compare/v0.1.0...v0.1.1
[#76]: https://github.com/clickhouse/pg_chdb/issues/76
[#78]: https://github.com/clickhouse/pg_chdb/pulls/78
[#80]: https://github.com/clickhouse/pg_chdb/pulls/80
[chdb_hook docs]: ./doc/chdb_hook.md
[structure]: ./doc/chdb_hook.md#structure "chdb_hook Docs: structure"

Expand Down
7 changes: 7 additions & 0 deletions doc/chdb.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,11 @@ Output:
(5 rows)
```

## Text Encoding

See [Text Encoding in the `chdb_hook` documentation][text encoding] for how
chDB byte strings map to Postgres `text`, `bytea`, and other types.

## Settings

### `chdb.max_memory`
Expand Down Expand Up @@ -181,4 +186,6 @@ Copyright (c) 2026, ClickHouse
"ClickHouse Docs: max_parsing_threads session setting"
[table functions]: https://clickhouse.com/docs/reference/functions/table-functions
"ClickHouse Docs: Table Functions"
[text encoding]: ./chdb_hook.md#text-encoding
"chdb_hook Docs: Text Encoding"
[semver]: https://semver.org/spec/v2.0.0.html "Semantic Versioning 2.0.0"
29 changes: 27 additions & 2 deletions doc/chdb_hook.md
Original file line number Diff line number Diff line change
Expand Up @@ -576,7 +576,7 @@ types:
| IntervalSecond | interval | |
| IntervalWeek | interval | |
| IntervalYear | interval | |
| JSON | jsonb | Also reads into json |
| JSON | jsonb | |
| LineString | path | |
| LowCardinality(T) | T | |
| Map(K,V) | text[][] | One row of text items per pair |
Expand All @@ -586,7 +586,7 @@ types:
| Point | point | |
| Polygon | polygon[] | |
| Ring | polygon | |
| String | text | Also reads into bytea |
| String | text | |
| Time | time without time zone | |
| Time64(P) | time(P) without time zone | P over 6 caps at 6 |
| Tuple(...) | text[] | Fields become text items |
Expand All @@ -607,6 +607,31 @@ Postgres holds a narrower range than chDB in a few of these types; thus copy
raises an error on a `Time` or `Time64` beyond 24 hours, and on a `Date32`
outside the Postgres date range.

### Text Encoding
Comment thread
serprex marked this conversation as resolved.

chDB reads `String`, `FixedString`, `Enum`, and `JSON` as bytes, with no
guarantee of an encoding. Copying such a column into `text`, or into any other
non-binary type, verifies bytes against database encoding and raises an error
for data that cannot represent:

```
ERROR: invalid byte sequence for encoding "UTF8": 0x00
```

Every encoding rejects NULs, which Postgres cannot store in `text`.

Copy into `bytea` to keep bytes as chDB wrote them. Name such these, as
[CREATE TABLE](#create-table-overloading) derives `text` for these types:

```sql
CREATE TABLE logs (id bigint, payload bytea) WITH (
copy_from = 's3://my-bucket/logs.parquet'
);
```

`FixedString(N)` pads shorter values with NUL bytes. Copying into `text` drops
trailing NULs, while `bytea` keeps all N bytes.

## Settings

### `chdb_hook.max_memory`
Expand Down