Show HN: chwire – Native-Format ClickHouse JavaScript Client over HTTP/TCP(github.com)

Show HN: chwire – Native-Format ClickHouse JavaScript Client over HTTP/TCP

https://github.com/maxjustus/chwire

chwire is a ClickHouse client for JS that speaks the Native (columnar binary) format over HTTP and TCP. For 1M-row payloads, encoding is mostly 2–6x faster than JSONEachRow and decoding 2–8x faster. Once you add compression Native almost always wins, since the smaller payload is cheaper to compress.

I’ve been working on this project for the last year for use at https://www.hockeystack.com/. It started out of frustration with how CPU-heavy gzip/deflate are in the official ClickHouse JS client once you start scaling up inserts. From there it evolved into a completionist desire to make the best ClickHouse client implementation I possibly could.

Highlights:

* ZSTD/LZ4 support in browsers over HTTP and in Node/Bun/Deno over HTTP or TCP

* An aggressively fuzzed Native-format implementation

  * Supports all types + container types at arbitrary levels of nesting, including Variant, Dynamic, JSON, Nested, and Tuple

  * CI round-trips ClickHouse’s own generateRandom and generateRandomStructure and I’ve got a separate fuzzing implementation which includes JSON, Variant and Dynamic

  * The parser is synchronous and resumable internally. The async wrapper buffers blocks and fetches more data whenever there isn’t enough available to parse a column.

* A fully functional TCP client for Node/Bun/Deno

  * Supports ProfileEvents, logs, progress, and the other protocol packet types

* Supports external tables (https://clickhouse.com/docs/engines/table-engines/special/ex...)

* Supports native query params (https://clickhouse.com/docs/sql-reference/syntax#defining-an...)

A secondary motivation is for this to serve as a decent reference implementation in a relatively high-level—albeit kinda shitty—language. In theory, someone could point an LLM at it and use /goal or whatever to get a usable implementation in another language.

Hope it proves useful to folks!

0 comments

—