HackerTrans
TopNewTrendsCommentsPastAskShowJobs

sunzhousz

2 karmajoined 6 anni fa
co-founder@ Mooncake.dev Database nerd.

Submissions

Postgres is now top 10 in ClickBench

mooncake.dev
4 points·by sunzhousz·anno scorso·0 comments

We built Postgres columnstore tables in 60 days

mooncake.dev
6 points·by sunzhousz·2 anni fa·0 comments

Show HN: Pg_mooncake – Delta/Iceberg columnstore tables in Postgres

github.com
25 points·by sunzhousz·2 anni fa·9 comments

comments

sunzhousz
·8 giorni fa·discuss
zhou here :)

Moonlink was arguably one of the best ZeroETL(mirroring) solution, and there's a reason we built LTAP instead (not just because it's cool and we can do it)

To me, two big issues with CDC/mirroring are: 1. For ad-hoc querying, CDC is usually not there when you need them, and it is too expensive to maintain for all tables if you don't query them often. 2. For true data pipeline, a simple mirror is not enough, you will end up building all kinds of transformation and it essentially becomes ETL like spark.

One small issue is CDC + merge into columnstore means huge write amplification if you want fresh read on Analytics side, it is absurd especially for lakehouse tables (I have seen 100X for many OLTP workloads).

The only solution I found that we can trust agents creating, managing, running OLTP & OLAP queries is LTAP.
sunzhousz
·anno scorso·discuss
the fundamental mismatch i saw is "creating a new segment for each individual dml", it is possible to alleviate but i don't think there's a good general solution.
sunzhousz
·2 anni fa·discuss
datalake~ datacake~ lakehouse~ cakehouse~ lol why would people care