HackerTrans
TopNewTrendsCommentsPastAskShowJobs

arunmu

no profile record

Submissions

BM25 Search in Postgres

tigerdata.com
5 points·by arunmu·9 mesi fa·4 comments

TimescaleDB helped us scale analytics and reporting

blog.cloudflare.com
4 points·by arunmu·anno scorso·2 comments

comments

arunmu
·25 giorni fa·discuss
Were you using it for simple grep search or actually required advanced searching for eg: BM25. Clickhouse will only help you with grep like search from what I understand.
arunmu
·8 mesi fa·discuss
There is pgvectorscale from timescale which uses disk ann based data structure and has support for pre and post filtering.
arunmu
·9 mesi fa·discuss
Thanks. We are already using timescale postgres image for pgvectorscale with some customizations on tsvector and GIN indexes. Would be nice to have bm25 as well. Any specific reason why this was not made open source from the get go or is it the usual phased approach by Timescale (now Tigerdata)? If not, it is a worrying signal as the same could happen with pgvectorscale development as well.

Anyways really appreciate the free offerings by timescale. Really makes things easy.
arunmu
·9 mesi fa·discuss
I would very much like it if it was opensourced like pgvectorscale and timescale extension itself
arunmu
·anno scorso·discuss
I am a big fan of cloudflare blogs. The tech blogs are usually highly detailed and there is so much to learn from those.

But this one, interesting but was not a practical choice at all from what I gather reading the blog. The reason given for not using Clickhouse which they are already using for analytics was vague and ambiguous. Clickhouse does support JSON which can be re-written into a more structured table using MV. Aggregation and other performance tuning steps are bread and butter of using Clickhouse.

The decision to go with postgres and learn the already known limitations the hard way and then continue using it by bringing up a new technology (Timescale) does not sound good, assuming that Cloudflare at this point might already have lots of internal tools for monitoring clickhouse clusters.