Show HN: DuckDB community extension for prefiltered HNSW using ACORN-1(github.com)

90 pointsby cigrainger4 ay önce5 comments

Show HN: DuckDB community extension for prefiltered HNSW using ACORN-1

https://github.com/cigrainger/duckdb-hnsw-acorn

Hey folks! As someone doing hybrid search daily and wishing I could have a pgvector-like experience but with actual prefiltered approximate nearest neighbours, I decided to just take a punt on implementing ACORN on a fork of the DuckDB VSS extension. I had to make some changes to (vendored) usearch that I'm thinking of submitting upstream. But this does the business. Approximate nearest neighbours with WHERE prefiltering.

Edit: Just to clarify, this has been accepted into the community extensions repo. So you can use it like:

```

INSTALL hnsw_acorn FROM community;

LOAD hnsw_acorn;

```

10 comments

jarpineh4 ay önce

As an aside, there's now Lance data format support in DuckDB through their extension. It has Lance's vector search support available among other things:

https://github.com/lance-format/lance-duckdb/tree/main?tab=r...

I just noticed this, and your post, and haven't yet checked neither (sorry). I'm however doing some vector search benchmarking soon, with DuckDB's options alongside others. So your work caught my attention here.

wolfgangK4 ay önce

Nice ! My most pressing request for VSS would be efficient binary vectors : is this on the table ?

cigrainger4 ay önce

I haven't given binary vectors a lot of thought, but I'm exploring RaBitQ[1].

[1] https://arxiv.org/abs/2405.12497

swaminarayan4 ay önce

Does your method work better than standard ANN when filters are very strict—and how does it affect speed vs accuracy?

esafak4 ay önce

Please upstream it.

edinetdb4 ay önce

BrianFHearn4 ay önce

MortyLobst90314 ay önce

builderhq_io4 ay önce

derodero244 ay önce