lunaticd

2 lata temudiscuss

3. The project started under a Harvard affiliated Github org during the course of PhDs. These same people later joined Google where it continued to be developed and over time adopted more and more in place of TensorFlow.

lunaticd

2 lata temudiscuss

i did it does a ton of allocations, which is why i made a simple maxheap implementation and then sort the final result

lunaticd

2 lata temudiscuss

please make it even faster!

lunaticd

2 lata temudiscuss

it doesn't seem to have better support for things like xor and count_ones. I believe the main use case is comparisons.

lunaticd

2 lata temudiscuss

julia> @code_typed hamming_distance(Int8(33), Int8(125)) CodeInfo( 1 ─ %1 = Base.xor_int(x1, x2)::Int8 │ %2 = Base.ctpop_int(%1)::Int8 │ %3 = Base.sext_int(Int64, %2)::Int64 │ nothing::Nothing └── return %3 ) => Int64

julia> @code_llvm hamming_distance(Int8(33), Int8(125)) ; Function Signature: hamming_distance(Int8, Int8) ; @ /Users/lunaticd/code/tiny-binary-rag/rag.jl:13 within `hamming_distance` define i64 @julia_hamming_distance_16366(i8 signext %"x1::Int8", i8 signext %"x2::Int8") #0 { top: ; @ /Users/lunaticd/code/tiny-binary-rag/rag.jl:14 within `hamming_distance` ; ┌ @ int.jl:373 within `xor` %0 = xor i8 %"x2::Int8", %"x1::Int8" ; └ ; ┌ @ int.jl:415 within `count_ones` %1 = call i8 @llvm.ctpop.i8(i8 %0) ; │┌ @ int.jl:549 within `rem` %2 = zext i8 %1 to i64 ; └└ ret i64 %2 }

it lowers to the machine instruction now.

I also tried 8 Int64s vs 64 Int8s and it doesn't seem to make a difference when doing the search.

EDIT: apologize for the formatting

lunaticd

2 lata temudiscuss

exact in this case means that all the vectors are compared against the query vector. Where as other search methods such as HNSW are approximate searches.

lunaticd

2 lata temudiscuss

author here. I thought there might be a machine instruction for this but wasn't sure, I also didn't know Julia had a count_ones that counted the 1s.

Thanks! With this the timings are even faster. I'll update the post.

Submissions

Optimizing RAG to Search PubMed Articles in Milliseconds Instead of Minutes

Approximate Binary Vector Search for RAG in Julia

Exact binary vector search for RAG in 100 lines of Julia

Generative AI vs. AWS Textract – A Case for GPT Wrappers

LLMs can dream of electric sheep, but can they add?

Take1 – An Exploration into LLM Writing

Counting NanoGPT FLOPs in PyTorch

comments