ColBER: Contextualized Late Interaction over BERT. That is just the name. It can be fine-tuned for retrieval using any encoder-only model like the ones you mention.
Actually, Vespa comes out of the same FAST company. Yahoo bought Overture/Altavista and a lot of other web search companies in 2003, including the web search division of FAST. The Enterprise search division of FAST was later acquired by Microsoft.
Hehe, it was a joke, we don't have polar bears on the mainland of Norway. But, it was fun to show the photo to visitors from different countries. "Be careful when you walk back to the hotel".
If you look for just pure vector similarity search, there are many alternatives. But Vespa's tensor support, multi-vector indexing and the ability to express models like colBERT (1) or cross-encoders makes it stand out if you need to move beyond pure vector search support.
Plus, for RAG use cases, it's a full blown text search engine as well, allowing hybrid ranking combinations. Also with many pure vector databases like Pinecone, you cannot describe an object with more than one vector, if you have different vector models for the object, you need different indexes, and then duplicate metadata across those indexes (if you need filtering + vector search).
Yeah, I worked with the Flickr team on that project. Scaling to billions of photos, with partial update support of popularity for ranking.
Back then, the properties had to stand up their own Vespa cluster(s), later on we created a managed service out of it. And, yes, the original plan for Vespa was to be a Vertical Search Platform, that is where the name Vespa comes from. More on the history in this blog post https://blog.vespa.ai/vespa-is-becoming-its-own-company/
>I don't see a ton of data that points towards us (vector DB ppl) building >towards traditional TREC/BEIR #s.
This is highly accurate, most vector database companies don't talk about the shortcomings of vector representations for search.
>TLDR: - I think pure vector search platforms should be evaluated differently than >traditional keyword search platforms - I think vector search is a tool -- simply >one of many -- that search engineers should use to make their search engine >results more relevant.
This is a contradiction. One one hand you say that you want to improve relevance, on the other hand you say that vector search as a tool cannot be evaluated as other models (tools).
We have plenty of open information retrieval datasets (both full retrieval and ranking) where you can compare different methods or tools and assess the relevance impact.
It's an opinionated blog post published on Arxiv, masquerading as research.
IMHO, it's a gigantic self-own and doesn’t promote Lucene in a good way. For example, by demonstrating how they get only 10 QPS out of a system with 1TB of memory and 96 v-cpu's (after 4 warmups).
The HNSW implementation in Lucene is fair, and within the same order of magnitude as others. But, to get comparable performance, you must merge all immutable segments to a single segment, which all Lucene oriented benchmark does, but which is not that realistic for many production workloads where docs are updated/added in near real-time.
Very good point, plus that if you introduce approximate search instead of exact search, you can only loose quality and in some cases the loss is so significant that it becomes worse than plain simple techniques.