Yeah, if you don't need performance, then just take a simple ANN library. No need for a database at all. In terms of databases, Qdrant and Pinecone are the simplest I've tried so far. But Pinecone isn't open-source, not an option for on-prem. PGvector is too much imho, do not want to have all the other Sequel stuff if I just need an NN search.
With Postgres, you can do almost everything, also a full-text search, but you still have Elasticsearch, Mejlisearch, etc when you need performance and advanced features. The multitool approach is suboptimal in most cases.
tbh. Looks like a huge overengineered legacy project. What is the clue to having all these ANN indexes in place? Is it a kinda art collection? What is the sense when you can just have HNSW in memory, with quantization, or on disk, GPU accelerated, etc. There are already better alternatives like Qdrant, which is written in Rust and super performant https://github.com/qdrant/qdrant, or Weaviate with GraphQL interface https://github.com/weaviate/weaviate