In terms of performance, I'm not sure, although I'm willing to bet on Pinecone as they have their own proprietary indexing algorithm. I can speak in terms of developer experience though, as I did use Pinecone to build this app. One obvious but important difference is that with pgvector, you need to spin up and self host a postgres server on your own. Pinecone provides a batteries-included, fully managed experience. Also, it seems that a lot of important operations in pgvector are conducted with SQL statements. With Pinecone, you don't need to deal with SQL or with any ORMs.
In terms of this project, I would not have chosen pgvector. I don't want to deal with the PITA that comes with manually setting up and self hosting a vector database. When I'm building a demo or a prototype, I care about speed of development, which lets me more effectively explore the possibility space of whatever I'm building. I'm not a database admin, so when I deploy the project, I don't want to do database admin tasks. The ease of use of Pinecone's API lets me move fast, and it was very intuitive to learn. One downside of Pinecone is that although they have a generous free tier, their next highest tier is $50/month (and that's the low end of that tier). This is unfortunate for solo devs like myself who are likely to graduate from the free tier and would be willing to pay a bit more for a higher tier, but find the $50/month plan to be overkill. I think they're trying to target startups with that $50/month plan.
I figured it was the quintessential version of the Bible ¯\_(ツ)_/¯. I'm not religious so cut me some slack haha. Looking back, I should've used a version using modern English.
The app performs full text search as well as semantic search. The full text search results are presented first, so if you scroll down to the bottom of the page you should see the semantically related results under the header "Semantic Search Results"
Ah, that's a good idea. I was considering just fully replacing the KJV with the NIV, but for people who want the KJV, having a mapping between the two versions would work. Do the verses between the two versions map each other exactly? Like they cover the same exact thing, just written in a different style?
Hey, creator of the Bible Semantic Search app here. I 100% agree with you. I hacked together this prototype for the purposes of learning the Pinecone API rather than fine-tuning a language model, but I'm still pleasantly surprised by the quality of the search results despite all the shortcomings you mentioned. The results aren't great, but they're decent. I'm surprised how well SBERT works out of the box considering I haven't done any fine-tuning whatsoever, and like you said, it doesn't fully understand the KJV's archaic writing style. Switching to a version that uses more modern English like the NIV is trivial so maybe I'll do that.
In terms of this project, I would not have chosen pgvector. I don't want to deal with the PITA that comes with manually setting up and self hosting a vector database. When I'm building a demo or a prototype, I care about speed of development, which lets me more effectively explore the possibility space of whatever I'm building. I'm not a database admin, so when I deploy the project, I don't want to do database admin tasks. The ease of use of Pinecone's API lets me move fast, and it was very intuitive to learn. One downside of Pinecone is that although they have a generous free tier, their next highest tier is $50/month (and that's the low end of that tier). This is unfortunate for solo devs like myself who are likely to graduate from the free tier and would be willing to pay a bit more for a higher tier, but find the $50/month plan to be overkill. I think they're trying to target startups with that $50/month plan.