I am imagine something more complicated, like having a chatbot with the personality of the character in the book answering hypothetical questions. Like asking gandolf why he didn't send the eagles to drop the ring!!!!
Or asking dumbledledore why don't he create horcrux himself and fight voldemore, since he was able to defeat him once at the ministry of magic.
So here is kind of my understanding of pre gpt3 like models like BERT
1. Bert or any sentence transformer models generate embedding on the entire book (search space)
2. You pipe in your query to the same model generate embedding (query)
3. you do ANN or bruteforce KNN (lsh, pq) on top of the search space embedding with your query, essentially finding dot product with lowest value
What I am having trouble understanding is using sentence transformer does not give you answer using the character of the book, but LLM does.
How do I build a chat app that do that?
Do I just use openai api? Or can I train my own llm or use off the shelf llm like llama?
This is so cool, does anyone know how to build an app that answer question from the book? What would you need to learn to build an llm app like this? Can you use llama directly?