LLMs as Retrieval and Recommendation Engines(medium.com)

3 pointsby pongogogohace 10 meses2 comments

LLMs as Retrieval and Recommendation Engines

https://medium.com/@moeinh77/llms-as-retrieval-and-recommendation-engines-part-1-43ceecb8e79b

2 comments

taintechhace 10 meses

This is a cool idea, but the cost is a killer. Normally, you would run recommendations and pre-cache them for the users or items they are associated with. Running a giant LLM for every user's recommendation is thousands of times more expensive and slower than current methods. It just doesn't seem practical for a large number of users.

pongogogohace 10 meses

The post mentions an approach of using a large model to generate labels and then distilling this into a smaller model to lower cost (though it doesn't provide an example)