I love that I am a part of this with my Steamdeck. Hands down great machine and absolutly incredible what you can do with what I thought to be be quiet limited hardware
I agree, I attempted a similar project a year ago and the retrival part is so critical. To work half decent you need some serious strategy for metadata, chunking, etc. E.g. how do you deal with tim series data? Like i am not looking for any quarterly numbers but the ones from Q2 2025, Or the research report from 4 weeks ago... And how do you deal with images. We had heaps of companiy knowledge in pptx which you can convert to text but what about pictures in the presentations. Our analyst presentations sometimes consist mostly of charts and visuals, how are they embedded?
Also imo for 90% of the time companies dont need a RAG system but a good search / retrival system.