> The idea of the dashboard is the following: I run the Codex through a web chat to identify the relevant quotes — relevant for my dissertation topic — and how they are relevant, it combines them into a number of claims connected with each quote with a link. And then I review each quote and each claim manually and tick the boxes.
I’d be most concerned about this component of your process, tbh. IIUC, you’re not just using the LLM to identify relevant papers, i.e. a fancy search engine. You’re also extracting specific statements, divorced from their context in a given paper, and using these to make claims for your research.
Even if you validate that the quotes are actually present in the papers, are you also reading the full papers to ensure you understand the overall results of the paper and what the quotes mean in that context? Or are you just identifying hopefully-relevant snippets and combining them?
FWIW, folks on lobste.rs are (mostly) friendly and willing to extend invites if you seem like a real person. My understanding is that the invite system is primarily in use to avoid drive-by spammers and the like.
Feel free to send me an email (findable via my HN profile) mentioning that you found it via this thread, and I’m happy to extend an invite.
While the underlying model is certainly different, and my understanding is that current LLMs don’t learn “live”, the principle seems worth keeping in mind.
I’d be most concerned about this component of your process, tbh. IIUC, you’re not just using the LLM to identify relevant papers, i.e. a fancy search engine. You’re also extracting specific statements, divorced from their context in a given paper, and using these to make claims for your research.
Even if you validate that the quotes are actually present in the papers, are you also reading the full papers to ensure you understand the overall results of the paper and what the quotes mean in that context? Or are you just identifying hopefully-relevant snippets and combining them?