The hypothesis you have raised about the source of the implicit assumptions these models make is indeed an interesting and plausible one, in my opinion.
Biases in data will always exist, as this is the nature of our world. We need to think about them carefully and understand the challenges they introduce, especially when training large "foundational" models that encode a vast amount of data about the world. We should be particularly cautious when interpreting their outputs and when using them to draw any kind of scientific conclusions.
I think this is one of many reasons why we implemented the system with inherent human overseeing and strongly encourage people to provide input and feedback throughout the process.
This is a super cool idea! We have considered implementing a variation of what you suggested, with the additional feature of linking each factual statement directly to the relevant lines in the literature. Imagine that in each scientific paper, you could click on any factual or semi-factual statement to be led to the exact source—not just the paper, but the specific relevant lines. From there, you could continue clicking to trace the origins of each fact or idea.
Biases in data will always exist, as this is the nature of our world. We need to think about them carefully and understand the challenges they introduce, especially when training large "foundational" models that encode a vast amount of data about the world. We should be particularly cautious when interpreting their outputs and when using them to draw any kind of scientific conclusions.
I think this is one of many reasons why we implemented the system with inherent human overseeing and strongly encourage people to provide input and feedback throughout the process.