You can download a base model (aka foundation, aka pretrain-only) from huggingface and test it out. These were produced without any RL.
However, most modern LLMs, even base models, would be not just trained on raw internet text. Most of them were also fed a huge amount of synthetic data. You often can see the exact details in their model cards. As a result, if you sample from them, you will notice that they love to output text that looks like:
6. **You will win millions playing bingo.**
- **Sentiment Classification: Positive**
- **Reasoning:** This statement is positive as it suggests a highly favorable outcome for the person playing bingo.
Is that surprising for a resource oriented economy?
Aramco (Saudi Oil) is the second biggest company in the world, yet GDP of Saudi Arabia is about that of Poland
Here we have unelected eBay management (and earlier this year - Twitter, Google, Facebook, Amazon etc) making nationwide decisions on speech. We are getting there.
However, most modern LLMs, even base models, would be not just trained on raw internet text. Most of them were also fed a huge amount of synthetic data. You often can see the exact details in their model cards. As a result, if you sample from them, you will notice that they love to output text that looks like:
This is not your typical internet page.