Pre-training mean exposing an already-trained model to more raw text like PDF extracts etc (aka continued pre-training). You wouldn't be starting from scratch, but it's still pre-training because the objective is just next token prediction of the text you expose it to.
Post-training means everything else: SFT, DPO, RL, etc. Anything that involves things like prompt/response pairs, reward models, or benefits from human feedback of any kind.
> Sometimes a law is just on its face and unjust in its application. For instance, I have been arrested on a charge of parading without a permit. Now, there is nothing wrong in having an ordinance which requires a permit for a parade. But such an ordinance becomes unjust when it is used to maintain segregation and to deny citizens the First-Amendment privilege of peaceful assembly and protest.
> I hope you are able to see the distinction I am trying to point out. In no sense do I advocate evading or defying the law, as would the rabid segregationist. That would lead to anarchy. One who breaks an unjust law must do so openly, lovingly, and with a willingness to accept the penalty. I submit that an individual who breaks a law that conscience tells him is unjust, and who willingly accepts the penalty of imprisonment in order to arouse the conscience of the community over its injustice, is in reality expressing the highest respect for law.
I always have to go back to read this part again because I feel like it's so unexpected. You don't really hear anyone saying quite the same thing today.
Writing a calculus book that's more rigorous than typical books is hard because if you go too hard, people will say that you've written a real analysis book and the point of calculus is to introduce certain concepts without going full analysis. This book seems to have at least avoided the trap of trying to be too rigorous about the concept of convergence and spending more time on introducing vocabulary to talk about functions and talking about intersections with linear algebra.
I recommend reading Amusing Ourselves to Death. The Lincoln-Douglas debates consisted of a series of hour+ long speeches by each candidate, all in a single sitting, and people went to them and listened.
Post-training means everything else: SFT, DPO, RL, etc. Anything that involves things like prompt/response pairs, reward models, or benefits from human feedback of any kind.