Every time I see a photo of Minister of Foreign Affairs of Iran, Abbas Araghchi, I instantly think I am looking at a picture of author Nicholas Nassim Talib (Black Swan, Fooled by Randomness).
Consider the classic ambiguity example from Natural Language Processing: "Time flies like an arrow; fruit flies like a banana".
Markov chains are mostly about memorized local co-occurrence, while transformers are much better at context-sensitive disambiguation and compositional structure.
I do not have deep knowledge about these subjects but this is what I think was meant by "encoding of meaning" (LLMs). I also do not think this capability has to be thought of as "reasoning".
A modern version of the idea encapsulated in posters and bumper stickers that read: "It will be a great day when our schools get all the money they need and the Air Force has to hold a bake sale to buy a bomber"