De-referencing null pointers is the ultimate tilt but our NNs needed a way to explore just a tiny bit of unmapped memory without halting a.k.a go crazy, surrender to fear etc, probably because planning and anticipating for the next threat/unknown is their main role. It seems that we have devised a clever way of offloading the exact point of no return of our pcregister to handling by fantastical powerful creatures. What if the next crop does not deliver, crash danger alert -> praise a god. What happens when we die -> establish a religion based on afterlife benefits etc.
At the same time and while this jump command is now installed/available it allowed us to explore alternative solutions (medicine, technology etc). Since one can always transfers his main program's control to GOD section, it feels less risky to process methodically information full of unknowns and failures (the context now is "with the help of god we/one can deliver a solution"). This particular section throughout history controlled our lower instincts, maybe even made us more empathetic to other (unknown) humans instead of always perceive them as a threat but ultimately controlled all aspects of life and when misused delivered bloodshed and misery. What a weird section! I think at some point we should become brave enough and refactor/remove it, it served a purpose but now is not required/obsolete.
One should also mention the creation of "God(s)" in any religion (sub)context. We are probably the only mammalian creature who delegated all his existential angst to an abstracted entity responsible for anything that it didn't make sense at the time. I think it is THE intellectual pinnacle of a brain trying to survive and process information full of null pointers without halting its programming:/)
The word human (usually) refer to all members of the Homo genus. The name Homo sapiens means 'wise man'. Except for the clever use of thumbs our genus successfully managed to apply logic consistently in order to survive. Human reasoning produced tools, technology and science.
So, no, it doesn't say it in the name. We occasionally are dumb, or make dumb comments, but the expectations are high.
Of course they hallucinate because we are training on random mode. +Since you mentioned 3blue1brown there is an excellent video on ANN interpretation based on the works of famous researchers who attempt to provide plausible explanations about how these (transformers based) archs store and retrieve information. Randomness and stochasticity is literally the most basic components which allow all these billions of parameters to represent better embedding spaces almost hilbertian in nature and barely orthogonal as training progresses.
The "emergent structures" you are mentioning are just the outcome of randomness guided by "gradiently" descending to data landscapes. There is nothing to learn by studying these frankemonsters. All these experiments have been conducted in the past (decades past) multiple times but not at this scale.
We are still missing basic theorems, not stupid papers about which tech bro payed the highest electricity bill to "train" on extremely inefficient gaming hardware.