Thanks for the insight, also for mentioning convolutional LSTM, I wasn't aware such a thing existed.
> Attention is basically a memory module, so if you don't need that it's just a waste of compute resources.
But aren't CNNs also like a memory module (ie: they memorize how leopard skin looks like)? I guess attention is a more sophisticated kind of memory, "more dynamic" so to speak.
Anyway, I'm glad to hear that a transformer architecture isn't totally stupid for my task, I will look up the literature, there seems to be a bit on this matter.
Noob question: I have some 1D conv net for financial time series prediction. Could a transformer architecture be better for this task, is it worth a try?
What scientists mean by "teleportation" is more like this: you have one particle A here, one particle B there, and by "teleportation" you make particle B have the same state as particle A.
Critical point: for each particle you want to "teleport", you have recipient particle standing by to receive it.
Or how many Russians don't understand why Ukraine for example is upset.
Is as if propaganda works. Who would have thought.
Once I was discussing with a Chinese colleague and the Tiananmen subject came up (it was in the news, I didn't bring it up), and he told me that the students were not that innocent either, that they killed 150 soldiers the night before. I asked him, how could some students with rocks kill 150 soldiers with machine guns, but he said it was true. As suspected, I couldn't find any reputable sources for this later when I did extensive googling.
We can't support 7 billion people if each one of them has a house with garden and grows it's own food. That's what the middle ages were, look at the population then.
> Attention is basically a memory module, so if you don't need that it's just a waste of compute resources.
But aren't CNNs also like a memory module (ie: they memorize how leopard skin looks like)? I guess attention is a more sophisticated kind of memory, "more dynamic" so to speak.
Anyway, I'm glad to hear that a transformer architecture isn't totally stupid for my task, I will look up the literature, there seems to be a bit on this matter.