Pierer Invests Millions in Sepp Hochreiters xLSTM(aipod.de)
aipod.de
Pierer Invests Millions in Sepp Hochreiters xLSTM
https://aipod.de/podcast-archive/230
6 comments
Is there a professional analysis about XLSTM? It seems they are secret/stealth [1]? With a quick analysis I cannot differentiate if it is a Theranos or another OpenAI.
[1] https://github.com/AI-Guru/xlstm-resources
[1] https://github.com/AI-Guru/xlstm-resources
Disappointingly I couldn't find any specifics either. I'd hope everyone involved did their due diligence.
I don't have that hope with past "due diligence" track records. Performing a quick peek for the author at Google Patents I only see mentions of his involvement with neural networks in past work [1]. I am not an expert in patents but I assume that they will apply there and/or publish papers before the company is launched. Except if the breakthrough is not there or it is really revolutionary but statistics are not in their favor.
[1] https://patents.google.com/?q=(sepp+hochreiters)&oq=sepp+hoc...
[1] https://patents.google.com/?q=(sepp+hochreiters)&oq=sepp+hoc...
I listened to the latest about it from Prof Sepp himself on the Industrial AI podcast channel. See the link below.
He consistently called Transformers... "nonsense" and said that xLSTM has beaten Transformer-based models across several small dataset benchmarks. They are busy scaling the training to large datasets and planning to keep the technology in Europe. I doubt they will make it public once it is available. The coming months will be interesting. More about the interview here.
https://podcasts.google.com/u/1/feed/aHR0cHM6Ly9raXBvZGNhc3Q...
He consistently called Transformers... "nonsense" and said that xLSTM has beaten Transformer-based models across several small dataset benchmarks. They are busy scaling the training to large datasets and planning to keep the technology in Europe. I doubt they will make it public once it is available. The coming months will be interesting. More about the interview here.
https://podcasts.google.com/u/1/feed/aHR0cHM6Ly9raXBvZGNhc3Q...
People say lots of stupid shit. If there is no code or even paper, there is no reason to believe. Beating benchmarks requires something more than a blind faith.
---
Can xLSTM really compete with Transformers, Mamba etc.?