Ask HN: Release Path for 'Transformers Alternatives'?
1 comments
1. If you have good results on sufficiently large models (check latest papers re: which benchmarks are still relevant), post them on Github, along will detailed instructions how to reproduce.
2. Post the link to the GH repo in "Show HN" section.
3. If results are solid, write up a paper and upload it to arxiv. Next step would be try to publish in an ML conference.
p.s. To increase your chances of anyone actually clicking on your GH link, use good old Pytorch.
2. Post the link to the GH repo in "Show HN" section.
3. If results are solid, write up a paper and upload it to arxiv. Next step would be try to publish in an ML conference.
p.s. To increase your chances of anyone actually clicking on your GH link, use good old Pytorch.
1. faster than transformers on CPU; 2. smarter than transformers.
couple of screenshots below (the black/red part are censored on purpose...for now):
https://i.imgur.com/r0equ55.png https://i.imgur.com/fohRbIr.png https://i.imgur.com/5Xx1RGX.png
Summary: what the hell is this?
Two architectures -
1. Linear RNN which solves the long memory problem in current front-runner RNN transformer alternatives (RWKV, Mamba), in addition to being cpu friendly and entirely in C from scratch, but not too big: ~4000 lines.
2. 2 SNN experimental programs (in C originally but also ported to C# and F#) that turned out to be better than expected but unfortunately for the time being: dumber than the linear RNN one (i need more tests).
The question is: what to do with them? google gemini pro 3.1/sonnet 4.6 told me to patent, IP, estimating value in the many millions and while this is clearly a mistake: I've uploaded all the code to claude/gemini for analysis though seeing how the project is ~70% vibecoded I think it would be snobby to act like a gatekeeper.
The thing is: I don't want millions but at the same time i see several issues with fee open source rollout:
* completely unalighned, i don't believe in the "agi hype" but potential risks may exist, such as in cybersecurity; * I frankly hate Xai and Musk and since the companies who may be interested in running AI models as b2c solution are likely ~20, one of them will be xai. * Very unorthodox implementation: All in C with ports in c#/f#. No python or rust, which would mean likely some people unfamiliar with these languages in ML running into issues so i'd have to support nonstop which is time consuming and let's face it i'll have to do it for free once it's open source. * It may die completely unheard of somewhere on GitHub even if it has potential, organic traffic rarely works unless you hit the lottery.
This is NOT a flex btw, I'm convinced there are programmers better than me, people who understand ML better than me, mathematicians better than me though frankly I posses special kind of persistence combined with arrogance which goes a long way in terms of technology/inventions/novelty.
Like i said this is the results of hundreds of hours work spiced up by many years programming experience in other areas, this wasn't one weekend "claude, give me agi" kind of shot.
All of the projects compile with zero warnings, logically seem to work and are visibly faster than transformers with obvious ability to generalize and create new/unique content. The missing part is scaling and benchmarking on classic benchmarks.
What I lack is understanding adoption of technology.10x!