Show HN: Proxy Block-CAGE, a new sparse block attention(github.com)

2 pointsby iqbal198019 ngày trước0 comments

Show HN: Proxy Block-CAGE, a new sparse block attention

https://github.com/iqbal1980/ai_research_assistant_rnd_projects/blob/main/proxy_block_cage_attention_introduction.md

Hi, I'm a PhD student in Bioinformatics/Computational Biology with a software engineering background,

I'm trying to pivot toward AI/ML research. I'm familiar with the practical side of AI as in using, Scikit learn, R, Pytorch, ONNX Runtime etc...

I was thinking if LLMs could be used as a research assistant to create better AIML algorithms. So I asked ChatGPT to help find better way to solved one of the most computationally intensive problems in Transformer architecture based model. I instructed ChatGPT to use genetic algorithms, genetic programming and other optimization techniques (Something I use extensively in my bioinformatics research) to find better Attention methods in transformers and this was the result. I would love to get feedback and comments from the AIML research community. Please read the md file and get back to me.

Hope you find this useful.

Thank you

0 comments

—