HackerTrans
TopNewTrendsCommentsPastAskShowJobs

agunapal

no profile record

Submissions

Most teams optimize the prompt. Agentic systems have more moving parts

aevyra.ai
3 points·by agunapal·2 เดือนที่ผ่านมา·0 comments

Show HN: Verdict – model evals on your own data, not someone else's benchmark

github.com
2 points·by agunapal·2 เดือนที่ผ่านมา·0 comments

comments

agunapal
·2 เดือนที่ผ่านมา·discuss
My first thought after reading the blog was, let me share the blog with Claude and ask it how bots can circumvent this.

imo AI bots have significantly affected OSS and we need better qualitative measures to define success
agunapal
·2 เดือนที่ผ่านมา·discuss
Yes, one can easily setup agents to bump up the stars, increase pip downloads etc
agunapal
·2 เดือนที่ผ่านมา·discuss
I think it comes down to "Is the juice worth the squeeze"

As someone who worked for a large organization maintaining an OSS project, one issue I faced was how do you show impact? We used to have many organizations really love and use our project , but they would hardly give anything back to the project, including writing blogs where they could have shared some success stories. IMO github stars/pip downloads etc are not good metrics and these are even worser metrics in today's agentic AI world. Its so easy to fake these nowdays.
agunapal
·2 เดือนที่ผ่านมา·discuss
Here is a paper from few years ago where they talk about 7x speed increase, which equates to savings.

https://arxiv.org/abs/2101.03961
agunapal
·2 เดือนที่ผ่านมา·discuss
Do you mean model sharding?
agunapal
·2 เดือนที่ผ่านมา·discuss
Very competitive price for the speed and intelligence being offered!
agunapal
·2 เดือนที่ผ่านมา·discuss
Nvidia had the first movers advantage. Nvidia spent so many years perfecting CUDA to work well with PyTorch. Before ROCM, there was only CUDA. There were so many developers building their use cases on top of PyTorch+CUDA, and bringing all that feedback to PyTorch, this made CUDA battle ready and stable. AMD can get there, especially now with demand for compute, but as someone already said here, the biggest focus needs to be on PyTorch
agunapal
·2 เดือนที่ผ่านมา·discuss
If you really think about why MoE came into existence, its to save significant cost during training, I don't think there was any concrete evidence of performance gains for comparable MoE vs dense models. Over the years, I believe all the new techniques being employed in post training have made the models better.