HackerTrans
トップ新着トレンドコメント過去質問紹介求人

agunapal

no profile record

投稿

Most teams optimize the prompt. Agentic systems have more moving parts

aevyra.ai
3 ポイント·投稿者 agunapal·2 か月前·0 コメント

Show HN: Verdict – model evals on your own data, not someone else's benchmark

github.com
2 ポイント·投稿者 agunapal·2 か月前·0 コメント

コメント

agunapal
·2 か月前·議論
My first thought after reading the blog was, let me share the blog with Claude and ask it how bots can circumvent this.

imo AI bots have significantly affected OSS and we need better qualitative measures to define success
agunapal
·2 か月前·議論
Yes, one can easily setup agents to bump up the stars, increase pip downloads etc
agunapal
·2 か月前·議論
I think it comes down to "Is the juice worth the squeeze"

As someone who worked for a large organization maintaining an OSS project, one issue I faced was how do you show impact? We used to have many organizations really love and use our project , but they would hardly give anything back to the project, including writing blogs where they could have shared some success stories. IMO github stars/pip downloads etc are not good metrics and these are even worser metrics in today's agentic AI world. Its so easy to fake these nowdays.
agunapal
·2 か月前·議論
Here is a paper from few years ago where they talk about 7x speed increase, which equates to savings.

https://arxiv.org/abs/2101.03961
agunapal
·2 か月前·議論
Do you mean model sharding?
agunapal
·2 か月前·議論
Very competitive price for the speed and intelligence being offered!
agunapal
·2 か月前·議論
Nvidia had the first movers advantage. Nvidia spent so many years perfecting CUDA to work well with PyTorch. Before ROCM, there was only CUDA. There were so many developers building their use cases on top of PyTorch+CUDA, and bringing all that feedback to PyTorch, this made CUDA battle ready and stable. AMD can get there, especially now with demand for compute, but as someone already said here, the biggest focus needs to be on PyTorch
agunapal
·2 か月前·議論
If you really think about why MoE came into existence, its to save significant cost during training, I don't think there was any concrete evidence of performance gains for comparable MoE vs dense models. Over the years, I believe all the new techniques being employed in post training have made the models better.