HackerTrans
TopNewTrendsCommentsPastAskShowJobs

sudhirb

no profile record

Submissions

CTO bench: an online LLM coding benchmark

cto.new
3 points·by sudhirb·7 mesi fa·0 comments

comments

sudhirb
·4 mesi fa·discuss
Is this not the good kind of problem to have? Subnautica 2 is doing so well that the developers get their earn-out bonus?

Seems like pure profit-maximizing/greed!
sudhirb
·4 mesi fa·discuss
I can hardly believe my eyes! I helped do some related research specifically concerning thin-film drainage from tubes, way back in my undergraduate days: https://doi.org/10.1016/j.expthermflusci.2018.04.015
sudhirb
·5 mesi fa·discuss
a 90% saving is huge isn't it?

for long agent sessions, I would expect a very high cache hit rate unless you're editing the system prompt, tools, or history between turns, or some turns take longer than the cache timeout
sudhirb
·5 mesi fa·discuss
So long as perceived LLM skill is still "spiky" - e.g. within a domain, still showing relatively high variation in ability (often depending on the task or user, to be fair), people will continue to dismiss it
sudhirb
·5 mesi fa·discuss
In the general case I think this is a great idea - if we do a good job of documenting intent etc. in commit messages, agents have an easier time understanding why lines of code exist with no additional specs/mechanisms/etc.

Interested to see what techniques in this area pull ahead and gain traction!
sudhirb
·6 mesi fa·discuss
Coding agents are such a congested space right now that to me this mostly reads as an advertisement.
sudhirb
·6 mesi fa·discuss
I have 150Mb/s FTTP for £37/month - upgrading to gigabit would be £75/month, for example!
sudhirb
·6 mesi fa·discuss
Interesting that non-salty water didn't make the string conductive(?) enough - I'd have thought that there might have been enough soluble stuff in string.

Also I believe this person runs the ISP I use (and I couldn't speak more highly of it - Andrews & Arnold).
sudhirb
·6 mesi fa·discuss
For me, a lot of the draw is that it's cheaper than managed db services for small/toy projects of mine (that I don't want to use dynamo db for) - that and in a previous job it was useful as relatively temporary multi-tenant storage.
sudhirb
·7 mesi fa·discuss
The partner for these projects has a benchmark that the top frontier LLM labs seem to be running on their new model releases - I think there's _some_ value to these numbers in helping people compare and contrast model performance.

https://andonlabs.com/evals/vending-bench
sudhirb
·9 mesi fa·discuss
I think that "manipulate people for financial and political gain" is an outcome of what social media companies actually do - I was under the belief that in a general sense, they want to maximise the time people spend on their apps so that they can sell this attention to advertisers, independent of whether or not a given ad buyer wants to manipulate people.
sudhirb
·9 mesi fa·discuss
Some mild whataboutery: is the purpose of a cancer ward to fail to cure a large fraction of its patients[0]?

https://www.astralcodexten.com/p/come-on-obviously-the-purpo...
sudhirb
·10 mesi fa·discuss
Both miracles are illness-recovery related and feel to me quite like regression to the mean, but I can imagine this is somewhat of a strategic move from the Catholic church to bring some relatability into things.
sudhirb
·10 mesi fa·discuss
For me, the USP Warp used to have was generating shell commands from prompts inside the terminal - but Cursor has had this in its embedded terminal for a while now so increasingly I find myself using Ghostty instead