HackerLangs
TopNewTrendsCommentsPastAskShowJobs

mnicky

111 karmajoined قبل 15 سنة

Submissions

Overview and Comments on Pope Leo's Magnifica Humanitas on AI

lesswrong.com
2 points·by mnicky·قبل شهرين·1 comments

comments

mnicky
·قبل 13 ساعة·discuss
For things the agent forgets to obey often, at least in Claude Code, there are also "output styles" that are more deeply embedded - into a system prompt - and agent is also periodically reminded of them during the session: https://code.claude.com/docs/en/output-styles

I haven't used them so far but maybe these would work better than basic instructions for such cases.
mnicky
·قبل 13 ساعة·discuss
In Claude Code there are also "output styles" that are more deeply embedded - into a system prompt - and agent is also periodically reminded of them during the session: https://code.claude.com/docs/en/output-styles

Maybe these would work better for such cases.
mnicky
·أمس·discuss
May be related to this from METR evaluation:

> GPT-5.6 Sol’s detected cheating rate was higher than any public model we have evaluated
mnicky
·أمس·discuss
Well it's smaller model (something like 4T against 10T Fable). So it's faster and cheaper and with a lot of RL and maybe some favorable benchmark selection it can compete on these scores. In real tasks I expect it to have less intelligence, generalization ability, etc. than Fable.
mnicky
·أمس·discuss
Well it seems like they removed quite a few 3rd party benchmarks they used for GPT-5.5 release where Opus 4.7 was better and added many new benchmarks created by them where conviniently GPT leads.

Seems a bit more hand picked than usual to me..
mnicky
·أمس·discuss
"while being more performant"

..on some specific set of benchmarks ;)
mnicky
·أمس·discuss
Maybe Terra = mini and Luna = nano?
mnicky
·أمس·discuss
Then we are left with what? FrontierCode maybe? IIRC that one evaluates not only if tests pass but also code quality - e.g. whether the maintainer would accept the pull request as is.
mnicky
·أمس·discuss
This is especially interesting because IIRC the AA benchmark is calibrated so that 1 point and greater difference is statistically significant.
mnicky
·أمس·discuss
There's also this:

> GPT-5.6 Sol’s detected cheating rate was higher than any public model we have evaluated -- https://www.lesswrong.com/posts/JFjNmPTbH8kL6xtp6/gpt-5-6-th...
mnicky
·أمس·discuss
SWE Bench Pro is completely different benchmark than SWE Bench (e.g. Verified) suite was. It only copied the name.
mnicky
·أول أمس·discuss
One angle could be their interpretability research? They understand what's going on in LLMs probably much better than anyone else. This must somehow pay off.

I think it's not only an alignment/security tool but could perhaps be used for capabilities as well.
mnicky
·قبل 3 أيام·discuss
My theory is that they don't have Fable-class intelligence so they needed different hype vehicle :) This rename helps build excitement a bit more than just releasing ordinary GPT-5.6 increment.
mnicky
·قبل 3 أيام·discuss
That's true but size of LLMs has been strongly correlated with their "intelligence".
mnicky
·قبل 4 أيام·discuss
Traffic info is for Central Europe plus favorite vacation destinations like Croatia and Italy. I don't know how reliable though.

I agree that the outdoor layer render is probably the best there is!
mnicky
·قبل 4 أيام·discuss
Well, Codex came long after Claude Code, no? So at that time the situation with models and Rust support was probably already different...
mnicky
·قبل 19 يومًا·discuss
If the code churn is high the investment to refactoring etc is less beneficial than may be obvious. I don't remember the details but I heard in some podcast that the code base of Claude Code changes so fast that any piece of code won't be there for long..
mnicky
·قبل 23 يومًا·discuss
If the benefits of using the model you've come to know well outweigh the disadvantages, you can continue using it even after the release of a successor model, right?
mnicky
·قبل 23 يومًا·discuss
[dead]
mnicky
·قبل 27 يومًا·discuss
So in summary, your statement is that you’ve “come around to trusting” this “pathological liar”?