HackerLangs
TopNewTrendsCommentsPastAskShowJobs

taegee

85 karmajoined last year

Submissions

Train-Before-Test: One Simple Fix That Makes LLM Benchmark Rankings Agree

ghzhang233.github.io
2 points·by taegee·3 months ago·1 comments

comments

taegee
·19 hours ago·discuss
Screen time is fundamentally bad for a five year old regardless of the content.
taegee
·5 days ago·discuss
I tried all of them (organic maps, OSMAnd, etc.) and eventually stuck with locus maps.

https://locusmap.app

The interface is a bit cumbersome, but it got way better over time and there are some really engaged developers behind it you can always contact. It's intended for biking and hiking, but you can also use it for cars. Meanwhile, they even support android auto.

It's the type of app that grew massively over decades and that offers basically every functionality that you can imagine.
taegee
·22 days ago·discuss
I just ditched Chezmoi in favor of GNU Stow …
taegee
·last month·discuss
https://i.imgur.com/fk1aI.jpeg :)
taegee
·2 months ago·discuss
> It's one step down from a pandemic emergency (which, to be clear, has not been declared).

Well, no wonder. Ot was introduced only in June 2024.
taegee
·2 months ago·discuss
I'm sorry, but this is precisely the reason I'm using DeArrow.

https://dearrow.ajay.app/
taegee
·2 months ago·discuss
You don't need conditional probability here, as the flips are independent.

It's just p(H)p(T).

And p(H)p(T) = p(T)p(H), thus 2*p(H)p(T) = 2p(1-p).
taegee
·2 months ago·discuss
Afaics it's just basic commutativity – p(H)p(T) = p(T)p(H) – since instances are independent.

Same, of course, holds for flipping it multiple times. But there you get more than Head or Tail (binomnk(n, k)).
taegee
·2 months ago·discuss
https://i.imgur.com/eAwdKEC.png

Edit: Alternative link: https://www.cyberciti.biz/media/new/cms/2017/04/dns.jpg
taegee
·2 months ago·discuss
While I partially agree with you, there IS work being done to make the metrics comparable. Eg:

https://ghzhang233.github.io/blog/2026/03/05/train-before-te...

It just hasn't been widely adopted yet. And it might be in each of their particular interests that it continues to stay so for a while. It's basically like p-hacking.
taegee
·2 months ago·discuss
It's impossible to label AI content in general. Actually, I'm quite sick of all the "label AI" mumble. The only thing you can reliably do is to prove that someone is a human.
taegee
·2 months ago·discuss
> Why are they unmanageable?

Just look at the statistics. E. g. https://en.wikipedia.org/wiki/Lists_of_nuclear_disasters_and...

To say that they're _potentially_ safe by waving at the US Navy is a fallacy for several reasons.

1. It's p-hacking. E. g. with the same technology the Soviets destroyed five of their reactors.

2. The world of civilian operators is completely incomparable.

3. Civilian power plants use different technologies.

> Waste is mostly a solved problem.

Not as far as I know. In Germany, for example, the search for a final disposal site is still completely open-ended, and the first final disposal site will not open until 2074 at the earliest, while, at the same time, the already collapsed storage facilities consume an enormous amount of money. I personally think it is absurd to assume that an underground nuclear waste storage facility can be operated safely over geological time scales. Needless to say there isn't even a single one worldwide for highly radioactive waste.

And to compare them with coal plants is classical whataboutism. "They can't be bad, because I found something other that's bad as well."

You're right about the minable uranium. That has changed over the last years, so the current estimate is 2080 in a high demand scenario.

But your criticism about the externalized costs falls short as well. Regarding the externalized costs, that is really hard to quantify and I don't know of reliable estimations. How do you want to come up with a number if you don't even know if humans still exist on the planet at that time?

What is clear is that for nuclear energy the majority of the costs is externalized. The bulk of the costs stem from the decommissioning of power plants, final disposal, and accident-related expenses. All three are typically passed on entirely to taxpayers.

The former German vice chancellor even said, he would agree [to build a new nuclear power plant] if <political opponent> found a private operator willing to build a nuclear power plant entirely without government guarantees, subsidies, or liability coverage.
taegee
·2 months ago·discuss
Here we go again ...

Did those plants suddenly became manageable? No.

Did those plants suddenly became cheap? No.

Do we suddenly have a solution for the waste? No.

Have new uranium deposits suddenly been discovered? No.
taegee
·3 months ago·discuss
"Model A wins on MMLU. Model B wins on ARC-Challenge. Model C wins on HellaSwag.

At some point you stop trusting any of them—not because benchmarks are meaningless, but because no two of them seem to tell the same story about which model is actually better.

[…]

We found a fix. It’s called Train-before-Test."
taegee
·4 months ago·discuss
Betteridge's law of headlines -.-

https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
taegee
·5 months ago·discuss
We've already been there. -.-

https://en.wikipedia.org/wiki/Deutsche_Mathematik

I recommend translating the German version as it is much more detailed.

> "But far more important is the educational value that stems from the spiritual kinship between mathematics and the Third Reich. The fundamental disposition of both is the heroic. […] Both demand service: mathematics demands service to truth, integrity, and precision. […] Both are anti-materialistic. […] Both desire order and discipline; both combat chaos and arbitrariness."

Same for physics:

https://en.wikipedia.org/wiki/Deutsche_Physik
taegee
·5 months ago·discuss
And must have tasted like highly concentrated rat piss. XD

Even taking more than a knife point renders a complete can undrinkable. Not to speak of 30 fucking grams.

I also had to immediately think of this case.
taegee
·6 months ago·discuss
No TLDR. -.-
taegee
·6 months ago·discuss
selinux?
taegee
·6 months ago·discuss
P. S. Some people do not seem to get it: It is the medium itself that is unsuitable for children in general and completely independent of the content.