cshimmin · HackerTrans

24 hari yang laludiscuss

Interesting, I wonder if the rugby thing is a common bias. I did find myself in the weights, as the top result. But apparently there are also Australian rugby versions of me!

cshimmin

bulan laludiscuss

That is simply not true. The naive “glorified auto-complete / stochastic parrot” argument may have some merit when applied to generic pre-trained models, which only learn from unsupervised next-token prediction. But the post training through reinforcement learning the frontier models undergo is very sophisticated and they genuinely learn to do novel things that are purely the work of the model being trained (and the work of the GPUs they burn along the way of course).

cshimmin

2 bulan yang laludiscuss

https://xcancel.com/pj_lambert/status/2057477629528150369

cshimmin

2 bulan yang laludiscuss

It's on the page, if you click the little info icon in the upper-right. Here's the text but there's some nice graphics there too:

  Snake Game, training entirely in the browser. Built on tinygrad: the rollout / targets / train graphs are TinyJits authored in Python, then compiled once to WGSL and replayed here under WebGPU.

  Observation: flat 10×10 board (100) + 4-dim prev-action one-hot = 104 dims. fc_pi.weight is zero-init so the opening policy is uniform over the legal actions; fc_v uses tinygrad's default Kaiming init.

  Per rollout: T=24 × N=384 parallel snakes (9,216 transitions), then K=3 epochs × 4 mini-batches of PPO updates. GAE γ=0.99, λ=0.95; AdamW wd=0.01; ratio clip ε=0.1; grad-norm 0.5; Huber value β=1, val_coef=1; entropy bonus 0.008333333333333333.

  Action mask + value clip + KL early stop. The 4-dim prev_a obs tail lets fc_pi zero the U-turn logit (the env silently overrides same-axis reversals anyway). Value loss is max(huber(v_new−td), huber(v_clip−td)) at ε=0.2. Approx-KL is sampled after each epoch and breaks the loop at 1.5·kl_target.

cshimmin

2 bulan yang laludiscuss

Very helpful! Naïve question (I haven’t had a chance to read TFA at all and diffusion/flow models are not my area of expertise). Doesn’t learning the integral/solution of the diffusion process in a single pass just take us back to like OG generative CNN that we had before diffusion models took over? Surely the answer is “no” but would love to hear your framing as to why.

cshimmin

3 bulan yang laludiscuss

Not gonna lie, the first thing I noticed was that the first author was in an anesthesiology department. Your guidelines for sniff-testing are not unreasonable, and can definitely be helpful to people who are unfamiliar with the research area. But I quite intentionally did not appeal to any of those. As a (somewhat) subject matter expert, it's important to _ignore_ things like ad hominem judgement, and instead address the paper on its self-contained merits. And more importantly, to share my assessment of those with the lay public.

cshimmin

3 bulan yang laludiscuss

When I was a research physicist I spent a lot of time looking at the effects of ionizing radiation in pictures, although mostly in the context of digital images. The mechanisms are a bit different for photo emulsions, but to me the reason I'd discount radiation is because they're specifically filtering for features that exhibit the expected point spread function (which is a geometric property of the telescope's optical assembly itself). I guess you could test by exposing emulsion plates to ionizing radiation and seeing how often you get PSF-like images by chance. Also, their search is for +/- 1 day of nuclear testing, which seems weird. Certainly radiation from fallout wouldn't make sense on the day before testing. It would have been useful to see +1 day and -1 day separately. Or 0-2 days. The way it's chosen makes me suspect they couldn't find a signal in those windows, and therefore it's probably just statistical noise that they've massaged out of the data.

But to me the biggest flag is that these images are from 50 minute exposures. The objects don't appear as streaks, so they are either very, very short flashes (much shorter than 50 min), or they are very far away. The authors interpret this to mean the objects should be in geosynchronous orbit, which doesn't make sense; objects in geosync would still appear to move relative to the star background over the course of 50 min. Yet this is the entire basis for their "shadow deficit" window calculation. You could constrain the duration vs distance by looking at the effect it would have on smearing the PSF, which would be interesting.

Overall it seems pretty unscientific. If you go looking through enough statistically noisy data for signals in enough places, you'll eventually find it.

cshimmin

3 bulan yang laludiscuss

It kinda sounds like a post-doc, in that it provides an on-ramp to working in the industry/institution. But without having to waste your time getting a PhD.

cshimmin

3 bulan yang laludiscuss

The 6 is part of 3.6, the model version. 35B parameters, A3B means it's a mixture of experts model with only 3B parameters active in any forward pass.

cshimmin

3 bulan yang laludiscuss

Incidentally, I recently learned the origin of the term. Cyber - short for cybernetic - is from the greek κυβερνήτης (kybernetes), meaning helmsman. The original use of cybernetics is in the context of automated control systems, so steering a rudder was a good analogy. It is also the origin for the name k8s.

cshimmin

4 bulan yang laludiscuss

Yeah perhaps a better term for Loser is Abstainer. Because the Sociopaths also can certainly lose at the game of maximum capitalist profit. Loser/Abstainer just chooses not to play the game.

cshimmin

7 bulan yang laludiscuss

It's basically just a way for the LLM to lazy-load curated information, tools, and scripts into context. The benefit of making it a "standard" is that future generations of LLMs will be trained on this pattern specifically, and will get quite good at it.

cshimmin

9 bulan yang laludiscuss

Was this after you and two zany friends made a scheme to divert a fraction of a penny from each of your employer's transactions into a bank account that you control? And then you gave it all back but the building burned down and Milton made off with the cash?

cshimmin

tahun laludiscuss

If I understood correctly, the global lock is so that notify events are emitted in order. Would it make sense to have a variant that doesn't make this ordering guarantee if you don't care about it, so that you can "notify" within transactions without locking the whole thing?

cshimmin

4 tahun yang laludiscuss

No comment about Chesterton's Fence but I have heard (but never confirmed!) one reason for left-handed stigma is to do with hygiene. The idea is that before modern hygienic standards (e.g. sinks with soap in every bathroom/kitchen), the left hand was reserved for "dirty work" (we are also considering a time before toilet paper...). So for example when you reach out to shake someone's hand, it would be rude to use the left.

cshimmin

4 tahun yang laludiscuss

This seems to be to opposite of the example GP was seeking. Unless you mean to say that a great many people (both in Brazil and internationally) were already practicing Capoeira before it was legalized...

cshimmin

11 tahun yang laludiscuss

I <3 github... anyone with more business sense than me care to speculate on what this means for us users?

cshimmin

12 tahun yang laludiscuss

That's an interesting approach; it would cause comments that are really good to float to the top no matter when they are posted. In terms of Bayes' theorem:

    P(upvote | good post) = P(good post | upvote) * P(upvote) / P(good post)

Let's neglect that opinions vary, and assume P(good post | upvote) = 1. P(upvote) should be proportional to P(post is read); but the readership at the bottom of the page (or wherever latecomers are dropped) is quite small.

But this begs the question: what does it mean to "float to the top" if highly-rated comments are being placed in a random location?