HackerTrans
トップ新着トレンドコメント過去質問紹介求人

joemoon

no profile record

コメント

joemoon
·2 か月前·議論
Spidey senses going off here. The first two comments read like an LLM.
joemoon
·2 年前·議論
Thank you, this was a very interesting interview.
joemoon
·2 年前·議論
While the post didn't particularly speak to me, I think you are being overly reductionist. You can dismissively summarize anything into a few "key points."

Just in the first couple of paragraphs alone:

- The author discusses the "why" by pointing to research on how new information triggers dopamine pathways. Maybe this is obvious to you but it's not necessarily obvious to everyone.

- The author points out that this would have historically been an evolutionary advantage (at least that's my interpretation), but access to modern high-volume, low-nutrient information has made this an addictive unhealthy habit.

- For those already familiar with how modern access to high-calorie, low-nutrient food (that triggers the dopamine response), the author is showing that the same mechanisms are at play with information.

I think it's actually your comment that contains the tweet-able one liners (as "key points") that are likely to result in head nods and smug self assurances but aren't doing anything to encourage deeper thought.
joemoon
·3 年前·議論
It's definitely a very similar method but fundamentally different in that the 'Distilling step-by-step' approach is a multi-task model.

As I understand it, rather that training the smaller model to produce the CoT/rationale as a part of the (decoded) response, it actually has two output layers. One output is for the label and the other is for the rationale. The other layers are shared, which is how/why the model is able to have an improved "understanding" of which nuances matter in the labeling task.