benob · HackerLangs

21 日前議論

Longest definition and semi-columns are strong biases for right answer. Also, my run contained a lot of adjectives for which it is pretty obvious that noun definitions do not match.

benob

先月議論

It may be a clever move. By using the same models as android (contractually?), they can compete on the user experience which they typically handle better than android phone providers.

benob

先月議論

And papers on bias amplification in ML predate LLMs. I remember this specific one which was a spotlight paper at EMNLP:

Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints, Zhao et al.

https://arxiv.org/abs/1707.09457

benob

2 か月前議論

Does changing the date fix it?

benob

2 か月前議論

Deployed it to a huggingface space: https://huggingface.co/spaces/benoitfavre/needle-playground

You can check the very simple docker file there.

benob

3 か月前議論

Here is llama-bench on the same M4:

  | model                    |       size |     params | backend    | threads |            test |                  t/s |
  | ------------------------ | ---------: | ---------: | ---------- | ------: | --------------: | -------------------: |
  | qwen35 27B Q4_K_M        |  15.65 GiB |    26.90 B | BLAS,MTL   |       4 |           pp512 |         61.31 ± 0.79 |
  | qwen35 27B Q4_K_M        |  15.65 GiB |    26.90 B | BLAS,MTL   |       4 |           tg128 |          5.52 ± 0.08 |
  | qwen35moe 35B.A3B Q3_K_M |  15.45 GiB |    34.66 B | BLAS,MTL   |       4 |           pp512 |        385.54 ± 2.70 |
  | qwen35moe 35B.A3B Q3_K_M |  15.45 GiB |    34.66 B | BLAS,MTL   |       4 |           tg128 |         26.75 ± 0.02 |

So ~60 for prefill and ~5 for output on 27B and about 5x on 35B-A3B.

benob

3 か月前議論

I get ~5 tokens/s on an M4 with 32G of RAM, using:

  llama-server \
   -hf unsloth/Qwen3.6-27B-GGUF:Q4_K_M \
   --no-mmproj \
   --fit on \
   -np 1 \
   -c 65536 \
   --cache-ram 4096 -ctxcp 2 \
   --jinja \
   --temp 0.6 \
   --top-p 0.95 \
   --top-k 20 \
   --min-p 0.0 \
   --presence-penalty 0.0 \
   --repeat-penalty 1.0 \
   --reasoning on \
   --chat-template-kwargs '{"preserve_thinking": true}'

35B-A3B model is at ~25 t/s. For comparison, on an A100 (~RTX 3090 with more memory) they fare respectively at 41 t/s and 97 t/s.

I haven't tested the 27B model yet, but 35B-A3B often gets off rails after 15k-20k tokens of context. You can have it to do basic things reliably, but certainly not at the level of "frontier" models.

benob

3 か月前議論

I miss the comment tagging system: insightful, informative, interesting, funny. It would make sense for hn.

benob

3 か月前議論

Space station tracking: https://flight-viz.com/cockpit.html?lat=40.64&lon=-73.78&alt...

benob

3 か月前議論

I just realized that a hash function is nothing less than the output of a deterministic random number generator xored with some data

benob

3 か月前議論

No, the failure is the human written prompt

benob

3 か月前議論

The author emphasizes accessibility and coherence as a benefit but another interesting one is composability which does not emerge naturally in the world of UI. Create a UI for a pair of websites like a command line for grep and wc. LLMs already provide that but under the natural language interaction primitive. UI could allow for branded experiences, ad delivery and whatnot in ways that natural language doesn't.

benob

3 か月前議論

"That allows us to license the open source project under the more permissive MIT license."

benob

3 か月前議論

I would say:

- decomposition: discover a more general form of Fourrier transform to untangle the underlying factors

- memorization: some patterns are recurrent in many domains such as power low

- multitask: exploit cross-domain connections such as weather vs electricity

benob

3 か月前議論

Ollama is a user-friendly UI for LLM inference. It is powered by llama.cpp (or a fork of it) which is more power-user oriented and requires command-line wrangling. GGML is the math library behind llama.cpp and GGUF is the associated file format used for storing LLM weights.

benob

4 か月前議論

Maybe they quantized a bit too much the model parameters...

benob

4 か月前議論

This is the worst lay-people explanation of an AI component I have seen in a long time. It doesn't even seem AI generated.

benob

4 か月前議論

This reminds me of Intel talking about faster web browsing with the new Pentium

benob

4 か月前議論

The real question is when will you resort to bots for rejecting low-quality PRs, and when will contributing bots generate prompt injections to fool your bots into merging their PRs?

benob

4 か月前議論

Reminds me of "Universal pre-training by iterated random computation" https://arxiv.org/pdf/2506.20057, with bit less formal approach.

I wonder if there is a closed-form solution for those kinds of initialization methods (call them pre-training if you wish). A solution that would allow attention heads to detect a variety of diverse patterns, yet more structured than random init.

コメント