Why do we have weird LLM sizes like 3B, 7B, 13B, 33B, 70B? Who invented those?(old.reddit.com)

16 pointsby mtmail3 tahun yang lalu7 comments

old.reddit.com

Why do we have weird LLM sizes like 3B, 7B, 13B, 33B, 70B? Who invented those?

https://old.reddit.com/r/LocalLLaMA/comments/18h9p51/eli5_why_do_we_have_weird_llm_sizes_like_3b_7b/

7 comments

superkuh3 tahun yang lalu

Heh. Those aren't the weird ones. Those are just the base model sizes for popular semi-open foundational models. The weird ones are the "merged" models where people arbitrarily combine layers from multiple models to make larger ones with more layers, like Solar-10-7B (https://huggingface.co/upstage/SOLAR-10.7B-v1.0) a 10B made from 7B Mistral fine tunes. Or the 22B "llamas".

htk3 tahun yang lalu

Anyone has any suggestions on the best models to run on a M3 Max with 128GB Ram?

htk3 tahun yang lalu

Thanks for the tip, will play with Mixtral.

superkuh3 tahun yang lalu

Probably the Mixtral-8x7B.

m3kw93 tahun yang lalu

mixtral-8x7b-v0.1.Q8_0.gguf

m3kw93 tahun yang lalu

What determines the number of parameters? How do you say we need 3b here and 7 there?

mistrial93 tahun yang lalu

> What are LLM Parameters? LLM Parameters are settings that you can adjust to control how the LLM generates texts. They can affect the quality, diversity, and creativity of the generated texts. Some of the common LLM parameters are temperature, number of tokens, top-p, presence penalty, and frequency penalty.

In this post, I will explain what these parameters are and how to use them effectively. I will also show you some examples of texts that can be generated by LLMs with different parameter values.

https://michaelehab.medium.com/the-secrets-of-large-language...