So the "E2B" and "E4B" models are actually 5B and 8B parameters. Are we really going to start referring to the "effective" parameter count of dense models by not including the embeddings?
These models are impressive but this is incredibly misleading. You need to load the embeddings in memory along with the rest of the model so it makes no sense o exclude them from the parameter count. This is why it actually takes 5GB of RAM to run the "2B" model with 4-bit quantization according to Unsloth (when I first saw that I knew something was up).
They had to put together ~14 hours of visuals, more than 5x what U2 had to do. There's nothing low effort about that. Some visuals were not as good as others as a result but I can promise you that the audience had their jaws on the floor all four nights.
You're just being pedantic and grasping at straws. Whether or not we're using the scientific term for an individual's mindset (which is not quantifiable to a high degree of precision but that doesn't meant it's not quantifiable at all, you can at least reduce most 'measurements' to a binary range) is irrelevant to its significance. The comment you replied to specifically said it would be hard to do a rigorous study of.
It's a medical journal, Trends in Immunology, the same type of journal that all medical studies are published in. Unfortunately we still have a broken publishing system in the US so it is insanely expensive to read articles/studies published in these journals unless you work somewhere with an institution-level account.
That sounds like a different planet to me. I don't know of any big tech company, let alone startups, that drug test. I wouldn't be surprised if this was the case for dev jobs in older industries but it doesn't seem to be the norm any more.
I believe it's reconstructed from the Latex source, which is how every paper is submitted to the ArXiv. Not to diminish this site but I'm guessing that generating HTML from Latex is a lot easier than doing it from PDF format.
These models are impressive but this is incredibly misleading. You need to load the embeddings in memory along with the rest of the model so it makes no sense o exclude them from the parameter count. This is why it actually takes 5GB of RAM to run the "2B" model with 4-bit quantization according to Unsloth (when I first saw that I knew something was up).