marci

12 วันที่ผ่านมาdiscuss

This was a preview release. They haven't finish training. The Pro contains more knowledge but it probably takes longer training than flash for the smarts to kick in.

marci

15 วันที่ผ่านมาdiscuss

  "That’s where EMO comes in.

  We show that EMO – a 1B-active, 14B-total-parameter (8-expert active, 128-expert total) MoE trained on 1 trillion tokens – supports selective expert use: for a given task or domain, we can use only a small subset of experts (just 12.5% of total experts) while retaining near full-model performance."

https://allenai.org/blog/emo

marci

15 วันที่ผ่านมาdiscuss

But with Apple's AFM 3 architecture, we might end up with huge SOTA adjacent on devices with limited RAM.

They use a technique where you only load between 1B and 4B of a 20B dense model for an entire prompt run, not token by token like a MoE, and use mostly the low power ANE instead of GPU cores.

Now, imagine if/when they scale up to 100B or more? On a chip using 2W?

marci

26 วันที่ผ่านมาdiscuss

Don't worry. Most people spend most of their compute time on a phone, where you're ability to filter ads is way more enshitified.

marci

2 เดือนที่ผ่านมาdiscuss

I wonder where's the line between using a font and copyright/trademark infringement.

marci

2 เดือนที่ผ่านมาdiscuss

"That’s where EMO comes in.

We show that EMO – a 1B-active, 14B-total-parameter (8-expert active, 128-expert total) MoE trained on 1 trillion tokens – supports selective expert use: for a given task or domain, we can use only a small subset of experts (just 12.5% of total experts) while retaining near full-model performance."

https://allenai.org/blog/emo

marci

2 เดือนที่ผ่านมาdiscuss

Did they modify their post? I can't see who claimed that consumer hardware will be able to build most things?

marci

3 เดือนที่ผ่านมาdiscuss

when you sign an app with your personal dev account.

marci

3 เดือนที่ผ่านมาdiscuss

That's just a regular rounndabout.

I thought you were talking about this: https://www.youtube.com/watch?v=6OGvj7GZSIo

marci

4 เดือนที่ผ่านมาdiscuss

Also everything from scratch by allen.ai.

Weights, datasets, code, multiple checkpoints...

I like their FlexOlmo concept.

marci

4 เดือนที่ผ่านมาdiscuss

I think it's mostly because most cpus that can run a gpu already have parts dedicated as h264 encoder, way more efficient energy wise and speed wise.

marci

4 เดือนที่ผ่านมาdiscuss

Unfortunately, the most extreme is that it's the new normal that now, there's >0 chance that someone, whether they are a US citizen or not apparently, child or adult, can end up in a camp, with no due process.

marci

5 เดือนที่ผ่านมาdiscuss

Imagine, a llm trained on the best thrillers, spy stories, politics, history, manipulation techniques, psychology, sociology, sci-fi... I wonder where it got the idea for deception?

marci

5 เดือนที่ผ่านมาdiscuss

Their issue with the mac was the sound of fans spinning. I doubt a dedicated gpu will resolved that.

marci

6 เดือนที่ผ่านมาdiscuss

"finetune"

Not

"Train from scratch"

marci

6 เดือนที่ผ่านมาdiscuss

It is so, so long... I barely reached the middle before my brain just "Nope."

They are talking about this kind of battery replacement: https://www.ifixit.com/Guide/Fairphone+3+Battery+Replacement... . The "TV remote" battery replacement kind.

marci

6 เดือนที่ผ่านมาdiscuss

If your family members ever had to mount an ikea furniture or equivalent, they'll probably have an as easy or easier time replacing a part on a fairphone. Especially for the battery. At least for version 3 and older. I don't know for later models. If you know how to swap batteries in a tv remote, you know how on this phone.

marci

6 เดือนที่ผ่านมาdiscuss

This is the draft, not the current version.

edit: and for the current unfortunately there's only a dead dropbox link.

marci

6 เดือนที่ผ่านมาdiscuss

When they bothered compensating, it was far from market value, so no.

marci

6 เดือนที่ผ่านมาdiscuss

It makes more sense to word it like this when you take into consideration historical trends, like drowned towns for lakes or dams, highway system along redline, thriving neighbourhoods erased to create parks… often preceded by violence and little to no compensation.

Submissions

[untitled]

comments