HackerTrans
TopNewTrendsCommentsPastAskShowJobs

ebalit

no profile record

Submissions

Self Forcing: Real-time streaming video generation on a single RTX 4090

self-forcing.github.io
4 points·by ebalit·el año pasado·0 comments

Fixing the Ransac Stopping Criterion

arxiv.org
3 points·by ebalit·el año pasado·0 comments

Predicted outputs: GPT-4o inference speed-up for editing tasks

platform.openai.com
3 points·by ebalit·hace 2 años·0 comments

AMD Open-Source 1B OLMo Language Models

amd.com
78 points·by ebalit·hace 2 años·33 comments

INT8 FlashAttention

arxiv.org
2 points·by ebalit·hace 2 años·0 comments

PyTorch Library for Running LLM on Intel CPU and GPU

github.com
308 points·by ebalit·hace 2 años·95 comments

comments

ebalit
·hace 10 meses·discuss
It seems like wero [0] is making some progress on this front without relying digital euro or on crypto.

0: https://en.wikipedia.org/wiki/Wero_(payment)
ebalit
·el año pasado·discuss
It's most probably ZenStack given the description: https://zenstack.dev/
ebalit
·el año pasado·discuss
The analogy between projects/companies and governments is missing big components though.

- "Benevolent Dictators" of companies or projects have to obey the law - They can't forbid competition or alternatives - Every participant can leave at any time - If they burn the organization to the ground, the worst case scenario is the organization get replaced and people move on

I think it shows that we're using the word "dictator" way too casually in that case.
ebalit
·el año pasado·discuss
Do you have a chat example you can share where you found it to work great? I feel that the performance varies a lot between domains.
ebalit
·el año pasado·discuss
You mentioned a model released this week. Is it Lumina new model by any chance?
ebalit
·hace 2 años·discuss
Also commits. I imagine that there is a lot of information to gather from the history of repos in addition to the "static view" of a codebase.

However, it doesn't seem trivial to do deduplication in that case without removing relevant/necessary context.
ebalit
·hace 2 años·discuss
I'm totally with you on the evolution of motor tech because of drone and also personal mobility (scooters and hoverboard motors are a steal for what they can handle).

While high torque motors got way cheaper, especially with MIT Cheetah "clones" getting easily available, they're still at least 200-500 a pop (depending on the torque needed for each articulation) from what I could find.

I might not know where to search for the real gems though. Where do you search for cheap powerful servomotors?
ebalit
·hace 2 años·discuss
I can't find any information that show a difference as large as 2x. Do you have a specific comparison point in mind?

From Nvidia and AMD, I read sparse fp8 at 7 PFLOPs for B100 [0] vs 5.22 PFLOPs for mi325x [1]

Nvidia doesn't give the dense fp8 so that's the easiest comparison I could get.

[0] https://resources.nvidia.com/en-us-blackwell-architecture [1] https://www.amd.com/en/products/accelerators/instinct/mi300/...
ebalit
·hace 2 años·discuss
But Meta is the main company behind Pytorch development. If they make it work and upstream it, this will cascade to all Pytorch users.

We don't have to imagine far, it's slowly happening. Pytorch for ROCm is getting better and better!

Then they will have to fix the split between data-center and consumer GPU for sure. From what I understand, this is on the roadmap with the convergence of both GPU lines on the UDNA architecture.
ebalit
·hace 2 años·discuss
When you already know what you building, but not when you're doing exploratory data analysis for example.

There is a good reason why the ML community took Python as the favorite language overall.
ebalit
·hace 2 años·discuss
Hi, Exxa's CTO here. Feel free to ask me anything!
ebalit
·hace 2 años·discuss
I think this might be key, in addition to some landmark tokens to quickly backtrack to. The big question is how to train such model.

There is a recent paper from Meta that propose a way to train a model to backtrack its generation to improve generation alignment [0].

[0] https://arxiv.org/html/2409.14586v1
ebalit
·hace 2 años·discuss
Transformers can also fetch at any moment any previous information that become useful.

RNN are constantly updating and overwriting their memory. It means they need to be able to predict what is going to be useful in order to store it for later.

This is a massive advantage for Transformers in interactive use cases like in ChatGPT. You give it context and ask questions in multiple turns. Which part of the context was important for a given question only becomes known later in the token sequence.

To be more precise, I should say it's an advantage of Attention-based models, because there are also hybrid models successfully mixing both approaches, like Jamba.
ebalit
·hace 2 años·discuss
There was a HackadayPrize 2023 competitor that worked on this [0]. He had to rethink the way those devices are built to bring the cost down.

That would be interesting to know if his solution could match the 4k$ in term of usability or if there is some issue like refreshing rate that make the piezo based system necessary for a good user experience.

[0] https://www.youtube.com/watch?v=BXi1tG78AW4
ebalit
·hace 2 años·discuss
He didn't say that Spanner is only a recruitment tool but that the blog posts about Spanner (and other core technologies of Google) might be.
ebalit
·hace 2 años·discuss
I'm not a lawyer but from what I understand the process is closer to what you mention for the US.

We have "garde à vue" and "détention provisoire":

- "Garde a vue" is similar to being in police custody and it's limited to 24-48 hours normally, but there is longer duration for specific crime, the maximum being 6 days for terrorism investigations [0].

- Then, a judge can decide that the defendant should stay in "détention provisoire" before a trial. It doesn't have a duration limit but should be motivated and can be re-examined multiple time [1].

0: https://fr.wikipedia.org/wiki/Garde_%C3%A0_vue_en_droit_fran... 1: https://fr.wikipedia.org/wiki/D%C3%A9tention_provisoire_en_F...
ebalit
·hace 2 años·discuss
We built the cheapest Llama 3.1 70B inference API, specialized for tasks that are not time sensitive (ie. batch processing jobs for example).

Without any quantization our current price is 30cts ingest and 50cts output per million tokens. [1]

1: https://withexxa.com/#pricing
ebalit
·hace 2 años·discuss
My question was genuine as I'm not from the gamedev domain and I might have missed the real state of the art.

Hard coded dialogs often feel very unnatural and limiting. I can see why people want to explore LLM to try to make new experiences possible.

I can see it becoming a new dimension of game design, open vs closed dialogs, like there is currently open vs closed world. And as in the open vs closed world, they will probably coexist instead of one type replacing the other.
ebalit
·hace 2 años·discuss
As a gamedev, what would you use currently instead for NPCs? And what would you say makes this/those solutions better than LLMs?
ebalit
·hace 2 años·discuss
But the price should be a factor. Your fair comparison would match a ~60k$ setup to a 20k$ according to prices we can find online.

I don't think it should be ignored, especially when the power consumption is similar.