augment_me

10 dni temudiscuss

Peer review WAS vital for a long time. Maybe the world looks different now, maybe LLMs can find value in things better than humans. When you make an assumption it's good to think about why you do so, in this case it seems to be for historical reasons.

augment_me

10 dni temudiscuss

You are then our of the normal probability distribution and out of luck, it's not profitable to cater for you for the company.

augment_me

11 dni temudiscuss

My comment is aimed to highlight that the "GPU Bubble" is frames as a general solution when it's not, its a specific bottleneck based on your model size. Your dont mention your model size anywhere, the reader has to infer it from the runtimes, and if they dont know the average forward pass of a model, well too bad, they will leave without understanding the actual trade-off.

The benchmarks you point to in the section titled "A cost model for the bubble" dont include any CPU overheads or the T_block-T_pipe you mention, they just give the improvement %.

In general, you answers here in the thread read as defensive and unhumble. They leave a sour taste of your company, you should consider how you engage with your audience.

augment_me

11 dni temudiscuss

As someone who works in the field, the blog is nice but it has a lot of CODEX fingerprints on it, and it's also very specific to the size of the model in question in a way that is not explicit from the blog until the very last section.

In general, for some reason CODEX loves CUDA-streams, it's the first optimization it goes for every time when writing GPU kernels. However in many cases this is not a bottleneck, it happens to be so here because the model in the blog is small (2.4ms FW-pass is tiny, and 9B params sit on a single GPU). Large models are closer to 30-40ms. The CPU-GPU sync is 1-2ms, when working on larger MoE models the scheduling of tokens in this way is much less important than for example scheduling of computation/communication or kernel optimization.

I wish the blog would state this at the start with the premise of what has been done, or show that this is indeed the bottleneck with some benchmarking. Otherwise is kind of overselling things imo.

augment_me

12 dni temudiscuss

Whether brains do gradient descent is irrelevant to a CFO deciding whether to staff one radiologist or three. The market doesn't care if the model "identifies" versus "returns a statistical value", it cares what the error rate costs versus what the salary costs, and it prices liability into contracts either way, same as it already does with autopilot and compliance software.

Also disagreement among human radiologists has been documented for decades, so the clean expert baseline you're defending doesn't actually exist outside this argument.

When the identifiers pass human-detection-rate percentages, it will most likely be cheaper to hire a fall-guy for the liability with a much smaller salary, I think this will be a big market in fact.

augment_me

13 dni temudiscuss

You could also view it from the perspective of that if every other major superpower has their mass surveillance and you don't, it becomes an assymetrical informational situation where foreign governments can influence your citizens, but you cannot influence the foreign citizens since they are surveilled and their informational diet is restricted.

In some sense Chat Control is a geopolitical necessity for the EU, there is no choice to not do it.

augment_me

15 dni temudiscuss

If you have a confounding variable or a dependency that influences the experiment to a degree that invalidates the premise of the experiment, you need to put more weight on this in the conclusion.

For me this reads a bit like if I added an AI software that scans for shoplifters, and then placed a security guard at the exit of the store that watches the people shopping at the same time, and then said that the AI software is responsible for the reduction of the shoplifting without accounting for the influence of the guard.

If you have place the model in the embedding space of 99% negative samples, it's doing the same thing, the initial premise of the experiment is not valid.

augment_me

15 dni temudiscuss

LLMs produce about 95% of the code at my company and review about 70% of it for 3 years now. Our team has downsized from 40 to 8 people in this time. My creative labor is spent writing harnesses and wrappers. When there is enough of a data distribution on this, the LLMs will be able to do that as well.

I have saved up a buffer in funds and bonds because it's going to be over at some point when the company moves from explore to exploit.

augment_me

15 dni temudiscuss

Both were noted, but then the conclusion drawn from these things is that the author is considerably more optimistic about the agents. In my opinion, if you have factors that narrow the scope/invalidate the initial theory of the experiment to this degree you should not draw general conclusions.

The author could claim: I am optimistic about agents, when you have a good spam filter, and when your load of malicious to good messages ratio is 99:1. This is quite different from a common scenario where this would be used.

augment_me

15 dni temudiscuss

Who cares if the consumer buys it and uses it? Information is worth nothing anymore, attention is, so if they manage to capture a larger audience somehow, they win.

augment_me

15 dni temudiscuss

1) Googles spam filter removed a lot of the attempts as you say yourself. 2) Model was tested under unrealistic conditions where 99% of the inputs are malicious, so the model is expecting to get hacked and is already in the cautious part of the embedding space.

I know it's hard to account for everything, but in my opinion this mostly showed that the first 3 attempts were unsuccessful.

augment_me

16 dni temudiscuss

Which is interesting given the Mojo blogs where they shit on the other pythonic eDSLs like Triton saying that it's a dead end

augment_me

19 dni temudiscuss

Bitter truth :(

augment_me

19 dni temudiscuss

Most people use social media such as discord or whatsapp in order to make social activities and communities simple with the majority of their friends. A majority of people do not give a shit about integrity. The only group I have ever managed to convince and actually use Signal for messaging out of all my groups of friends and peers is the Computer Science Dept PhD students.

For most people, it's not realistic to give up their social bonds, they are too far in. If you are hoping for some revolution or change in this aspect is way too late. You can have small fringe groups engaging in this, but at the end of the day you are overestimating how many people actually give a shit.

augment_me

w zeszłym miesiącudiscuss

> The correct solution is just to not attend it if you know you aren't requested to participate and are just here to grow the numbers and make your company waste money.

If this argument actually worked in practice, the world would be a better place

augment_me

w zeszłym miesiącudiscuss

I feel like the whole blog and the point can be reversed. If your bottlenecks are meetings and emails, and you make an agent take notes and summarize things for you, you gained focus to work on what you find meaningful.

> He explains that this happens because knowledge work often relies on “pseudo productivity,” where visible busyness is treated as a proxy for real value. Digital tools reinforce this by making people look active: sending more messages, producing more drafts, attending more meetings, and generating more work artifacts. To avoid the trap, he recommends measuring real outcomes, identifying the true bottlenecks in one’s work, and separating deep work from shallow work so that digital tools support meaningful progress instead of consuming attention.

---

Like, you are just as well make the argument that if you replace the pseudo-work, you end up with 8 hours of deep work for things that bring you value.

augment_me

2 miesiące temudiscuss

TLDR:

Authors realize that global row-wise dependent functions like RMSNorm/LayerNorm have baked-in scales that are commutative in certain setups, so they can be moved out after a subsequent projection and be partially aggregated on tiles of rows.

So ((W1 @ gamma * globally_computed_scale) * W2 can be written as (W1 @ gamma * W2) * globally_computed_scale as long as we have row-only interactions for the scale.

This was usually not done before because left-to-right graph compilers like torch.compile can't assume that a global row-wise reduction between GEMMs can be commutative.

augment_me

2 miesiące temudiscuss

And the point of the comment you are answering is that the market you are talking about has taken on a different form.

The difference is that normally, people would perhaps pay a company to buy a PS license, or pay a professional to edit their images, however now this market segment will just use VLMs to edit the image to what they need.

augment_me

2 miesiące temudiscuss

I shop for things with AI as well. For example for haircare, or skincare, there is no way to figure out what ingredients are fine in various products. I pulled down 600 shampoos, prices and their ingredients, and made the AI choose which one based on my hair type and what I want.

I have another pipeline that pulls down all the groceries from stores every week in a 3km-radius and then builds cheap, healthy recipes from them, then orders the things I need by how the stores are laid out.

In general I spend about 65% of what I used to, so I think that the incentives for consumers are there.

augment_me

2 miesiące temudiscuss

I think that if this becomes an actual problem, there will be such a massive incentive to add AI to the scale/compression/risk avoidance side that there will be automated tools specialized in that kind of work.

I feel like this is shooting from the hip from a single point of view from some semi-large corpo.

comments