danbruc · HackerLangs

12 godzin temudiscuss

Why would a RTX 5090 with 32 GB not be able to deal with a 40 GB model? Is there anything preventing me from swapping the weights that do not fit into VRAM in and out of RAM? PCIe 5.0 x16 should max out around 64 GB/s, so slower than the unified memory machine, but at least it should be possible.

danbruc

14 godzin temudiscuss

Vaguely related, there is also the unfinished Cathedral of Justo [1] near Madrid built by a solo developer since 1961 and essentially by hand and more or less from scrap and whatever people donated. Justo Gallego Martínez died in 2021 at age 96 and donated the building to some foundation for completion.

[1] https://en.wikipedia.org/wiki/Cathedral_of_Justo

danbruc

przedwczorajdiscuss

Planetary Defense Officer [1]

[1] https://en.wikipedia.org/wiki/Planetary_Defense_Coordination...

danbruc

8 dni temudiscuss

What do you prefer, a plane crash or a nuclear war? But that does not matter, I did not say a airplane crashing is preferable over something, I said it is not a bad event for everyone. It is bad for the people on board and their friends and families but there are also people that benefit from a crash. That still does not mean that people benefiting from the event are in favor of more airplanes crashing. And for the majority of people it will essentially be neutral, half the planet might never even hear about this particular crash and for many more it will be a thirty second news segment they once saw.

danbruc

9 dni temudiscuss

The point of the example is that something that would generally be judged as obviously bad often turns out to not be bad from all perspectives. The same obviously also holds for good things. But I did not intend to imply any economics. The crash investigation might uncover an issue that gets fixed and prevents future crashes, also in this sense the crash had a positive effect. The better outcome in some overall sense would probably still have been for the airplane not to crash and the families spending the money on something better than funerals.

[...] some detractions are more worthy of consideration than others.

This is probably quite hard to justify in general.

I want zero cents of my money to be wasted on climate change, I live now and I want to enjoy my life as much as possible. I do not care if the planet gets burned to a crisp in a hundred years when I am dead.

One can certainly have a different perspective and vehemently disagree, especially if you have children that will have to live through that future, but otherwise? How would you argue that this position is somehow less valid than any other?

danbruc

9 dni temudiscuss

1. An airplane crashes, everyone dies. Clearly a bad thing. Not so quick. For the funeral industry this means additional business, a good thing. And this is true for a LOT [1] of things, they are not good or bad, right or wrong, their judgment depends on perspective and personal preferences.

2. Which means that there is generally no policy that makes everyone happy. So you need a party with a program that aims at finding compromises that are acceptable for everyone.

3. But nobody will vote for such a party. Why would you vote for a party that gives you 50 % of what you want if there is a different party that is more aligned with your views and preferences and promises to give you 90 % of what you want?

4. In consequence the political direction tends to hop between extremes instead of settling on compromises. One group gets really unhappy with the current situation, shows up for elections, votes their party into power, moves the situation into the direction of a different extreme, until others get unhappy enough to start the process all over again.

5. Even in political systems where [sometimes] a coalition of parties exercises the power and they are forced to compromise, the outcome is all but ideal. Things move slowly because finding compromises is hard if you do not really want to compromise. Voters look down on the party they voted for because they are not delivering what they promised but only compromises.

I guess the moral of the story is that the voters have to realize that their view is not the only valid one and that voting for compromises would probably yield better outcomes than voting for extremes and either going in that direction for some time until turning around or maybe arriving at a forced compromise that no one voted for.

[1] Exercise for the reader, find something politically relevant that does not depend on perspective and personal preferences.

danbruc

9 dni temudiscuss

What did those movies cost? More like a movie ticket or more like a Blu-ray?

danbruc

22 dni temudiscuss

Bug report. In blueprint the selection rectangle does not align with the mouse cursor except at the origin.

danbruc

26 dni temudiscuss

It was, tutorial video from 2015 [1].

[1] https://www.youtube.com/watch?v=RnX8DLmjkCA

danbruc

26 dni temudiscuss

This already existed ten years ago in the desktop version, not sure if it also was in the web version all the time.

danbruc

26 dni temudiscuss

Not only nuclear power plants have cooling towers. Here [1] is, for example, a coal-fired one in Poland.

[1] https://en.wikipedia.org/wiki/Coal-fired_power_station#/medi...

EDIT: To elaborate a bit, if you are burning oil or gas in a turbine, you do not need a cooling tower, the waste heat goes into the atmosphere with the exhaust. If you use fossil or nuclear fuels to produce steam for a steam turbine, you either need a river with enough flow to not boil all the fish if you reject the waste heat into it or you need a cooling tower to reject the heat into the atmosphere.

danbruc

29 dni temudiscuss

Died in August 2010 [1], so he was still alive when this happened in August 2006.

[1] https://www.aachen-gedenkt.de/traueranzeige/profdr-ingwalter...

danbruc

w zeszłym miesiącudiscuss

I am not a mathematician and did not read the unit distance solution too carefully, but my impression was that it used a variation of a known technique to solve the problem. And that makes perfect sense to me, there are a lot of techniques and lot of less relevant problems, I am not surprised that one can solve some of them with known techniques that just nobody has tried [hard enough] before. I am much more sceptical when it come to the important unsolved problems where every known technique has probably been tried several times over. In those instances it will probably take a true leap in understanding to solve them and I am sceptical that large language models are well suited for that because of the way they work.

danbruc

w zeszłym miesiącudiscuss

Related but not strong enough. 17 x 17 x 17 = 4,913 is 2^8-smooth - no prime factors larger than 2^8 - and it is less than 2^16, but 17 x 17 = 289 does not fit into a byte. Smoothness is required but not sufficient for a product representation to exist.

danbruc

w zeszłym miesiącudiscuss

This is just looking at it from a different perspective. Both, one 64 bit integer and the product of two 32 bit integers represent a number up to 2^64 with 64 bits. But while all 64 bit integers are unique, there are, as you say, several representation for some numbers as the product of two 32 bit integers and therefore it is impossible to represent all 64 bit integers. Commutativity alone costs you about 50 percent of all numbers in the range as x * y and y * x represent the same 64 bit number but with two different representation as a product of two 32 bit numbers, at least if x and y are different. But this tells you nothing about the numbers that you can not represent, only that they must exist. I was looking at it from this other perspective, which numbers are not representable as a product and why.

danbruc

w zeszłym miesiącudiscuss

Each x is prime with probability 1/ln(x), each x has M/x multiples less than M, as a fraction of M that is just 1/x. Together that makes 1/(x ln(x)) with the indefinite integral ln(ln(x)). If we plug in 2^32 and 2^64 [1], we get ln(2). So about 69.3 % of all 64 bit integers should have a prime factor larger than 2^32 and therefore not be the product of two 32 bit integers. That leaves about 13 % unaccounted. Three prime factors all larger than 2^32/2, five prime factors all larger than 2^32/3, and so on cannot be packed into two 32 bit integers. Not sure to how much this will add up.

[1] The bounds are important because they guarantee that there is at most one prime factor from that range and this ensures that we are not double counting anything. If the upper bound was larger than the square of the lower bound, then we would have to worry about double counting numbers with more than one large prime factor.

danbruc

w zeszłym miesiącudiscuss

No RAM. Instead of having a general purpose multiplier that multiplies an input with a weight stored in RAM, just have a multiplier that hardcodes the weight. In some sense replace each weight with a specialized multiplier and wire them together with accumulators and activation functions in between. And some registers for pipelining. If one goes for four bit quantization, one could have sixteen optimized multipliers, one for each possible weight, and the one just selects and connects them according to the model weights and structure.

Example. If you have a neuron with 16 inputs each 8 bit wide and with a 4 bit weight per input, you will have 16 specialized multipliers each scaling its input by the corresponding weight and then the 16 scaled inputs feed into an adder tree and finally an activation function.

danbruc

w zeszłym miesiącudiscuss

Did some try to estimates what it would take to bake interference for a capable large language model into silicon so that one can pipeline inputs through it and produce outputs at one token per clock cycle?

danbruc

w zeszłym miesiącudiscuss

All the primes above 2^32 are out, but that accounts for only two point something percent.

danbruc

w zeszłym miesiącudiscuss

Probably still does not work. Assume a request takes X ms and let us look at what you will observe depending on where within a tick period it arrives.

If it arrives anywhere from 0 ms to (20 - X) ms after a tick, it will complete before the next tick, so the measured duration will be between X ms and 20 ms. If it arrives later in the tick period, it will miss the next tick and have to wait an additional tick period, so the measured duration will be between 20 ms and (20 + X) ms.

If you make N repetitions, you would normally see a spike of density 1 at X. With the 20 ms tick wait, you will see a uniform distribution of density 1/20 between X and (20 + X).

You would have to perform each request and then return the result exactly 20 ms after it was received in order to mask the request duration. But that just creates a new target, your timers and queues to delay the response. Or making the load so high, that requests take more than 20 ms.

comments