HackerTrans
TopNewTrendsCommentsPastAskShowJobs

calaphos

no profile record

comments

calaphos
·7 maanden geleden·discuss
> heat-pump equipment costs roughly €500,000 per megawatt of installed capacity

Interestingly enough the price for these giant heatpumps is pretty much in line with domestic ~10kw units.
calaphos
·8 maanden geleden·discuss
That is comparing an all to all switched Nvlink fabric to a 3D torus for TPUs. Those are completely different network topologies with different tradeoffs.

For example the currently very popular Mixture of Experts architectures require a lot of all to all traffic (for expert parallelism) which works a lot better on the switched NVlink fabric as opposed where it doesn't need to traverse multiple links in the torus.
calaphos
·8 maanden geleden·discuss
It's the one exception in the semiconductor supply chain where Europe is still leading. For all other parts of the value creation Europe is either a niche player at best or completely absent, well into the actual application layer.
calaphos
·9 maanden geleden·discuss
The hardware is heavily optimized for low precision matrix math, pretty much only used for AI.
calaphos
·10 maanden geleden·discuss
> to OpenAI's model gpt-4o-mini

Why a model specifically distilled down for logical reasoning tasks? I would expect larger models to produce a wider variety of outputs.
calaphos
·12 maanden geleden·discuss
But these social third places have also shifted. Younger generations aren't going out as much but e.g. playing video games specifically with other close friends is very popular.
calaphos
·vorig jaar·discuss
Inference throughout scales really well with larger batch sizes (at the cost of latency) due to rising arithmetic intensity and the fact that it's almost always memory BW limited.
calaphos
·vorig jaar·discuss
If it's just filtered out in the training sets, adding the information as context should work out fine - after all this is exactly how o3, Gemini 2.5 and co deal with information that is newer than their training data cutoff.
calaphos
·vorig jaar·discuss
Something where you're reachable for any legal purposes- in Germany this sadly remains a physical address.

There are various service which offer a 'virtual' address with digital forwarding of letters for less than 10Eur/Month, so it's not an insurmountable obstacle.
calaphos
·vorig jaar·discuss
There's still a throughput/latency tradeoff curve, at least for any sort of interactive models.

One of the reasons why inference providers sell batch discounts.
calaphos
·vorig jaar·discuss
There's also big efficiency increases when batching multiple requests, making clouds inherently more cost effective for normal use cases.

Way better utilization of expensive hardware as well ofc.
calaphos
·vorig jaar·discuss
Mixture of experts involves some trained router components which routes to specific experts depending on the input, but without any terms enforcing load distribution this tends to collapse during training where most information gets routed to just one or two experts.
calaphos
·vorig jaar·discuss
It's not quite waste heat because the cold side of thermal power plants wants to be colder than district heating temperatures for best efficiency. There is some loss in electrical efficiency compared to non cogeneration plants, but the combined efficiency is a lot higher.
calaphos
·5 jaar geleden·discuss
Given the enourmous costs of decommissioning nuclear power plants already I fail to see how a reactor containment that lasts 10-20 years at best improves the situation.