I feel like Jeff hawkins was working on this with HTMs, but they never really took off. I suspect it was hardware adaptability based, memory access patterns for sparse graph training aren't ideal candidates for t/gpus.
I understand charge time is a concern, but it is'nt the theoretical limitation of really any battery technology. From the computing parlance it's embarrassingly parallel. More, smaller cells allows for more current => faster charge times.
theoretical limitations: energy density, number of cycles
economics limitation: material and manufacturing costs.
infrastructure limitations: grids to power these charge rates at scale
Sodium Ion is promising because it drastically lowers the material cost. Cheap batteries can help solve the infrastructure problems as energy reservoirs, but I am more or less not swayed by the fast charge time break through. I can show doubling charge time with some AAs.
Do they mean deterministic k-means, k-means++ ... ? Global optimal k-means is NP-Hard, so linear speedups aren't terribly helpful. It's nice, until you add more input. Standard k-means would be nice, or the k-means++ seed algorithm.
I've considered the move to wired not for quality but for the sad state that Bluetooth pairing headphones has become. Theycan't just be headphones anymore; They require their own app and pairing protocol. They want 19 different touch points and permissions to implement a handful of never used features I get people being frustrated at why they can't just do what copper did for the last century.
probably a lot of economics going on, such as early age vendor lock-in, and new market acquisition loss-leaders, but ultimately it's not cutting edge hardware. So the same reason the laptop you bought 2 years ago is half the cost it is today. Granted, even that is not purely a cost only decision. Stratify any market and see how much you can get each segment to pay, and convince them they are getting the best deal for their money.
are you referring to this paper https://arxiv.org/abs/1501.01711 ? i believe they won best paper at icml or other impact journal. the published paper and algorithm i recall being compact and succinct, something that took less than a day to implement.
i agree, not just the multinomial sampling that causes hallucinations. If that were the case, setting temp to 0 and just argmax over the logits would "solve" hallucinations. while round-off error causes some stochasticity it's unlikely to be the the primary cause, rather it's lossy compression over the layers that causes it.
first compression: You create embeddings that need to differentiate N tokens, JL lemma gives us a bound that modern architectures are well above that. At face value, the embeddings could encode the tokens and provide deterministic discrepancy. But words aren't monolithic , they mean many things and get contextualized by other words. So despite being above jl bound, the model still forces a lossy compression.
next compression: each layer of the transformer blows up the input to KVQ, then compresses it back to the inter-layer dimension.
finally there is the output layer which at 0 temp is deterministic, but it is heavily path dependent on getting to that token. The space of possible paths is combinatorial, so any non-deterministic behavior elsewhere will inflate the likelihood of non-deterministic output, including things like roundoff. heck most models are quantized down to 4 even2 bits these days, which is wild!
for an interesting reversal of the "problem" of the speed of light, IEX is a stock exchange design to combat HFT by adding a physical speed bump by way of 38 miles of fiber optic cable. The general idea being to level the playing field and improve market liquidity using physical communication limits of light.
https://en.wikipedia.org/wiki/IEX
i agree, also add to that, that many python modules are foss projects that are maintained on a limited basis or budget. Refactoring code that may have some unsafe async routines would be costly for an org, and dreadful for recreation.
So you can either have a rich library of modules, or go async and risk something you need not working then having to find a workaround.
Personally, if parallelism is important enough, i use ctypes and openmp. If i need something more portable, i have a few multiprocessing wrappers that implement prange and a few other widgets for shared memory.
my slightly next gen todo is a notebook on my remarkable. added features are sharing between devices, and since it's eink its a good paper like alternative to sticky-notes. For me beating procrastination can be more important than organizing many subtasks.
FWIW, i only use this for work todos and differentiate todo with calendar(paper calendar and dry erase board for home, outlook for work calendar)
I've used pbzip2 which takes the same parallel blocked compression approach 7zip seems to be taking (using AI's analysis of the changes). Theoretically the compression is less efficient, but i haven't noticed a difference in practice.