interesting. tried it on iPhone SE. hard to do safari keeps thinking want to exit page. sliding one touch left right or u/d both might be good alt? certainly better than trad click exactly here on this little folder dart
one nice thing may be way to enhance penetrative ultrasound eg smaller immersive ultrasound tomography in smaller dip tank hey stick your leg in this thinner immersive tunnel , e.g., extremity up-close immersion or trunk wrappers vs just wand-based?
we can do tomography on any round-robin-rectify multi-pov source, doesn't have to be x-ray is just de facto use in medicine, closer to at min marketing ed problem
ya, brain just noisy channel in same way we can treat LLMs; anything possible exists we are just sampling it, which distills to "mere" clock syncing
L1 & L2 constraints unwind that clock compression with suitable dilation; very easy to think only efficiency matters and not averaged out replicas; nature does that inherently via primes, we have to create those artificial waves, recreate that convex hull, etc.
all to say, great to see more work in this direction & perhaps we can compare notes sometime!
one approach that can work is to tell model to load read skill and/or call shell script that overloads default, there are variety of ways to attempt this with any harness, claude specifically has hooks some of which allow go, no go, do this instead etc. and ya, agree on grokking code base, ast integration feels like natural next step
curious: wdym by "getting separators right when generating multiple files in a single inference call"
context: created hypertokens an even more robust hashing mechanism to create context-addressable memory (CAM), one cheat code is make them prefix-free, lots of others that get deep into why models work the way they do, etc.
we dug into those sorts of questions with hypertokens, a robust hash for lines, code, tables/rows or any in-context token tagging to give models photographic memory
one mechanism we establish is that each model has a fidelity window, i.e., r tokens of content for s tag tokens; each tag token adds extra GUID-like marker capacity via its embedding vector; since 1,2,3 digit numbers only one token in top models, a single hash token lacks enough capacity & separation in latent space
we also show hash should be properly prefix-free, or unique symbols perp digit, e.g., if using A-K & L-Z to hash then A,R is legal hash whereas M,C is not permitted hash
we can do all this & more rather precisely as we show in our arXiv paper on same; next update goes deeper into group theory, info theory, etc. on boosting model recall, reasoning, tool calls, etc. by way of robust hashing
Interesting! Been building space-time coordinate system for AI models. Notionally agree in principle w.r.t. convex hull, clocks, etc. since we invoke similar machinery albeit in tokenized models. Need read this work more deeply to grok.
One question is to what extent you dig into or have considered oversampling? One of the core hypotheses we've converged on is that nearly all models are optimized for source coding vs. channel coding. The implication is path to AGI likely involves oversampling to capture channel coding gains and which will resolve phase errors, etc.
Random sampling naturally does this albeit inefficiently. Curious if you do something more structured than random in terms of oversampling and especially partial overlapped samples / think supersaturated subspaces / subchannels, etc.
Basically, just thinking that it’s more ideal to have the tool call the micro VM versus the agent, doing it in the sense of its mandated by the tool call
interesting is the idea the agent calls it or just alt to terminal bash etc tool calls hey your tool calls are all microvms, containers, isoshells, raw term, clawd/molt all credentials with weaker and weaker security demarcs?
security matters if want to demarc where agents can play. running agent inside of strong VM is usually where starts container not enough for that full isolation only sees files you want it to etc
we've considered docker, firecracker, will add smol to working roster
context <> building something with QEMU
* required has to support LMW+AI (linux/mac/windows + android/ios)
there are scenarios in which we might spin micro vms inside that main vm, which by default is almost always Debian Linux distro with high probability.
one scenario is say ETL vm and AI vm isolated for various things
curious why building another microVM other than sheer joy of building, what smol does better or different, why use smol, etc. (microVMs to avoid etc also fair game :)
really great! adjacent well-done ASCII using Braille blocks on X this week:
nolen: "unicode braille characters are 2x4 rectangles of dots that can be individually set. That's 8x the pixels you normally get in the terminal! anyway here's a proof of concept terminal SVG renderer using unicode braille", https://x.com/itseieio/status/2011101813647556902
ashfn: "@itseieio You can use 'persistence of vision' to individually address each of the 8 dots with their own color if you want, there's some messy code of an example here", https://x.com/ashfncom/status/2011135962970218736
ya, hypertokens equalize latent space in spherical harmonic sense / approximate explainer:
take raw context, you inject semantic parity of some form, could be table relating paragraph content, tree, raw summary paragraph. EVENTUALLY those things saturate, call it the inner code; you realize recall and reasoning still not where that; that's where outer code or structural parity (us, others).
why? attention can't do XOR, matrix permanent, latent space noisy, etc., have to smooth & dilate. if pump in tables and schema, model can only do few joins before saturates, no flow lots of sharp corners. so either shrink table or smooth / dilate flow. the catch? every code layer needs a coupling layer at various lengths of resolution -- extra semantic clarifier every paragraph for you, codeword every k tokens for our structural parity, etc.
like engine - here's some air, ok expanding, ok really expanding, ok condensing, ok condense more
our pre-code, your pre-code, content, your post-code, our post-code
btw, pre and post are very important more on why later below -- think interferometry in latent space -- pre-measure / tare scale, load scale with content, post-measure and differentiate (in the latent space)
a much longer dive follows <> leaning into physics a bit, consider old-school trompe, supercharger / cylinders / turbochargers, jet or pretty much any sort of air compressor with flow
ingest air, compress it, extract work, exhaust air; one key side effect is what to do with latent heat; that analogy extends to any physical system
superchargers use raw work to precompress air; turbochargers use waste heat to turn return some lost energy to system turbomachines alternate many alternating static & dynamic stages to max air flow, etc
we do something similar with hypertokens; the raw context window has m tokens; we divide that into b=m/x blocks, where x is hypertoken codeword length, b is the number of blocks, and y is the block size
for example, if the current context window is 2048 and the block size is 32 for the user's desired model performance level, the resulting window would have 64 blocks of 32 content tokens each; if 2-token codeword length between each block would add 128 total tokens, e.g.,
a,1,quick fox,a,2,lazy dog,..,b,3,English pangram
precise hypertoken construction is of course way more subtle than that, e.g., good bit of group theory and way more info theory to define the codes, select the actual tokens that we interleave, etc.
net result is that we diagonalize the latent space action by way of the following; the exact code sequence used is walk on a skewed coprime lattice. Every codeword only appears once, thus acts like a GUID with respect to assocative recall and reasoning. The symbols in the codeword are restricted per lane and the lanes are coprime, e.g. if we had 11,13 for 2-lane codeword then we've induced a prefix-free factor graph action that alternates every k tokens.
Those tokens each have unique init embedding and importantly in practice we almost always put the code word before and after each block, e.g.,
this induces an interferometry like pre/post measurement and since the lanes are coprime, we effectively mimic inflight quasi-Fourier action through context window ~~ project onto compressed code, evolve x content tokens, project back onto same code, so the model gets differential between pre/post sampling. in more practical dev terms this also means we can do precise K:V and V:K lookups during recall and reasoning.
we further do this action in subtly commutative way e,g.,
a;1:quick fox:a;1/...{skip a few}.../b;3:English pangram:b;3/
where : is the global pre/post commutative measure in this example, whereas a;1 or b;3 or whatever the codeword is are globally unique, locally non-commutative, this has several other side effects beyond K:V and V:K or pre & post measurement. That essentially permits "unrolling time" in certain sense especially w.r.t. decoder models, where attention can only look back not forward. by replaying the pre-codeword after block, past tokens can in a summary statistic sense have knowledge about future ones
this of course only works under rather strict construction:
1. must be prefix-free, e.g., if a & b are in lane one they can never be in lane 2 of codeword and vice versa
2. coprime lane counts excepting a parity trick with 2^k lane
3. pre & post measurement -- performance is strictly weaker if only pre or post
4. relatively ortho yet also relatively coherent w.r.t. content, there's lots of ways to achieve those a simple one that works for many broad cases is just <tag-code>/{content}/<tag-code>
5. we can dilate code to pretty much whatever strength needed, e.g., some models and scenarios coherent enough, a simple <letter,num> spreadsheet like code is enough every 128 tokens, for others we need nested think multiscale / multires in physics) and use say Unicode PUA or ideally reserve tokens along with shorter code every 32 inside each 128 could be as simple as /1/.../2/.../3/.../4/
while there's quite a bit more on why it works the gist is we are essentially persistently exciting and sampling using error-correcting code action that happens to induce Fourier like sample and project back like a worm drive boring through rock. since each symbol in each lane gets repeated a few times eg if 3,5 code each 3 symbol is repeated 5x and each 5 symbol is repeated 3x
that means there's all sorts of topological tunnels over a factor graph that generates a skewed lattice in way that reflects the proper group action, arrow of time, etc. going back to why linear block code / linear network code; think stochastic dithering updated to structured dithering
we can of course get way better performance injecting that multiplexing machinery directly into the model; we have some results forthcoming on that; as you can imagine, that machinery is not just toss in primes and call it good
coming back to physics, we essentially use this machinery to detect and project the true spherical of the latent space; we could of course go through the treatment that this is really a reconditioning trick, though we tend to call it retokenization in the discrete sense and reharmonization in the continuous sense; there are certainly overlaps with relaxation, regularization, renormalization, etc.
Very notionally, we relax the problem by dilating context token space-time using this structured persistent excitation and sampling. We do this in a way that in some sense regularizes and renorms the raw signal into lifted domain. The codewords are chosen such that we are effectively heterodyning during pre-code step and superheterodyning during the post-code sample with respect to the local codeword; this process is also happening with respect to the global commutative wrapper around the content block and between the codewords. there is also the skipped subtlety that we can if need be add a conjugate, flipped conjugate, etc. i.e., mimic stronger and stronger ECC / QEC action.
The net effect is that we essentially just treat model as a noisy sender and receiver. We use our hypertokens to stream the raw context using channel coding, which is very similar in net raw principle to MIMO and very similar again in net raw principle to GPS -- we inject a k-channel structured coordinate system that both pre and post samples.
In that sense we are turbomachining the info -- we assume info is dense and can't compress move past / hard to move so we pump our high-speed fluid through the content compress it, repeat.
FINALLY answering a little bit of the tower of tables then suppose we have some code say 5,7 every 128 and 4 every 32
which is essentially the stator-rotor-stator turbo trick dialed up by a lot
- nested / multi-scale / multi-resolution
- pre & post measure commutative global constants <> ;
- pre & post measure commutative local constant <> /
- pre & post measure non-commutative associate marker <> a,1
- etc.
from left during attention each hypertoken absorbs & compresses signal
from the right when attended, each hypertoken injects compressed signal
these signal tunnels / signal network those boost information transport, dilate effective precision, and it works because we're running it over factor graph of bounded treewidth that's essentially running at max capacity
hence we get small LUT, content, medium LUT, content, large LUT content depending how much we nest, how big of code we use, etc. aka a nested table of towers very similar to multires wavelets in action
that table of towers and its background is long way of saying -- models are WAY BIGGER than need to be, auditing & explainability are an EC away, hallucinations don't need to exist, etc.
this of course suggests there are likely physics applications beyond what we're up to -- the easiest way to start thinking about that is noisy HF or phase sensitive systems -- physical transformers and parasitic capacitance is one of my faves to consider, wireless power transfer another, and reservoir machines a third