HackerTrans
TopNewTrendsCommentsPastAskShowJobs

contravariant

9,173 karmajoined قبل 11 سنة
Identity: X -> X

comments

contravariant
·قبل 4 أيام·discuss
There are a couple of references, most on some obscure detail. The ones that might be worth a try for an introduction are:

- Walters, P., An Introduction to Ergodic Theory, Graduate Texts in Mathematics, 79. Springer-Verlag, New York-Berlin, 1982

- Billingsley, P. – Ergodic Theory and Information, John Wiley and Sons, 1965.

You may need a refresher on measure theory, or just ignore that part I suppose
contravariant
·قبل 5 أيام·discuss
Not sure, I'm basing most of this on lecture notes I got at university.
contravariant
·قبل 6 أيام·discuss
I think looking into ergodic theory would help fill in the gap between completely random and completely rule based. In ergodic theory you have some kind of 'state' and a function way to 'advance' the state, if you partition the state space you also get a sequence of 'bytes' to send (simple example: take a number in [0,1] multiply by 10, send the first digit and continue with the remainder). The state needs a distribution, that needs to be preserved by advancing the state, but it's pretty close to running an algorithm otherwise.

You'll probably also want to look at uniquely ergodic systems. Those are interesting in the way that they behave 'random' for every input, one example is if you keep multiplying a number by anything other than a power of 10 and divide it by a power of 10 so it's between 0 and 1. Turns out you get the same distribution of results every time (eventually).

And while you can define an entropy rate without I think ergodic theory also helps inform why that is the correct generalisation. Your current article only considers the case where each digit is independently random (it's clear why, it's a lot simpler and all that information theory tends to talk about).
contravariant
·قبل 6 أيام·discuss
I think the explanation of entropy's blind spot is a bit off. It's not actually a problem for entropy if something is generated by a rule, you can calculate entropy for things like the continuous fractions for instance, with an easy rule to generate them for any particular number. Likewise for decimal expansions.

The real blind spot is that entropy is meaningless for a specific sequence, you can't really ask about the entropy of pi if you don't have a theory for how the numbers are generated. Sure if it is pick a uniformly random real number between 0 and 10 then both files have equivalent entropy, but sending pi is also vanishingly unlikely.

There's actually a more subtle way in which this is a blind spot, which takes a bit more machinery. You can define entropy for an ergodic system, which could be considered a kind of mathematical RNG. Now as it turns out this provides a way to generate something almost equivalent to a particular distribution except that this argument only holds for most starting points not all. A direct example would be how pi generates a perfectly fine random distribution of digits (we think) but something like 1/3 does not.
contravariant
·قبل 17 يومًا·discuss
If compliance was the main issue we wouldn't have had to invent ways for computers to do something other than exactly what they were asked.
contravariant
·قبل 24 يومًا·discuss
You'll recognise the good engineers as the ones that remove code.
contravariant
·قبل 26 يومًا·discuss
Even that is overselling the effort. Last time I checked you could find IDs with a simple image search.
contravariant
·قبل شهرين·discuss
I just want them to tell me if they don't know.

It's the one question that AIs seem unable to answer correctly.
contravariant
·قبل شهرين·discuss
If you feel the need to hide how you got the answer then you know something is wrong.
contravariant
·قبل شهرين·discuss
To minimise the KL you just calculate the surprisal. The integral can be approximated by sampling over your training data. It's a direct expression of the information loss between your real data and your fitted probability distribution.

Calculating the JSD could be more difficult, the expression uses a mixture between the 'true' and 'fitted' distribution. You can still simulate this, but half the time you'd be fitting the model to itself, and I just don't see why that would be useful.

I think the JSD is most useful when you need an actual metric, but as long as you have a fitted and target distribution the KL divergence is a natural fit since you can interpret the result as information loss.
contravariant
·قبل شهرين·discuss
I wonder if using a hilbert curve (or perhaps the simpler z-curve) access order would help things. If ought to work well regardless of cache size.

Actually I think I recall some GPUs storing textures that way, but I'm not entirely sure.
contravariant
·قبل شهرين·discuss
I was expecting more like a minimal usage that makes sense for a butchery or something, not get rid of the people using email.
contravariant
·قبل شهرين·discuss
What's the LLM equivalent to email and a basic webpage?
contravariant
·قبل شهرين·discuss
That's just a baseless assumption. To use AI well you should do the things that allow you to use stuff well. You shouldn't just use it any way you can because you assume that 'not using it at all' is not the best option.

This is literally the same with every single technological development.
contravariant
·قبل شهرين·discuss
You've somehow confused using AI well with using it extensively.

Sometimes using something well involves not using it at all.
contravariant
·قبل 3 أشهر·discuss
I'm also pretty sure 14 points font is a bit outdated at this point, 16 should probably be a minimum with current screens. It's not as if screens aren't wide enough to fit bigger text.
contravariant
·قبل 3 أشهر·discuss
Oh that's annoying, seems to me there wouldn't have been an issue if you just merged B into A after merging A into main, or the other way around but that already works fine as you pointed out.

I mean if you've got a feature set to merge into dev, and it suddenly merges into main after someone merged dev into main then that's very annoying.
contravariant
·قبل 3 أشهر·discuss
Huh interesting, my mental model is unable to see any difference between them.

I mean a branch is just jamming a flag into a commit with a polite note to move the flag along if you're working on it. You make a long trail, leave several flags and merge the whole thing back.

Of course leaving multiple waypoints only makes sense if merging the earlier parts makes any sense, and if the way you continue actually depends on the previous work.

If you can split it into several small changes made to a central branch it's a lot easier to merge things. Otherwise you risk making a new feature codependent on another even if there was no need to.
contravariant
·قبل 3 أشهر·discuss
I think being connected is easy enough, being non-cyclic is trickier I suppose. If you do it badly the shape of the maze is going to depend on the order it's generated in. I imagine some people may have looked into it.
contravariant
·قبل 3 أشهر·discuss
I'd probably go with something like the wave function collapse algorithm. It should be possible to make it generate trees with somewhat uniform probability.