HackerTrans
TopNewTrendsCommentsPastAskShowJobs

contravariant

9,173 karmajoined 11 năm trước
Identity: X -> X

comments

contravariant
·4 ngày trước·discuss
There are a couple of references, most on some obscure detail. The ones that might be worth a try for an introduction are:

- Walters, P., An Introduction to Ergodic Theory, Graduate Texts in Mathematics, 79. Springer-Verlag, New York-Berlin, 1982

- Billingsley, P. – Ergodic Theory and Information, John Wiley and Sons, 1965.

You may need a refresher on measure theory, or just ignore that part I suppose
contravariant
·5 ngày trước·discuss
Not sure, I'm basing most of this on lecture notes I got at university.
contravariant
·6 ngày trước·discuss
I think looking into ergodic theory would help fill in the gap between completely random and completely rule based. In ergodic theory you have some kind of 'state' and a function way to 'advance' the state, if you partition the state space you also get a sequence of 'bytes' to send (simple example: take a number in [0,1] multiply by 10, send the first digit and continue with the remainder). The state needs a distribution, that needs to be preserved by advancing the state, but it's pretty close to running an algorithm otherwise.

You'll probably also want to look at uniquely ergodic systems. Those are interesting in the way that they behave 'random' for every input, one example is if you keep multiplying a number by anything other than a power of 10 and divide it by a power of 10 so it's between 0 and 1. Turns out you get the same distribution of results every time (eventually).

And while you can define an entropy rate without I think ergodic theory also helps inform why that is the correct generalisation. Your current article only considers the case where each digit is independently random (it's clear why, it's a lot simpler and all that information theory tends to talk about).
contravariant
·6 ngày trước·discuss
I think the explanation of entropy's blind spot is a bit off. It's not actually a problem for entropy if something is generated by a rule, you can calculate entropy for things like the continuous fractions for instance, with an easy rule to generate them for any particular number. Likewise for decimal expansions.

The real blind spot is that entropy is meaningless for a specific sequence, you can't really ask about the entropy of pi if you don't have a theory for how the numbers are generated. Sure if it is pick a uniformly random real number between 0 and 10 then both files have equivalent entropy, but sending pi is also vanishingly unlikely.

There's actually a more subtle way in which this is a blind spot, which takes a bit more machinery. You can define entropy for an ergodic system, which could be considered a kind of mathematical RNG. Now as it turns out this provides a way to generate something almost equivalent to a particular distribution except that this argument only holds for most starting points not all. A direct example would be how pi generates a perfectly fine random distribution of digits (we think) but something like 1/3 does not.
contravariant
·17 ngày trước·discuss
If compliance was the main issue we wouldn't have had to invent ways for computers to do something other than exactly what they were asked.
contravariant
·24 ngày trước·discuss
You'll recognise the good engineers as the ones that remove code.
contravariant
·26 ngày trước·discuss
Even that is overselling the effort. Last time I checked you could find IDs with a simple image search.
contravariant
·2 tháng trước·discuss
I just want them to tell me if they don't know.

It's the one question that AIs seem unable to answer correctly.
contravariant
·2 tháng trước·discuss
If you feel the need to hide how you got the answer then you know something is wrong.
contravariant
·2 tháng trước·discuss
To minimise the KL you just calculate the surprisal. The integral can be approximated by sampling over your training data. It's a direct expression of the information loss between your real data and your fitted probability distribution.

Calculating the JSD could be more difficult, the expression uses a mixture between the 'true' and 'fitted' distribution. You can still simulate this, but half the time you'd be fitting the model to itself, and I just don't see why that would be useful.

I think the JSD is most useful when you need an actual metric, but as long as you have a fitted and target distribution the KL divergence is a natural fit since you can interpret the result as information loss.
contravariant
·2 tháng trước·discuss
I wonder if using a hilbert curve (or perhaps the simpler z-curve) access order would help things. If ought to work well regardless of cache size.

Actually I think I recall some GPUs storing textures that way, but I'm not entirely sure.
contravariant
·2 tháng trước·discuss
I was expecting more like a minimal usage that makes sense for a butchery or something, not get rid of the people using email.
contravariant
·2 tháng trước·discuss
What's the LLM equivalent to email and a basic webpage?
contravariant
·2 tháng trước·discuss
That's just a baseless assumption. To use AI well you should do the things that allow you to use stuff well. You shouldn't just use it any way you can because you assume that 'not using it at all' is not the best option.

This is literally the same with every single technological development.
contravariant
·2 tháng trước·discuss
You've somehow confused using AI well with using it extensively.

Sometimes using something well involves not using it at all.
contravariant
·3 tháng trước·discuss
I'm also pretty sure 14 points font is a bit outdated at this point, 16 should probably be a minimum with current screens. It's not as if screens aren't wide enough to fit bigger text.
contravariant
·3 tháng trước·discuss
Oh that's annoying, seems to me there wouldn't have been an issue if you just merged B into A after merging A into main, or the other way around but that already works fine as you pointed out.

I mean if you've got a feature set to merge into dev, and it suddenly merges into main after someone merged dev into main then that's very annoying.
contravariant
·3 tháng trước·discuss
Huh interesting, my mental model is unable to see any difference between them.

I mean a branch is just jamming a flag into a commit with a polite note to move the flag along if you're working on it. You make a long trail, leave several flags and merge the whole thing back.

Of course leaving multiple waypoints only makes sense if merging the earlier parts makes any sense, and if the way you continue actually depends on the previous work.

If you can split it into several small changes made to a central branch it's a lot easier to merge things. Otherwise you risk making a new feature codependent on another even if there was no need to.
contravariant
·3 tháng trước·discuss
I think being connected is easy enough, being non-cyclic is trickier I suppose. If you do it badly the shape of the maze is going to depend on the order it's generated in. I imagine some people may have looked into it.
contravariant
·3 tháng trước·discuss
I'd probably go with something like the wave function collapse algorithm. It should be possible to make it generate trees with somewhat uniform probability.