pafoster · HackerLangs

mese scorsodiscuss

Markov chains are themselves a kind of state machine, namely a probabilistic deterministic finite automaton (PDFA), albeit where state is solely governed by the N most recent symbols. (Deterministic means that given a sequence, we can always infer the associated state transitions unambiguously). I believe the example in the reference you provide represents the more general case of PDFA, which is not representable as a finite order Markov processs.

pafoster

2 mesi fadiscuss

JSD is slightly different to forward KL + reverse KL (which is unbounded, whereas JSD measured in bits is in the range [0, 1]).

One way to interpret JSD(P, Q): Associate the distributions P and Q with two target classes, respectively. Pick a target class based on a fair coin flip. Then sample either from distribution P or distribution Q, depending on the outcome of the coin flip. The JSD is the mutual information between the resulting mixture distribution and the target class.

Alternative intuition: Suppose we want to measure the correlation between a feature X and a binary target class Y. We have a tabular data set with two columns X and Y, whose rows correspond to individual samples. JSD is the mutual information between the feature X and the target class Y, but after we resample our data (rows) to ensure that we have a balanced representation of the target class Y. If we measure the JSD in bits, the quantity 2^(JSD-1) is the fraction of times X correctly predicts Y, assuming balanced classes.

pafoster

4 mesi fadiscuss

Developer here. Thanks for all the kind feedback!

pafoster

4 mesi fadiscuss

Many thanks for the feedback and for flagging this up. I've tweaked the drop-down.

comments