Even with the birthday problem, sqrt(52!) is still about 10^34, which is still huge. It's unlikely that any two (sufficiently random) shuffles in history have been the same.
You're right (and nitpicking nitpicks seems appropriate to me :P).
But, I'm pretty sure the assumptions of logistic regression are even stronger than just that. The inputs are assumed to be independent given the output class, and the log odds of the output vary as a linear function of each input. The first one is essentially the naive Bayes assumption, and the second one is completely unreasonable for almost any problem ever (roughly equivalent to assuming every dataset has a multivariate normal distribution). If they are both correct, though, you get a perfectly good Bayesian posterior probability of each output class.
I think the lesson is that gradient descent will build a decent function approximation out of pretty much anything powerful enough, which is why neural networks still work even when probability theory has been thrown completely out the window.
I think the correct way to state it is: "all true statements are either tautologies, or can be determined empirically (with arbitrarily high but not necessarily measure-1 probability)". This statement is itself a tautology, because tautologies are by definition true, and because "determined" implies some method of determination, which if it actually can be used to determine truth, means it can be used empricially. Determination and empiricism are secretly defined in terms of each other, basically. The reason this tautology is worth stating is that it gives a simple criterion for discarding non-questions: questions that have no method of determining whether they are true or false (with arbitrarily high probability), are always non-questions.
Especially if you assume that the scientific method is always valid--that all true statements can be determined empirically--you can't make statements like "Ultimate Meaning doesn't/does exist". If Ultimate Meaning cannot be defined, it cannot be tested for or measured, so no statements about it can be true or false. The answer to meaning would not be false, but null (or maybe 42).
Just like 42, "atoms and the void" here is just a science-flavored attempt to answer a non-question.
> ... all possible combinations of 8-16 characters with 100 character possibilities ...
Yes, but this doesn't even come close to describing the typical users' password, which is most likely a 6-letter English word with a capital letter and a 1! appended to the end. Your calculation here isn't really relevant, because it's all about the worst or common case. (You also assume that people are using a GPU for a compute-bound problem, when much faster FPGAs are also available, but either way it's moot.)
Security through obscurity, which is what you're proposing with the shuffled salt idea, is also not normally considered the right way to go. If you wanted to use a similar but much simpler and straightforward method, you could just encrypt the salted hashes before storing them in the database.
Kolmogorov complexity is definitely meaningful, but it's not (Shannon) entropy, just conceptually similar. Many people think of something like Kolmogorov-complex sequences when they think of "random" sequences, which is (IMO) why they have trouble thinking of entropy as being about a probability distribution.
The one case where they coincide (sort of) is if you believe your random sequence is generated by a randomly chosen Turing machine, which I've only really seen in philosophical settings.
A uniformly chosen 64-bit integer still has exactly 64 bits of entropy, regardless of how much Kolmogorov complexity the actual bits you generate have.
Right, the amount of entropy per bit of sequence is always between 0 (deterministic) and 1 (every bit is independent and 50/50) (... or between 0 and log2(k) in general if the element varies over a set of k things). These "weak" sources just have low entropy per bit. They could be biased (more 0s than 1s) or correlated (long runs of 0s/1s or periodicity), or just have some other pattern that sometimes holds.
A deterministic PRNG's sequence has exactly the entropy of it's seed, actually, but it has 0 bits of entropy per symbol, because its sequence is infinite.
The thing most people get confused about with entropy is in thinking that entropy is a property of some single object, like a bit string. Really, entropy is always a measurement about a probability distribution, just like mean or variance is. In the usual case with random streams, the distribution is P(x_i | x_i-1 ... x_0) for bits x_i in the stream, i.e. the distribution remaining for the current bit even if we know all previous bits. For a deterministic PRNG, once we can extract the key from the history (given unlimited compute power) that distribution becomes deterministic, so the entropy is 0.
That's because this result is not about combining weak deterministic PRNGs, it's about combining entropy sources (like two hardware random number generators).
This has always been possible, but it sounds like they've lowered the minimum entropy needed in the source streams to produce a high-quality output.
Interestingly, that may be the new ordering if the disks are SSDs, but the typical seek latency on a spinning disk (~5 ms) is definitely higher than the latency to read data from another machine's memory across ethernet (a few hundred us), and even the bandwidths are comparable (~150 MB/s).
So, now it has jumped from (disk -> network -> memory -> ...) to (network -> disk -> memory -> ...), which is a big change.
Sounds a lot like python's "pickle" module (which is super-useful for prototyping), but with the same achilles' heel: all of your serialized objects can now run arbitrary code when you deserialize them!
Of course, knowing that extremists are usually engineers only helps you if you have a known extremist and want to guess his profession. Extremists are a pretty small group, and engineers a much larger one, so I'd have to assume that extremism is still very rare among engineers, meaning knowing whether someone is an engineer or not is completely useless for predicting if they're an extremist.
But hopefully, you've got a STEM degree, and can figure that fact out out yourself ;)
I think one of the common terms for this is a "motte and bailey doctrine", where people define their terms one way during an argument to make their statements super-defensible, but later redefine them to mean whatever crazy thing they actually meant. People seem to use this intentionally and unintentionally a lot.
I'm only a bit disappointed that the author seems not to realize that Bayes' theorem is just a simple consequence of probability theory, and should be attractive not because "maybe the brain is Bayesian", but because it is based on sound set-theoretic and analytic principles. If Bayes' theorem is false, so is probability theory, and so is nearly everything we know about probability.
There are already multiple known molecular mechanisms for long-term memories though.
One of the simplest is spike-timing-dependent plasticity [1], which is caused by the behavior of NMDA receptors shortly before and after depolarization. (This is why ethanol, an NMDA antagonist, can produce a "blackout" in high doses, where no memories are formed.)
In general, neurons have a lot of mutable long-term state. The graph of how neurons are connected can change, the strength of those connections can change, the internal chemistry of the neurons can change through gene expression factors, and, apparently, levels of long-lived prions can change.
This is not to say that the whole mechanism of long-term memory is understood, but that this discovery is just one of a class of mechanisms that may all be working in parallel or even independently.