I've done that many times and never used exact equality comparisons.
If you do exact comparisons for any non-trivial cases, you'll find different compilers, optimization settings, runtimes, and processors give different results.
Only if each value is equally likely. If you see $1,000 but figure the envelope-filler is a lot more likely to have been willing to put $1,500 in than $3,000 then you should stick.
You're assuming here that there are discrete stages that do different things. I think a better way to conceptualise these deepnets is that they're doing exactly what you want - each layer is "correcting" the mistakes of the previous layer.
Standard RL algorithms will converge to optimal play versus a fixed opponent, but will not find an optimal policy via self play.
One intuitive way to see this is that a sequence of improving pure policies A < B < C < etc. will converge to optimal play in a perfect information game like chess, but not necessarily in an imperfect information game like rock/paper/scissors where Rock < Paper < Scissors < Rock, etc
Seems to have missed the existence of jax.jit, which basically constructs an XLA program (call it a graph if you like) from your Python function which can then be optimized.
I agree that rejection is always tough. I spent a year manoeuvring to be made redundant from a job I wanted to leave (e.g. told my boss I wanted to go, made sure everything I did was covered by other people). And still, when they did finally let me go, the rejection did hurt quite a bit.
You don't necessarily need the distribution under the alternative to determine the values for which the likelihood ratio will be highest. In your example, the tails will be the areas of maximum likelihood for any (symmetric) alternative.
A more general method is to use the likelihood ratio, ie the ratio of the likelihood of an outcome under the alternative hypothesis to its likelihood under the null hypothesis. And then pick the outcomes which for which this ratio is highest as the ones which will cause you to reject the null hypothesis. Equivalently, the p-value is the probability under the null hypothesis that the likelihood ratio would be at least this large.
This works in the discrete case too, and gives p=1/32 in the original coin flip case.