Randomization in Controlled Experiments(queue.acm.org)
queue.acm.org
Randomization in Controlled Experiments
https://queue.acm.org/detail.cfm?id=3778029
2 comments
> So our farmer should compare a treatment group of turkeys that watches TV with a control group that doesn’t.
For human subjects if you do that you get an "A vs. A+B study" which has a high probability of bias towards intervention B. Just sitting and talking with someone in a white lab coat will improve outcomes over those who do not.
You must compare to some sort of placebo and the closer it is to the intervention you are testing the better. Testing a drug with side effects against an inert placebo can break blinding and show an effect for the drug that isn't there (Type I error.)
This experimental design is very common in the world of So-Called Alternative Medicine (SCAM) e.g. "patients receiving conventional treatment vs. patients receiving conventional treatment plus a chiropractic technique." If you look for this you will find it virtually every time a SCAM practitioner presents an RCT as evidence for its efficacy.
For human subjects if you do that you get an "A vs. A+B study" which has a high probability of bias towards intervention B. Just sitting and talking with someone in a white lab coat will improve outcomes over those who do not.
You must compare to some sort of placebo and the closer it is to the intervention you are testing the better. Testing a drug with side effects against an inert placebo can break blinding and show an effect for the drug that isn't there (Type I error.)
This experimental design is very common in the world of So-Called Alternative Medicine (SCAM) e.g. "patients receiving conventional treatment vs. patients receiving conventional treatment plus a chiropractic technique." If you look for this you will find it virtually every time a SCAM practitioner presents an RCT as evidence for its efficacy.
You don't need sequences of independent samples for assignment. You need sequences that are independent of the sequence of subjects being assigned. If you have a very small number of possible sequences then you can't get independence by the pigeon hole principle, but both 2^(64*N) (from N random 64 bit numbers) or 2^64 - 1 (for N samples from a PRNG) are so large that it doesn't matter.
The argument about a sample of seeds not producing a flat histogram is also weird. Of course it doesn't. That's irrelevant to whether uniformly sampling a sequence of N 64 bit numbers from the space the PRNG provides produces independent sequences from the subjects.