1. Choice of language only matters up to an additive constant (e.g. you could just write a simulator so language A can run language B).
2. If you want something with less physical grounding, you could use lambda calculus instead of Turing machines.
3. Kolmogorov Complexity and Shannon Entropy are compared with one another because they both are talking about the same thing: optimal compression. Kolmogorov Complexity talks about the compressibility of individual objects and Shannon Entropy talks about compressibility of streams of i.i.d. random variables.
2. If you want something with less physical grounding, you could use lambda calculus instead of Turing machines.
3. Kolmogorov Complexity and Shannon Entropy are compared with one another because they both are talking about the same thing: optimal compression. Kolmogorov Complexity talks about the compressibility of individual objects and Shannon Entropy talks about compressibility of streams of i.i.d. random variables.