> Why use floats when you can basically have doubles for the same cost?
Floats costing the same as doubles is a myth stemming from x87 arithmetic, which is obsolete. On an x64 CPU running 64-bit code, your compiler can often pack four floats into one SSE register. Even when that doesn't happen, CPU microcode can likely do more with floats than with doubles.
Lastly, memory bandwidth usage and cache occupancy doubles, which is true even with x87.
> The difference is meaningless in scalar code. Not everything is or can be vectorized.
Multiple scalars can be packed into one SSE register even in scalar code. This is not the same as vectorization.
> Pretty much no difference between one double and one float in a SSE XMM register.
Even then, some arithmetic operations have higher throughput with floats than with doubles.
> I have no idea what that means. As far as I know, there's no CPU microcode dealing with floating point numbers.
Your CPU doesn't execute the ISA directly in hardware, it first converts it into architecture specific micro-ops. That includes FP operations.
> So use floats when you have a lot of data.
That's not the point. A lot of the time, the cost difference will indeed be irrelevant. The point is that there is a cost difference.