This is a byproduct of their time generally being worth so much more than yours. You would not complain if you had to visibly pay for any time a doctor is waiting with no patients.
As for structuring appointments more appropriately, it's quite easy to think of reasons an appointment could take more time than anticipated, and if you had to buffer for that time, you would have to pay for the buffer.