I agree with you tomnipotent. Growth is not equal to more people. We all know that WhatsApp before taken by facebook has only 65 quality people team as compared to big giant companies having thousands of employee with same value.
But in some cases, even without pulling people out of the team, growth is impacted by other factors e.g. competition. How to avoid such factors is definitely a challenge.