Microsoft's head of marketing at the time posted about this on internal Yammer. He said there were two main reasons: first, "Windows 10" just did better in focus groups than Windows 9 or other alternatives; second, since the plan at the time was for all future changes to be delivered as updates rather than a newly branded Windows release†, it felt better to end on a round number.
† btw, what became Windows 11 started development as just another Windows 10 feature update, and what gets branded as a "new release" vs an "update" is mostly a marketing decision.
I'm not sure there's any crisply definable bright line separating them. Generally embrace/extend strategies do involve trying to make extensions that some people will consider valuable.
certain core systems components are often developed by small teams or individuals even within large organizations. the .net GC was written/maintained by 1 dev for a long time (Patrick Dussud, later Maoni Stephens), the Windows thread scheduler was written and maintained by Dave Cutler over many releases, etc.
some development efforts are just really hard to scale out.
† btw, what became Windows 11 started development as just another Windows 10 feature update, and what gets branded as a "new release" vs an "update" is mostly a marketing decision.