Update: I only just saw Part 2 of the post, which is fantastic too.
I like how it describes different tools by the different values they prioritise. I agree with the post's thoughts in the tradeoffs involved in Emacs's development.
This is a great post showing effective performance analysis.
It's worth noting that Emacs's built-in way to find files (project-find-file) in a project does not suffer these performance issues. Try it, C-x p f.
* project-find-file doesn't call expand-file-name for all 70k files (each of which involves a network roundtrip).
* project-find-file will use "git ls-files" (see project--vc-list-files) rather than find (or expecting the user to install and enable fd)
* project-find-file uses completing-read, which wouldn't by default call Ivy/Ivy prescient.
I can see how the author has ended up here: projectile predates the built-in project support, so it must only seem natural to use projectile as a base, and apply performance fixes on top.
Emacs's built in functionality tends to work better with other Emacs functionality, hence why project-find-file works well with TRAMP.
It's worth trimming your config occasionally, and rebasing onto built-in functionality.
The author's fast-project-find-file has a more responsive UI, however. It'd be great to have that in core.