If you didn't already see https://news.ycombinator.com/item?id=44878151 (Building a web search engine from scratch in two months with 3 billion neural embeddings), then you might enjoy it, even if it's way overkill for your use case.
Focusing on speeding up find_close_polygons instead of realizing that you're matching many points against the same set of polygons is also unfortunate, since that function being slow is a red herring. You can create a scipy.spatial.KDTree for example and just query all the points against that.
It could be worth looking at Wiki.js (https://js.wiki/). It's not perfect, but I started using a self hosted instance recently and it has worked out ok so far. With markdown and git sync it doesn't feel like I'm stuck if I want to move on to something else in the future, or if I want to do some automation/scripting with the contents.