Honnestly, my congratulations! I am in the same boat as you and 3 years is a long time especially when doing big/risky refactoring. I did my refactoring mainly to manage my application state, I really thought I would never gonna make it (and I was working in Swift, in Javascript it would have been nearly impossible to do the same thing...). But it was worth every second I spent on it: code is the main value of independant developers, so we should take great care of it!
I wish you good luck with your project and hope it will work.
> Running a project for three years alone on the side takes a lot of discipline.
My congratulations!
I could not agree more! I also worked on a Markdown editor for many years (in my case 5). But, the unique perspective I take at this was requiring that much time. In my case, it is really an editor for developers that know their Web stack... Stylo[0] supports styling the Markdown using CSS, live. It allows to do things such as highlighting tagged text parts that we want to concentrate on among other things. Only for Mac though as I also am an independent developer and want to concentrate on features that actually matters for producing texts.
It took me every bit of discipline I had, and it continues...
I have done exactly this in the Markdown editor I developed: Stylo: https://www.textually.net if you're curious.
I used what I call "partial compilation" for each edit made in the Markdown editor that "re-compiles" only affected parts of the text. A function is responsible for computing the ranges to be recompiled based on the edits made in the text. In Stylo it was absolutely essential since CSS is used for syntax highlighting and a complete DOM is kept in memory for the complete Markdown text. The necessity to keep the complete DOM in memory could have been avoided without "following sibling" selectors handling but I really wanted "complete" CSS support. Since Markdown can be used for texts ranging from 1 word to 2000 pages, complete recompilation was unthinkable, so partial one with some exceptions was implemented.
The main difficulty with this kind of approach is that it's possible that the compilation, for whatever range determined, does not stop. Meaning that the effect of the edit goes far beyond the determined range, which function can not know this without compiling... It can happen for example if you edit in C and type "{" then all the scopes below are affected by this opened (and not closed yet) scope. In Markdown it can happen when you open a fence code block (using the ``` opening fence). So you need to have a proper way to handle these edge cases. The other problem is to handle the state associated with these regions and handle the non-stopping compilation case talked before.
All this, as I have discovered, make this kind of compilation and it's associated syntax highlighting really difficult to implement properly and fast enough.
In order to be fast, this process needed to be as asynchronous as possible, because there was some tiny but noticeable lags on the main thread. Even partial compilation was more around 8ms to 12ms on my computer (an old macbook pro 2012). But there is also some cases where asynchronous is not suitable, in which cases synchronous processing is used, but, only when absolutely necessary. So, a mechanic which allows to switch between asynchronous and synchronous processing was needed.
All this to say, it's possible but really difficult to do properly. Every bit of the CPU power have to be extracted to make it comfortable to use. I can just imagine with the complexity of C++ what a nightmare it would be to manage, but it should be possible with a proper compiler architecture that modularise partial compilation and encapsulates the inherent complexity of this approach.
I wish you good luck with your project and hope it will work.