Yeah, I think this is the right idea (or the most optimistic path towards M-series power/performance). If you wanted something fully/aggressively open you could do something like build a mainboard compatible with one of MNT's fully open SOMs like [1].
A book certainly worth reading for anyone who hasn't. It's interesting to see how little common (modern) project and management pitfalls and tricks have changed in 50 years!
[1] https://shop.mntre.com/products/mnt-reform-rcore-rk3588-proc...