Nilay has been discussing these ideas on podcasts hosted by The Verge over the last few weeks. I can tell it's something that's been top of mind for him especially on his podcast series Decoder where he interviews CEOs about their approach to integrating AI in their products and how consumers feel about this.
Fair point! When I first started on this I went down a deep rabbit hole exploring all the ways I could set this up. Ultimately, I decided to start simple with hardware that I had laying around.
I definitely will want to have a dedicated NAS machine and a separate server for compute in the future. Think I'll look more into this once RAM prices come back to normal.