If you can get all the images into a filesystem (on a NAS or similar server share), you can use the External Libraries feature in Immich (https://docs.immich.app/features/libraries/). This allows it to crunch through the media files via an async import job (a bit more reliable than having to directly upload via the web api).
In my setup, I exclusively use the external libraries feature, pointed at a read-only share from my NAS mounted onto my Immich server. (The external libraries are set to resynchronize to the database every few hours). This means I can manage all my media assets myself without worrying about Immich accidentally corrupting them, and if I eventually move off Immich, I just have a single folder of media files organized by date to port around.
The only downside is that I don't directly upload any media files directly to Immich, but that's okay. I have Syncopoli sync files from my phones (on a scheduled cadence) to an intermediary server which organizes and cleans exif data from media files before dropping it into its permanent home on my NAS share. No manual steps to get photos from my phone to my Immich instance!
Thanks for sharing this! I'm going to put this on my list to play around with. I'm not really an expert in this tech, I come from the audio background, but recently was playing around with streaming Speech-to-Text (using Whisper) / Text-to-Speech (using Kokoro at the time) on a local machine.
The most challenging part in my build was tuning the inference batch sizing here. I was able to get it working well for Speech-to-Text down to batch sizes of 200ms. I even implement a basic local agreement algorithm and it was still very fast (inferencing time, I think, was around 10-20ms?). You're basically limited by the minimum batch size, NOT inference time. Maybe that's a missing "secret sauce" suggested in the original post?
In the use case listed above, the TTS probably isn't a bottleneck as long as OP can generate tokens quickly.
All this being said a wrapped model like this that is able to handle hand-offs between these parts of the process sounds really useful and I'll definitely be interested in seeing how it performs.
Let me know if you guys play with this and find success.
In my setup, I exclusively use the external libraries feature, pointed at a read-only share from my NAS mounted onto my Immich server. (The external libraries are set to resynchronize to the database every few hours). This means I can manage all my media assets myself without worrying about Immich accidentally corrupting them, and if I eventually move off Immich, I just have a single folder of media files organized by date to port around.
The only downside is that I don't directly upload any media files directly to Immich, but that's okay. I have Syncopoli sync files from my phones (on a scheduled cadence) to an intermediary server which organizes and cleans exif data from media files before dropping it into its permanent home on my NAS share. No manual steps to get photos from my phone to my Immich instance!