Cool to see the Coqui team dive into this too. I made a video using Stable Diffusion, Coqui TTS with a VITS model, Frame Interpolation for Large Motion, and Wav2Lip a couple weeks ago:
https://youtu.be/BnrnJ_6wiYs
Theres links to the walkthroughs if anyone wants to set up the open source components and try it out.
My voice model wasnt great for this, so it shouldnt be considered ideal performance by any means.
Compositional generation. Our method can compose multiple diffusion models during inference and generate images containing all the concepts described in the inputs without further training"
Few have those high end cards, but they don't need to anymore. Huggingface is saying it needs 12gb, but the source was forked with some smart mods to chunk the loading on to the GPU.
Itll comfortably run on 6gb now. gtx1600 series cards need to run in full precision mode to produce output. The HLKY fork has improved the Gradio GUI and integrated realesrgan and gfpgan for those with beefier cards.
Someone else also figured out how to load and run it all on a CPU, so pretty much anyone can in theory run the model now.
There is an elaborate Colab notebook linked in the HLKY repo that seems to get more point and click user friendly every time i look at it. I think it even launches the gradio webui so you can use the Colab instance with a webui remotely.
Have you tried giving things time off to heal with an elemental diet? It wasn't pleasant, but I survived off powdered amino acids, oil capsules, honey and cocoa for a couple months. The commerical/hospital preparations are for tube feeding patients and outrageously expensive.
Look into nicotinic acid, a form of niacin. The "flushing" one. If you supplement with it start slow, get used to the burning sensation. It goes away after a few times.
My voice model wasnt great for this, so it shouldnt be considered ideal performance by any means.
Heres Bill Gates rapping: https://youtu.be/1ztm7aBssgA