Ask HN: Gen model to 1) design characters 2) make a consistent comic with them?
6 comments
You need to build out a baseline style using text-to-image and then always add those images as "style reference" images in future prompts e.g. "Create a four-panel comic of bla bla bla. Use the attached images strictly as stylistic references only".
That will help keep the model on track.
That will help keep the model on track.
What model allows that?
Preferably one that can be self hosted
Preferably one that can be self hosted
Self-hosting is going to be more difficult. You’ll need to learn how to use Stable Diffusion and some of the common frontends like ComfyUI and Automatic1111 (recommend the NeoForge variant), etc.
Really strong character consistency is usu. done by fine-tuning a custom character LoRA. If that’s more work than you want to take on now, you could probably get by using ControlNets. The ones you’ll probably want to experiment with first are either `revision` or `reference_only`.
You’ll also want to start with a solid base model. Since I’m not familiar with the exact type of art you’re going for, it's hard to give advice here. As a first step look through some of the Illustrious and/or Anima models.
https://civitai.com
Really strong character consistency is usu. done by fine-tuning a custom character LoRA. If that’s more work than you want to take on now, you could probably get by using ControlNets. The ones you’ll probably want to experiment with first are either `revision` or `reference_only`.
You’ll also want to start with a solid base model. Since I’m not familiar with the exact type of art you’re going for, it's hard to give advice here. As a first step look through some of the Illustrious and/or Anima models.
https://civitai.com
Thank you for the detailed reply!
I want to learn
My idea was to go with a "generic" super hero color comic book look. But can be something else too. In fact each character could be different, as long as they stay consistent (-ish).
On a side note, in real comics & manga they start with character design / references too.
Any tips what tools / how to fine tune Lora?
My idea was to go with a "generic" super hero color comic book look. But can be something else too. In fact each character could be different, as long as they stay consistent (-ish).
On a side note, in real comics & manga they start with character design / references too.
Any tips what tools / how to fine tune Lora?
I haven't used it in a while but OneTrainer [1] used to be the "industry standard" for a solid LoRA trainer you can use locally - though a lot of people use CivitAI's LoRA training tools online [2] as well.
Biggest tip I can give is that more images is not necessarily better, rather the most important thing is that you caption/tag the images well so the LoRA learns the right "concepts".
[1] - https://github.com/Nerogar/OneTrainer
[2] - https://education.civitai.com/using-civitai-the-on-site-lora...
Biggest tip I can give is that more images is not necessarily better, rather the most important thing is that you caption/tag the images well so the LoRA learns the right "concepts".
[1] - https://github.com/Nerogar/OneTrainer
[2] - https://education.civitai.com/using-civitai-the-on-site-lora...
I've seen people doing this with GPT Image 2.
The idea is to create consistent character designs of each character first (say ogre, mage, dragonborn)
and then use those designs for panels that would be based on my inputs about the story (I have notes from a longer campaign).
The models I know / tried are not consistent at all.
They dont store the character design, so each panel is different in style (apart from those model that generate designs ripped off from Studio Ghibli).
Is there any free or paid model that allows that?