That's really cool! I made something similar but much simpler using R Shiny.
Edit: Looking at the featured story, I notice that the character illustration is not consistent currently. I got relatively consistent character illustration from scene to scene by asking the LLM to write a prompt for image generation based on the story text and use the same character description in each sentence. That got me pretty consistent character drawings. Also, to keep the same drawing style, my prompt was like:
Drawing style e.g., Digital painting {drawing prompt}
This was a fun project for shiny app contest. This app allows users to generate AI-written stories and illustrations. The app captures user inputs—such as the story’s opening sentence, number of sentences, drawing instructions, and title—and utilizes Cloudflare’s Workers Model API for text generation (Llama 3.1) and image generation (Stable Diffusion). The app compiles story content and images into a Quarto-based revealjs slide deck that is generated in the app. Users may also change the theme of the slide deck. I hope you have fun generating different stories if you give it a go.
I am working on an app for creating stories with illustrations in a revealjs slide deck.
The idea is that a user would handwrite or type the first sentence of a story they want to write. This sentence would be used by llama 3.1 as a prompt to make a short story. Then the story is fed to stable diffusion to create image illustrations. Finally, all of it is combined into a slide deck that users can see in the app and also download if they want.
What do you think about the mechanics of machine learning book by Jeremy Howard? It seems to have a systematic approach to fit models and improve them.
Edit: Looking at the featured story, I notice that the character illustration is not consistent currently. I got relatively consistent character illustration from scene to scene by asking the LLM to write a prompt for image generation based on the story text and use the same character description in each sentence. That got me pretty consistent character drawings. Also, to keep the same drawing style, my prompt was like:
Drawing style e.g., Digital painting {drawing prompt}