Yes! That's where our heads are at as well. The reality with a lot of multimodal / image proc style code is that it's never truly serverless - image manipulation in node.js is tragically bad so you always end up needing python endpoints to do it.
Re: version / client languages etc - right now we don't have block versioning but it's definitely going to be required. As of now the blocks are each their own endpoint, by design. We're thinking about allowing people to share their own blocks and perhaps even outsource compute to endpoint providers, while we focus on the orchstration laters.
Better observability and monitoring is definitely on the docket as well. Especially because some of these tasks take a really long time - some times even going past the expiry window of the REST api. We'll be switching over to queued jobs and webhooks
It's a combination of things. The idea is that you can build workflows that chain functionality from ai models, as well as lower level image processing tasks. For lower level tasks we use the usual suspects - PIL, ImageMagik, OpenCV etc.
Thats true. We started off with a base set of blocks but i think the real utility will come in the easy orchestration and api end point building. We're pushing in the direction of apis and shareable workflows so hopefully some of these comparisons get clarified soon
We started this to solve bulk processing issues we had when building a previous eCommerce tool so I 100% know what you mean. We're adding API support soon and we'll add some examples of how to connect this to Shopify or something like Airtable/ Strapi / Retool etc for workflow automations
That being said, you're not wrong. It's definitely inspired by ComfyUI. But, with much simpler abstractions, much broader utility and extensions like building a user front end coming up shortly
Oh yea I know what you mean. There are several parallels here with shader nodes for sure. We've been thinking about a voyager/agent-style approach where an agent can start to learn "skills" where skills are individual blocks. Each skill represents a certain function applied to an image and based on a specific instruction set we should be able to craft a sequence of actions that will lead to that result.
One way to leverage that is building the graphs via a prompt, but another way might be to not think of the workflow as a pre-constructed graph at all. Rather perhaps we build dynamic graphs whenever you ask for a certain action - like a conversational image editing interface.
So you say something like make the woman's hair purple. We apply segmentation to the hair, and then add a puple color overlay exactly to that area.
Nice! Sharing workflows is coming up in approximately 2 sprints. We're working on 2 flavors of sharing. The first is sharing the workflow directly and letting someone copy it for the dev community. The more interesting option though is the second, where we'll let you build a read-only dashboard that will just show inputs and outputs. that should be useful when you share it with a marketing team that doesn't need to mess around with the graph but would use the workflow for things like repetitive image editing tasks.
Currently there's no way to steer it, but I'm thinking about ways I can allow steering the image gen. Maybe just showing the prompt open ai spits out so you can edit it, but I wonder if there are better ways to allow control
BannerGPT is an AI agent that can auto-generate banner images for any blog post.
The premise is pretty simple. You paste your article into BannerGPT, it analyzes the text and generates a banner image that can complement your article.
Under the hood, the raw post is sent to GPT 3.5 with a prompt to describe a scene that illustrates the main ideas in the text. The description is then sent as a prompt to Stable Diffusion (hosted on Replicate). Finally the title is framed on the image and turned into a downloadable image.
BannerGPT is an AI agent that can auto-generate banner images for any blog post.
The premise is pretty simple. You paste your article into BannerGPT, it analyzes the text and generates a banner image that can complement your article.
Under the hood, the raw post is sent to GPT 3.5 with a prompt to describe a scene that illustrates the main ideas in the text. The description is then sent as a prompt to Stable Diffusion (hosted on Replicate). Finally the title is framed on the image with a nice gradient overlay and turned into a downloadable image.
If you have a personal or company blog, BannerGPT is a nifty little tool that can help add some color to your posts.
I’m curious to see this tested in the wild, so if you share any generated banners in the comments or on twitter I can add them as examples on the site.
yea for sure. i guess i'd like this to be something usable on mobile as well. but directionally you're right. there's a lot to be gained from optimizing for local compute in many applications. something to look into.
Re: version / client languages etc - right now we don't have block versioning but it's definitely going to be required. As of now the blocks are each their own endpoint, by design. We're thinking about allowing people to share their own blocks and perhaps even outsource compute to endpoint providers, while we focus on the orchstration laters.
Better observability and monitoring is definitely on the docket as well. Especially because some of these tasks take a really long time - some times even going past the expiry window of the REST api. We'll be switching over to queued jobs and webhooks