Well, it started off this way, but I went through several iterations with the audit prompt and the system prompt to improve the issue detection, the solutions suggested, tag them, reduce hallucinations, provide approximate image coordinates, provide examples to the model, etc.
It's currently at over 20 lines of prompting and I guess this will grow over time.
Glad to hear it did find something that you can correct.
Regarding the points about fine print (and the screenshot not loaded correctly) - would you kindly reply with the URL of the website you audited here, or email it to me to feedback at flawless.is ?
I'd love to take a look and see if these issues could be fixed or improved.
I think now I better understand your original question.
Yeah, to run an audit again on that same site, you'd need to pay again.
Otherwise, I'll have to put in place user management & user rate limits to avoid abuse, since someone could run this 50 times on their website within a given day, and currently OpenAI limits us to 100 queries per day.
I think GPT-4V perception of size might be distorted, since it downsizes everything to be 512px wide before scanning it. I wonder how can this be overcome, I'll try to play with the prompts to instruct it specifically about sizes.
Regarding the "Meet Slack for Enterprise" - you're right, it is weird, looks like it's hallucinating. I'll try and dig deeper.
Eventually, Hacker News ended up bringing 20,000+ visitors.
Had to upgrade our urlbox screenshot API account twice in 1 hour just to keep up with the demand.
This Hacker News post could have cost $1,000+ if the API rate limit hadn't kicked in. Oh well, silver linings...
Since almost no one could use the app (too many people, and not enough quota from OpenAI), I added a $1.99 one-time payment per audit. This would probably decrease the number of users and also help cover the insanely high costs.
Thanks.
Yeah, it seems that even on Tier 5 the limit is 100/day, that's not enough for any production app.
I hope they go out of preview mode soon and therefore increase the limits
Yeah, I think there are limitless possibilities.
Yesterday I saw on twitter someone using GPT-4V to narrate a video game in realtime, just mind-blowing. The future is here.
Yeah, I feel like the Vision API will unlock many new opportunities.
Honestly, it's been super easy to work with it.
Just sad that they have such a low rate limit in the preview phase.
Interesting about the handwriting - I very rarely handwrite anything anymore, but maybe I should do it ocassionally.
Typically when I type something digitally, I forget about it pretty quickly.
I may try it for some product/architectural stuff where recall is more important.