I tried the same thing last year (with openai models), back then it worked to reduce prompt tokens, but you needed way more completion tokens, ultimately more expensive (and slower)
https://pagewatch.ai/blog/post/llm-text-as-image-tokens/
Still working https://pagewatch.ai/ , my ai readiness audit tool.
Currently having fun building a MCP app for it inside Claude / ChatGPT, its oddly tricky to get things behaving consistently.
There is so much opportunity in AI that is not just a chatbot, I almost feel there should be category of tools that is LLM powered, but not [here is empty textbox]
Best of luck!
There is a surprising amount of edge cases that can cause ChatGPT or others to misunderstand your pages. Some models can handle div based tables, some want alt tags but cannot understand title tags, etc.
I built the tool to check your site as close as possible to what a human would see and then compare it with LLM's.
It was a weird journey trying to tease this info out of the models, they will happily lie, skip checking sites or just make things up.
A task framework could be very useful, setting up celery task can get complicated very fast, especially once you have multiple servers with rolling deploys or recurring task (celery-beat).
Rails has active-job and is always something I felt was missing from Django.
https://render.com/ is probably the closest, I'm really enjoying using them. Workflow is the same as heroku, but cheaper, no nightly restarts, supports new python versions etc..
Pretty clever to host the malware on a sites.google.com domain, makes it look way more trustworthy.
Google should probably stop allowing people to add content under that address.
Astro is really fun to work with, cool that you can add components from vue/react etc all on the same page.
Sounds like they are planning to use the money for their AI builder, hopefully the core keeps getting worked on also.
Well I just signed up and tested one of my Heroku apps, got it working with just a few clicks, really impressive.
Are there any way to see stats like memory usage? What happens if an app goes over the limit? (heroku has some swap space before it kills an app, and then I can see it in the logs)
I have the timeout set to 10 seconds which certainly helps, I think the issue is how gunicorn/gevent handle web requests. I think each request spawns a new redis connection, and as far as I can see there is no global pool I can use :(
On heroku you are limited to 20 connections in the free tier, and it quickly gets expensive.
The max monthly spend is awesome, I'l probably try it out just because of that :)
Its a bit unclear to me though how exactly the heroku deploy works? Is it basically a replacement for the web dynos that heroku provides, but then still connecting to existing postgres instances for example?
What are the limits for the redis store? I'm using it on heroku but constantly running into max connection issues, if you can improve that experience it is also a great win.