Prototaxites was a massive, trunk-like organism (up to ~8m tall, ~1m wide) that dominated land ~420–370 million years ago, long before trees or complex plants existed. It looked like a tree, but chemical evidence suggests it didn’t photosynthesize. Internally it was made of interwoven microscopic tubes, unlike plant tissue. It’s often described as a giant fungus, but it doesn’t cleanly match modern fungi either, and some researchers think it may represent an entirely extinct branch of eukaryotic life. In other words, early “forests” may have been dominated by something we don’t have a modern analog for.
When you have LLM requests you don't mind waiting for (up to 24h) then you can save 50% in costs. Great for document processing, image classification at scale, anything that you don't need an immediate result from the LLM provider and costs play a role.
I needed a Python library to handle complex batch requests to LLMs (Anthropic & OpenAI) and couldn't find a good one - so I built one.
Batch requests take up to 24h but cut costs by ~50%. Features include structured outputs, automatic cost tracking, state resume after interruptions, and citation support (Anthropic only for now).
I didn't know that about Maybe Finance. Was that what prompted you to os inboxzero?
What are the pros and cons? Aren't you afraid that it makes the SaaS offering less enticing, especially to highly technical people (which seems like might be a big chunk of your potential users)?
Retool for sure, they are pumping out features really quickly too. The model makes a lot of sense from a developer's point of view. Now that they are also going into public apps I would really consider them seriously.
Hoping for:
- Better pricing for public apps.
- Better ability to easily customize the frontend design without touching CSS/HTML/JS.
There are too many variables to predict the future.
Yet, I believe that it will empower more 1-3 person businesses to compete with the big incumbents.
Imagine you could run teams of marketing/sales/support ai-agents and focus on your core business. "The next 1B, 1 Person companies".
Just 15 years ago you'd need teams of 10+ people to do basic ml tasks you could do today. Now you have an LLM or a SaaS service that outperforms those teams for the fraction of the cost.
I personally use a bit of a different prompt, so it's tailored for my repo.
But yeah, you can also customize it pretty simply, like you mentioned in the ./src/ai.ts file.
This is a blueprint on how to have checks in the CI, it took me sometime to build and get working - do you know anything that does this out of the box?
Thanks for the feedback eitherway!