Got me down a rabbit hole of trying to find the ROI of doing something like this. This is what every GM at AWS thinks about. So, a back of the napkin calculation of running at FP8 and approx 35GB. You can fit that on a g6e.xlarge @ 48GB for about $900/year at reserved capacity.
Assuming you are netting at $5.5 (post stripe fees), and maybe $100/month for API/auth box for egress, you probably need about 170 paying customers to breakeven. This is assuming light/bursty usage.
Kudos! This is a a win win for both you and customer using it as it removes the undifferentiated heavy lifting.
Agree. Seems like claude is losing out on smaller startups. If the hypothesis is that there will be unicorns with 2-3 person company, then why fix at 5?
I used to like Electron but most recently became a fan of Tauri. Electron is great from a UI perspective but seems like it tends to get bloated very quickly. Any suggestions on how to deal with it?
I would love to have an option where instead of just redaction; I'd love to swap it with something else when it goes to AI and then swap it back when the AI returns it. Thanks for sharing the github. I might submit a PR if I don't find that feature