We don't spend that much money every day but here's the gist: We have a distributed system that has several components that don't meet the performance requirements of the next uplift we need to do. We need to carefully consider the tradeoffs of things like how to shard a few of the databases, how to rearchitect the ETL flow that comes off the system and is used for analysis. We think of a few approaches and then we get the coding assistants to blast through the end to end development of each approach discovering all the known unknowns and unknown unknowns along the way. Then we can load test each method, profile them, analyze them manually and with the LLM. Then we can pick the solution and take another shot at implementing it with the coding agent, but more carefully and with more oversight with all the things we learned.
We don't hit those high numbers every day. An average day is $50-100 max.
As far as home projects. Something like write a GUI desktop or phone application from scratch. The LLM has to reference a lot of code and API docs to figure out what to do and spends a lot of time thinking while debugging. It gets expensive :/
I have at least walked 20 miles before in my life. I've never written anything with as much breadth in 20 years of coding until I started using these tools. I also have quite a deep backlog from trying.
When I sit down and dial in on a serious project at work I easily spend $500-1000+ day of API usage. At home I just hit the limit and give up / cancel my sub.
Wish I could just go all out one weekend a month. I hardly code outside of work but sometimes there’s a project I have an itch for.
may open up other opportunities due to people being more mobile. more commerce activity in general. I heavily dislike having to find parking or hiring a non waymo ride.
this prompt is actually in claude cli. it says something like implement simplest solution. dont over abstract. On my phone but I saw an article mention this in the leak analysis.
creating plans in claude and asking chatgpt via api to review loop was my strategy this week. I'm not a big fan of codex as a coding harness because it seems to just give up quite easily where claude will search the problem space and try things but I think gpt does a much better job of poking holes and asking clarifying questions when prompted.
yeah i experienced this the other day when asking claude code to build an http proxy using an afsk modem software to communicate over the computers sound card. it had an absolute fit tuning the system and would loop for hours trying and doubling back. eventually after some change in prompt direction to think more deeply and test more comprehensively it figured it out. i certainly had no idea how to build a afsk modem.
If the test is taken on personal laptops it's almost impossible to stop cheating. You can easily take video capture and send it out via 5g modem on a completely separate computer hidden inside of the laptop. Then you can receive answers back in very discreet ways like nearly imperceptible LEDs.
Same reason DRM-data rights management or DLP-data loss protection technologies are rather pointless against determined attackers. Anything that is permitted to be viewed can be copied.