Much later, I got to deploy 20,000 PS5 chips to mine ethereum. When PoS happened, we shut it all down. Now those boards (BC-250) are being sold on ebay for $200 and people are running AI on them.
there will always be a special place in my heart for a/ux. i ported a lot of open source software to it. ran a bbs, cu-seeme server, gopherd, httpd, and many other early internet services on it. this really gave me an early taste for what the internet would become.
I bought a cheapo tuya wifi fan controller for a "dumb" ceiling fan. Connect to it from SmartThings app on my phone. The app can be paired with siri and now I can just say: "living room fan on/off". Works great.
Your CW analogy is wonky since that isn't how it went down. You know my name (I don't know yours. edit: Michael), so you should know a bit more about my history in the space too, right? I can explain it out, but afraid of either being called names, or just not being worth it to you (or me for that matter).
EC2 has preemptable and reserved pricing. It is possible to build autosizing solutions, this is what Google did with AppEngine and later GCP Functions/Cloud Run. Just like optimizing start times, it is also possible to optimize those idle resources. For me, I'd go with the idle resources as the lower hanging fruit over trying to shave ms off making things available on-demand, since it affects the customer experience first.
Sure, you're right. I edited to remove that bit. Thanks for calling me out. I was getting frustrated for having felt like I was extremely clear in what I wrote and the person kept repeating something that I had clarified.
> At any rate, warm pools aren't cost free. If you overestimate demand, you'll waste too much money on idle resources.
Depends on how you're running your business. If it is your hardware, it isn't much of an expense at the benefit for having a product that makes your customers happy.
if people want custom features, then of course there is a cost to that. but if the majority of your customers are running on defaults, then there is a benefit. yes, it creates other issues, such as pool management, and if you do that wrong and you can't predict capacity well enough, then people get your "slow" path. but, overall, my experience is that the warm pools are extremely well regarded and not something that most people think of.
Just hot stage a bunch of VMs and then there is no startup time. Every time someone finishes, just start another one and leave it running waiting for the next customer.
He might not have had that choice. Investors can put money into a bank account, and just as easily take it out. This is what happened in the 2000 dotbomb.
This is also why creating a regular account is so difficult on all the social networks. You sign up and it is instantly banned and you have to go through a whole review and approval process just to use it. Incredibly user and company hostile.
I agree with you though, serving up inference is secret sauce for a lot of teams and not everyone publishes how to do it because of the costs involved in doing so. They need an ROI.
https://oj-hn.com