Want to also add that the model doesn’t know how to respond in a user-> assistant style conversation after it’s pretraining, and it’s a pure text predictor (look at the open source base models)
There’s also what is being called mid-training where the model is trained on high(er) quality traces and acts as a bridge between pre and post training
128gb unified memory is enough for pretty good models, but honestly for the price of this it is better just go go with a few 3090s or a Mac due to memory bandwidth limitations of this card
~~Likely much less than .25/m if that’s mbps. The issue is you’d have no shortage of money at that scale - I run one of the two main Arch Linux package mirrors in my country and while it’s admittedly a quite niche and small distro in comparison, I’m nowhere close enough to saturate 1gbit on normal days, let alone my 10gbit link~~
It’s a trade off I suppose - you can very well host your own streaming solution, and for the same price you can get a great single node, but if you want good TTFB and nodes with close proximity to many regions you may as well pay for a managed solution as the price for multiple VPS/VM stacks up quickly when you have a low budget
Edit: I think I missed your point about bandwidth pricing lol, but the second still stands
Assuming you’re a tourist, are you sure it’s not your bank flagging the transaction?
When visiting in April I too thought it was an issue with the card when trying to reload on apple wallet, but after two cards declined, found out the transaction was flagged haha
Same experience with reloading on the physical machines. It’s quite amusing
There’s also what is being called mid-training where the model is trained on high(er) quality traces and acts as a bridge between pre and post training