Every single one minus Federal Gov. if you are in a leader position you absolutely should be assessing hosting options, trade offs, costs etc, and looking forward as things grow to make sure you aren’t just burning cash. Only my experience from Australia, the low interest environment could be different in the US, but I would expect this attitude to change if interest rates stay above 5%.
What makes you think “most of them will just pay 10 or 20 times more for AI”?
They can’t measure ROI, and it will start costing more than their staff. You might be right, but I can’t think why any competent C suit would agree to this..
Thanks for this, reading "water fast" and "3 days" gave me a shot of adrenaline. The "water" prefix is just confusing, the word for abstaining from food is just "fast" for those interested.
The cross cutting concerns with ECS + ALB + ASGs + Capacity providers + target groups + EC2 instances + launch templates means that ECS is somewhat central to the larger context of your hosted system. I really like ECS, but AWS have done a poor job of presenting all the linked parts stemming from ECS that this app actually tries to solve. Yes, it wraps several services to achieve this because it has to, but ECS is central enough that having an app like this to show things in a more contextualized way I think can be extremely useful.
This is great, and will be keen to have a look at helping for Linux support you have planned. ECS is one of the better products AWS has produced IMO. It has its warts, but it has made very intentional trade offs between simplicity and control/flexibility, and for a very large amount of use cases, I think they have made some really good choices. Thanks for sharing!
Hasn't there been a _lot_ of debt to buy up Nvidia GPUs? I follow this stuff somewhat closely and it feels intentionally confusing, so I've likely lost track.
They also banned crypto mining which previously was using the free to cheap electricity, so if AI data centres are using those now under utilised supply, very possible subsidies are very low.
> Overall, economic growth is good for society as a whole, so it makes sense that a state should encourage it as much as it can.
The problem with basing the language in the aggregate, is implies that distribution doesn’t matter, and all modern economic models agree. This is a big problem for the reality of people living in ever increasing inequality. Money is a competitive resource, we use it to bid for real resources. If constant economic growth disproportionately goes to the already wealthy, it worsens inequality when resources to exploit become more scarce. It’s one thing to have massive swaths of untouched natural world to exploit for human benefits, but those days are long gone IMO.
This was a large part of my problem with Claude code, it is far too eager to get to the code writing. Matt Pocock's skills and Codex I have found to work together quite well. You still have to ensure design/architecture is being followed, and review carefully obviously, but Codex by default seems to look for minimal change approach a lot more than Claude does/ever did.
It absolutely does and a very good effort of compatibility with GitHub actions. It’s not perfect but migrating is far less of a pain than I experienced moving to others
I've heard stories of bills like that and is wild to me.. I built a SaaS that had just over a PB in data and our monthly was low 5 digit and largest part was S3, and co-location was already on the table. I can't imagine getting to 6-7 digit a month.. I understand how it happens with rapid growth, but even 6 months of that I would be scrambling for other hosting options.
Honestly surprised to hear that GPT OSS 20B runs slow on mac hardware. It's absolutely one of the fastest models I've run on local GPUs for its size, but only tried Nvidia cards.
Edit: TIL it is MoE and only has 3.6B active, explains a lot.
I might be reading the pricing wrong but you have to pay per hour for the port plus per GB transfer? And looks like the cheapest is $0.02 per GB? Is that really the 'cheap' option? That looks fine for a TB or two, but still crazy when getting closer to PBs.
Very impressive. One thing that seems odd to me is that is at like 4 minutes before it starts a response for large input? I don't use mac hardware for LLMs, but that is quite surprising and would seem to be a pretty large stumbling block for practical usage.
Edit: Caching story makes a lot more sense for regular usage:
> Claude Code may send a large initial prompt, often around 25k tokens, before it starts doing useful work. Keep --kv-disk-dir enabled: after the first expensive prefill, the disk KV cache lets later continuations or restarted sessions reuse the saved prefix instead of processing the whole prompt again.
I agree, the mechanical refactoring of modern IDE tooling, especially with typed languages is so much faster and safer, it's not even close. These tools can be useful for sure, but I think in general they are being wayy over prescribed to different tasks.