m_ke

4,412 karmajoined 14 yıl önce

michal.io

comments

evvelsi gündiscuss

With agentic coding the costs of tokens compound with each message / tool call and etc. Having to load in and update large files makes things slower and way more expensive.

Databricks actually just posted some of their own benchmarks on how harness alone impacts costs https://www.databricks.com/blog/benchmarking-coding-agents-d...

simple things like passing more file context, model having to explore the code base at start of each session, writing comments or markdown docs ends up increasing, running into test / build issues can 3-10x your costs.

PS: my code is still mostly TS and rust but I'm considering moving some of my annotations into .d.ts files and having them generated from runtime types (ala MonkeyType).

evvelsi gündiscuss

After running out of Fable credits in a day on my max plan I started looking around for ways to trim down my token usage and came to the realization that all of the type spaghetti that opus wrote is probably eating up like 50-70% of my tokens.

A clean django project is probably 3-4x less code than the equivalent TS based service.

It made me consider dropping strict mode and defaulting to js for most simple things.

5 gün öncediscuss

Deep Learning models are designed to get max throughput on GPUs, which ends up being batched workloads.

You'll never get proper price competitive utilization on personal hardware vs a cloud inference provider that can batch and pipeline requests optimally to maximize utilization, unless you yourself start running batch jobs.

Even once local hardware and models catch up to todays frontiers, by that time there will be 10x better cluster sized models available at a similar discount.

5 gün öncediscuss

Model is the product, people will put up with anything as long as you're on the pareto frontier of performance x cost.

5 gün öncediscuss

https://openrouter.ai/models

you can use these in hermes, cursor, openclaw, opencode, etc with 2 lines of config that claude code will happily do for you if you ask

GLM 5.2, deepseek 4 Flash and the newly released Hy3 are Opus 4.8 and Sonnet 4.6 level models at a tiny fraction of the cost.

I'm on the $200 claude plan and blew through my weekly limits with Fable in a day, then ended up wasting $20 with opus 4.8 overages in an hour to finish out work in active sessions. Since then I've been using GLM 5.2 with openrouter + opencode and am spending less than $5/day for equivalent output.

5 gün öncediscuss

It's looking like Anthropic is realizing that they're about to get squeezed so they need to juice revenue for their IPO before it becomes obvious to everyone else.

Open models like GLM 5.2 are getting good enough to handle 90% of tasks, and will eat most of their usage unless they start serving it at cost. And on the 1% work they fearmongered their way into falling under government control, which will limit how much they can commercialize the frontier.

Nobody will keep paying their premiums and put up with their BS when they can have similar models at cost of inference in any harness that they want.

7 gün öncediscuss

I mostly use claude code on mobile and desktop with remote sessions and do most of my coding on the go now.

I have been tempted to distill something like GLM 5.2 into a smaller html + css only model for super fast interactive UI editing, because right now it's really annoying to do with large slow models. I'm sure it would be doable to do the same for individual language / frameworks, including potentially doing a final few steps on your own code base with some LoRAs that could be kept up to date to avoid having the model have to explore the code base each time.

Doing UI work with composer 2.5 and live reload is a way different experience than slogging through it with opus 4.8

25 gün öncediscuss

i love when these errors bust my long running sessions and render them unusable

2 ay öncediscuss

until it doesn't.

scaling laws are a power law, you can only stay ahead for so long when each minor improvement gets exponentially more expensive

2 ay öncediscuss

see elevenlabs as a prime example

3 ay öncediscuss

more like 0, once they get AGI they'll capture all value themselves or sell to highest bidders

3 ay öncediscuss

So just a typical dropshipper

4 ay öncediscuss

just think of what could someone do if they got into your personal email account?

4 ay öncediscuss

No these fascists will celebrate this as not being woke, this is just them defending "the west" from muslims...

4 ay öncediscuss

Guess who is running product and other related functions at OpenAI and Anthropic now

4 ay öncediscuss

having been an early employee and founder of a few startups and then working at a few larger companies, most people who only ever worked at FAANG have no idea how much more productive tiny teams with ownership are.

4 ay öncediscuss

I think the DNC and the media might need to get some of that blame for being empty vessels for corporate interests that allowed this conman to get elected twice

4 ay öncediscuss

Don't blame the voters, they didn't get to pick her and did not run her campaign.

4 ay öncediscuss

We can all thank the VCs and CEOs who fully embraced and enabled this administration

4 ay öncediscuss

Imagine if OpenAI fails one day and sells to a company like Palantir. What would happen to all of the sensitive data that they're sitting on?