HackerTrans
TopNewTrendsCommentsPastAskShowJobs

ghshephard

no profile record

Submissions

Caltech Researchers Claim Compression of High-Fidelity AI Models

wsj.com
3 points·by ghshephard·3 ay önce·0 comments

comments

ghshephard
·16 gün önce·discuss
M3 Ultra w/512 GB was released 1 year ago for $9500. I bought one (with a friend's Apple Employee Discount) and originally had a bit of buyer's remorse, because performance was less than some of the Cloud Providers - but recent releases of the quantized GLM 5.2 models are actually pretty speedy and are probably as good or better than any online model I had a year ago - and the discontinuation of the M3 512 has erased that remorse finally.
ghshephard
·24 gün önce·discuss
The Different Model approach is where from tasks to task - I can switch from Opus 4.8, GPT 5.5 and (very often) composer 2 at 1/10th the cost.

It's not perfect, btw - to some degree you are at the mercy of which models they support - currently only 27 from Gemini, OpenAI, Anthropic, Grok, and Kimi (Just K2.5) - presumably because they have commercial arrangements with them. The "Bring your own Model" model requires you plunk in your API key - which sucks. And only one at a time.

To the best of my Knowledge, Claude Code only supports one model at a time if it's not one from Anthropic (which will use the the entire suite of Anthropic Models depending on the task) - and you have to override it to a single model with an environment variable at startup - no ability to flick between models from task to task.

Depending on your workflow - you can save 70-90% on costs just by chosing a reasonable model for really extensive tasks that don't require thinking, max context, etc....
ghshephard
·25 gün önce·discuss
With the one caveat that it does not (yet) support copy/paste buffers or copying from the scroll back buffer. It’s also adamant that you use a mouse to select/copy - anything more complex and you need to pipe everything into neovim (or whatever text editor you use) and do your work there. I love zellij - it feels like the future - but hard to give up keyboard based select/copy and pulling from my scrollback.
ghshephard
·25 gün önce·discuss
You absolutely don't. I use all three products. My preference is Claude Code for my personal project. The one at work is kind of sandboxed off - but does have the benefit of an MCP for every enterprise service we have (Kibana, Victoria Metrics, Grafana, Jira, etc...) - which is nice.

Over time - I expect Composer will be cheaper than Opus 4.8 - but the nice thing about Cursor - you can flick between models.

And (this is purely a personal thing) - I really like the extensive collection of "Plans" that cursor tracks - there isn't really a similar thing in Claude Code - but I really like the Claude.AI interface for everything else. It's also a much better general knowledge agent - the Cursor Chat interface isn't as nice.
ghshephard
·25 gün önce·discuss
It's a solid question - and to some degree what https://programbench.com/ tries to measure.

Some of the issues (off the top of my head):

- Note - that my "product" was about 3,000 lines of code - so tiny. But https://metr.org/ should give you some insight into the complexity the models are capable of.

- you have to be able to imagine the product. If I have the time, and energy, to imagine what I want - the model will build it. Here is an example of a much better programmer than I and something he wanted built - https://www.boatbomber.com/blog/claude-fable-5

- These are the first drafts. On average - any complex system needs about 10 years and at least 1000 active and enthusiastic about reporting users to really get robust code. Writing if via LLM doesn't (at least so far in my experience) help that much in reducing bugs if you were previously following any semblance of TDD. Lots of bugs in the code - the products you listed above have literally tens of millions of years of user experiences and bug reports that got them to where they are today. No silver bullet yet - just faster, less effort - and it enables non-technical people to create (still buggy) products.
ghshephard
·25 gün önce·discuss
You nailed it - in fact, most of Anthropic's early revenue came from Cursor - much of claude code programming components is essentially a feature copy of Cursor, so it makes sense they are similar.

Cursor does have it's own model - it's a heavily reworked version of KimiK2, called "composer" - that I use a lot of the time when I have fairly straightforward tasks that don't require a lot of exploration or independent thought. Lot cheaper - the Input/CacheWrite/CacheRead/Output costs of Opus 4.8 are $5/$6.25/$0.5/$25 per mm tokens, vs $0.5/-/$0.2/$2.5.
ghshephard
·25 gün önce·discuss
Oh- this is super intriguing - can you share the highlight elements of Warp Terminal that make it feel so useful to you?
ghshephard
·25 gün önce·discuss
I think many of the people who have become "AI Pilled" (I'll include myself here) had it happen in the last 3 months. Even over the Christmas break, when the Wiggums loop got so much coverage - I still wasn't that blown away going into January/February- 50%+ of the time I'd just write the code myself. I like coding.

But - I don't know if it was April, or May - but very recently - the coding harnesses paired with decent SOTA models like Opus 4.8/GPT 5.5 - just started showing a lot more consistency, and completeness, and sometimes downright clever behavior - that they started to become way more useful.

Just one out of hundred+ examples - I gave Claude Code (Opus 4.8 High) a complex task that involved consul, vault - but I had neglected to give it sandbox permission to download from hashicorp.com. So - it created a entire test harness that simulated both the behavior of Vault and Consul - created all it's test cases, verified that they passed - and when I came back 40 minutes later said that it was all done.

It's test harnesses so accurately simulated the behavior of Vault/Consul - that on first try - no refactoring whatsoever - all of the protobuf/AESGCM/API behavior (that has varied significantly between versions) - worked.

This was something that would have taken me, someone super super familiar with the code and tools and APIs - a minimum of 3 solid days of work - and that would likely involve hundreds of attempts and refactors as I unwound all the weird encryption and packaging layers. It zero-shotted a full solution without having an API to test against

If these agents actually have an actual test-harness - It's honestly hard to imagine what they can't do - subject only to imagination and budget at this point.

Speaking personally - something changed Between January and, Let's say May - in which instead of seeing these things as mostly interesting technology demonstration, in which the flaws outweighed the benefits - I now genuinely think they are the future of programming. I'm dubious that I'll write much software manually in the future - beyond what I do for personal pleasure.
ghshephard
·25 gün önce·discuss
I think the argument for Cursor is that it's the dominant tool that enterprises are using for coding, so the theory is Cursor wins that as the "model agnostic", it has a phenomenal Enterprise Sales Team.

From a valuation model - $4B ARR with rapid growth, and the ability to shift traffic to internal models (honestly, massive amount of the time "composer" - their internal model is fine, and obviously going to get better). Say 17x Multiple which isn't unheard for a rapidly growing Startup with solid future structural profit elements (moving to internal model) - that gets you to $68B.
ghshephard
·25 gün önce·discuss
I think all of the cues that you just described are in the plan.

For example - I might (real world example from this morning):

"Create a script that installs hashicorp vault and consul, store the data on consul. Then create ahelper script that will fill the vault server with sample data. Add HTTPS support. Now write a framework that reads and decrypts the encrypted data in consul. Support old (pre 1.3) and new (post 1.3 vault). "

That generates a 6 page plan using Opus 4.8 w/1mm context, including notes on what to prioritize, what format to create the scripts in, etc... (My cursor guidance already has a couple months of hints as to what I want in terms of scaffolding unit tests, canonical linux, performance, security, etc...)

That 6 page plan is the "Prompt" - but it's entirely generated by Cursor/Opus. It's there to tweak if you want to emphasize, or provide some taste - but, honestly - it probably does a better job than I would - so ~90% of the time I just accept the plan as is.
ghshephard
·25 gün önce·discuss
As someone who has spent the last 10+ years working in Tmux - but is entirely comfortable on Mac, Windows and Linux desktop environments - here are the key reasons why the terminal experience is superior for me.

- I work a lot with data - and streaming data through text tools is twitch fast. If someone has a question about data - before anybody else can log in to their superset, or analytics database, and try and work through the SQL queries or charts to get the answer - I've already jammed the data through awk and got an answer.

- As an SRE - I work with a lot of systems that have pretty rich APIs - so being able to send a request, get the answer back in json, dump it into jq, select the parts I care about - maybe -c to compress it and ripgrep a subset out - is just fast.

- I work in a lot of contexts with a lot of different systems, datacenters, applications - tmux lets me keep all of them cleanly organized in a separate windows and subpanes. I'll have 15-20 windows open per week, and maybe a 5-6 panes in each- keeping 100+ different contexts (and scroll backs, bash history) - all nicely organized is really useful.

- I'm also a systems guy - and there is no other way to dig into a system but the terminal - netstat, ps, dmesg, /proc - these are all components that have only one credible path to investigation and discovery. If you aren't super comfortable in the terminal - zero way to learn about this stuff.

- Working remotely - means ssh. So - once again - terminal.

The Focus on the terminal is that it's the best tool (and in some cases the only tool) for so many of these tasks - and by performing these tasks a lot - you learn about systems - so the people who spend a lot of time in the terminal tend to know a lot more about systems than people who don't.
ghshephard
·25 gün önce·discuss
I use cursor 8+ hours/day at work, and have full (and effectively unlimited) access to Claude Code and Codex - tools which I also use personally. I suspect that your "constant popups" were when you were using the editor - a mode that I'll confess I haven't touched in 3+ months.

Workflow in Cursor is actually awesome - I'm a little outdated in how I use it - I still establish goals/objectives, rather than managing the loop which does so - but if you can think broadly enough - I find it's pretty efficient.

Key things I like about Cursor (and I recognize I'm dating myself a bit here): - Plan Mode is really solid - I shift-tab, have it go create the plan using whatever insanely expensive SOTA model is available - I will usually spend 5-10 minutes on the Plan - review it, maybe even tweak it a little. (though 90% of the time it's fine out of the gate)

  - Ability to select any model for every task - I'll switch between Opus 4.8 High/xHigh/...  I'll even switch to 1M context for the planning phase upfront.   

  - It does an *excellent* job managing permissions and looping the agents and spinning up sub-agents for you - you set the goal, run the plan mode - and then let it churn for however long is required - pretty common to have a 30-45 minute run and come back to a fully created/tested product.
The nice thing about Cursor (and honestly Claude Code, Codex) - there isn't really any "prompt engineering" involved. You just say, "Go Build me x - it should have y,z features - and build it in golang for me" - and that's it - the 3-4 page Plan comes back - usually pretty credible - and then you click "build.".
ghshephard
·30 gün önce·discuss
Worst case it gets access to gmail. And Github. And the Internet. I'm increasingly appreciating the importance of a physical finger-press on Yubikey to trigger the FIDO2 + OIDC Auth. I don't think there is an easy way for it to hack a new session.
ghshephard
·geçen ay·discuss
Depends no the Enterprise - obviously - in the bay area - 0% of the tech companies care in the slightest. And I'm willing to wager < 5% of enterprises would send their traffic to OpenRouter. Most of them don't even want to send traffic directly to Anthropic or OpenAI - which is why Bedrock has gotten so much traction lately.

But - these $3k-$5k/month/engineer bills are going to start to get attention soon - only question is whether the response is to slow down on the $$$ spending or reduce the # of engineers.
ghshephard
·geçen ay·discuss
See: https://www.axios.com/2023/11/18/how-openai-board-is-structu... for the OpenAI Structure.

1) In order to fund research - this stuff costs 10s of billions of dollars - everyone, from Ilya, to Elon, to Sam - all agreed that they would require a profit-arm to raise money. Nobody was going to sponsor that 10s of billions of dollars to a non-profit.

2) The non profit is still there - and controls the commercial element.
ghshephard
·geçen ay·discuss
I do a lot of code bisecting with Claude Code - and it spends hours running experiments - looking at experiment results, making guesses as to what to try next for an experiment - until it eventually comes around to a working code pattern. I mean - maybe this is as much a reflection on me as anything else - but it's pattern of logic isn't that much different from what I would do. It knows, in general, what tools and APIs it can call - it tries something - observes the result, and then comes back and tries different experiments based on success/failure - mostly efficiently bisecting to a solution.

I'm still lower-down of the capability scale - as I'm still manually directing agents to do these wiggins loops - obviously the next step up is to direct the code-loops which control the agents. I just haven't got my tooling nailed in place to the point where I find that's more productive.

I actually might agree with you that this is mostly just "next token prediction" - if I can concede that's really all I do as well.
ghshephard
·geçen ay·discuss
I think people are continuing to view these systems as pure LLMs - when that ship sailed 6+ months ago. Between being able to review memory, using agent harnesses and sub agents and skills to go out and discover information - modern systems (Codex, Claude Code, Cursor) - use LLMs - but the LLM is only a small component of it. Compare what you get from sending a request to a chatbot like ChatGPT - to what you can from a modern harness. The output is influenced by the LLM, but it's no longer a "model making a token prediction based on training material and RLHF" - that's a very 2025 way of looking at these systems.

Even Gary Marcus is starting to come around and realize that his priors are no longer as relevant as they once were.
ghshephard
·2 ay önce·discuss
Just realize that Gwynne Shotwell is the driver for 99% of the day-day at SpaceX and you can ignore everything else.
ghshephard
·2 ay önce·discuss
You are getting a bit of grief down thread- but this is cool as all get out.

The best use of these systems would be to combine the various procedures:

First, and foremost - don't leave garbage behind in the first place. Think twice before bring sequins and feathers in costumes (the biggest culprit in my experience from 2003-2010). Film cannisters for cigarette

Second - Every Camp does a combination of complete-grid clean up on their own "lot" - I've done that three times - and it was honestly great - plus an hour of "community time" - where you walk the play off your lot and clean it up as well. Your camp packs off 99% of the garbage, and then a grid search, plus heavy rake, finds the last 1%. About the only debate my camp ever had was whether it was acceptable to just dump their potable water onto the Playa (I thought it was fine - as long as you didn't just pour it all in one place - within 15 minutes you would be hard pressed to ever find out where it was poured out).

Third - the two-week "walk the line" where the detailed MOOP maps get created. 150 people for a 80,000 person 7+ day festival seems entirely reasonable - and it's a big part of BRC.

Finally (and I really mean do finally, it's almost a thing that shouldn't be really visible) - show up with the heavy gear to find all the submerged stakes/rebare/moop). Just rake the hell out of the Playa (absolutely fine - I've never understood people who think that it's a problem - it really isn't - you sure as hell aren't going to disrupt any ecology - except for a few random sand-fleas - it's entirely devoid of any life) - and the first bit of rain completely and 100% eliminates any trace of what you did.
ghshephard
·2 ay önce·discuss
The "Markedly Different things" in each bin was a big Amazon Warehouse advance in warehousing. Traditionally - things that were "alike" were put on shelves/bins - but (according to Amazon) it was far more efficient for pickers (at least back in the day - may have changed since then) to have random things on shelves located near each other to allow for equal access to popular items by pickers.