sgk284

8 ชั่วโมงที่ผ่านมาdiscuss

Yea, that's an interesting result as well. The Terra apps don't feel 35% less feature-rich. So it seems quite token efficient.

sgk284

9 ชั่วโมงที่ผ่านมาdiscuss

Similarly, we updated our model arena (52 apps each built by 26 models) to have GPT 5.6 Sol, Terra, and Luna today:

https://arena.logic.inc/

It's really interesting to see the Sol/Terra/Luna apps side-by-side.

I need to add these stats somewhere in the UI, but one interesting take away: Terra took 1/2 as much wall-clock time as Sol, but Luna took more wall-clock time than Sol (by about 23%). It's still much much cheaper, but it seems like Terra is likely a more optimal time/cost balance for most use cases.

The Terra quality is usually nearly as good as Sol, but much faster and cheaper. I do appreciate Sol's design sensibilities (see, for example, the audio sequencer). It's the first model in a while that is clearly distinct on that front. They'd all converged to very similar visuals for a while.

sgk284

เมื่อวานซืนdiscuss

Awesome - will work on getting those in.

sgk284

เมื่อวานซืนdiscuss

If you like this kind of comparison, we have an arena of 52 apps one-shotted across 21 models here: https://arena.logic.inc/

I keep it pretty up to date (tomorrow Grok 4.5 and Sonnet 5 should be pushed).

sgk284

6 เดือนที่ผ่านมาdiscuss

Yep, 100% correct. We're still reviewing and advising on test cases. We also write a PRD beforehand (with the LLM interviewing us!) so the scope and expectations tend to be fairly well-defined.

sgk284

6 เดือนที่ผ่านมาdiscuss

It doesn't require removing them if you think you'll need them. It just requires writing tests for those edge cases so you have confidence that the code will work correctly if/when those branches do eventually run.

I don't think anyone wants production code paths that have never been tried, right?

sgk284

6 เดือนที่ผ่านมาdiscuss

FWIW all of the content on our eng blog is good ol' cage-free grass-fed human-written content.

(If the analogy, in the first paragraph, of a Roomba dragging poop around the house didn't convince you)

sgk284

6 เดือนที่ผ่านมาdiscuss

I suspect it will still fall on humans (with machine assistance?) to move the field forward and innovate, but in terms of training an LLM on genuinely new concepts, they tend to be pretty nimble on that front (in my experience).

Especially with the massive context windows modern LLMs have. The core idea that the GPT-3 paper introduced was (summarizing):

  A sufficiently large language model can perform new tasks it has never seen using only a few examples provided at inference time, without any gradient updates or fine-tuning.

sgk284

6 เดือนที่ผ่านมาdiscuss

I never claim that 100% coverage has anything to do with code breaking. The only claim made is that anything less than 100% does guarantee that some piece of code is not automatically exercised, which we don't allow.

It's a footnote on the post, but I expand on this with:

  100% coverage is actually the minimum bar we set. We encourage writing tests for as many scenarios as is possible, even if it means the same lines getting exercised multiple times. It gets us closer to 100% path coverage as well, though we don’t enforce (or measure) that

sgk284

6 เดือนที่ผ่านมาdiscuss

Can you say more? I see a lot of teams struggling with getting AI to work for them. A lot of folks expect it to be a little more magical and "free" than it actually is. So this post is just me sharing what works well for us on a very seasoned eng team.

sgk284

7 เดือนที่ผ่านมาdiscuss

Reranking is definitely the way to go. We personally found common reranker models to be a little too opaque (can't explain to the user why this result was picked) and not quite steerable enough, so we just use another LLM for reranking.

We open-sourced our impl just this week: https://github.com/with-logic/intent

We use Groq with gpt-oss-20b, which gives great results and only adds ~250ms to the processing pipeline.

If you use mini / flash models from OpenAI / Gemini, expect it to be 2.5s-3s of overhead.

sgk284

8 เดือนที่ผ่านมาdiscuss

We put this together mostly just to do side-by-side comparisons, though you make a good point. It'd be fun to blind-vote on your favorite impl.

Submissions

Show HN: Homecrew – Share agent skills across your team and keep them in sync

Three Years of Predicting Football with LLMs

On The Obsolescence of Interns in the Age of AI

AI is forcing us to write good code

Engineering Is Becoming Beekeeping

Your Team Uses AI. Why Aren't You 10x Faster?

Machine-Driven Code Review

Magic-Image: A React Component That Lets Coding Agents Create Images

Codex is a Slytherin, Claude is a Hufflepuff

Intent: An LLM-Powered Reranker Library That Explains Itself

How to Ship Confidently When Your Back End Makes Things Up

Deep Onboarding

Fine-Tuning Is (Probably) a Trap

Show HN: Turn your startup logo into a holiday Google doodle

Show HN: Agentic Arena – 52 tasks implemented by Opus 4.5, Gemini 3, and GPT-5.1

Show HN: Logic, Inc. – Automate human-in-the-loop fuzzy decisions

comments