jaggederest

vor 9 Stundendiscuss

You can suggest a different one and it works pretty well, the bar is very low.

I asked for hemingway in a planning document one time and the result was highly amusing to everyone. "We will not wreck it with small greeds." was an all time favorite for me.

jaggederest

vor 9 Stundendiscuss

I always laugh because I've been practicing prompting every single day for the last few years, if they want to start a prompt fight, brother let me at 'em.

jaggederest

vor 10 Stundendiscuss

I've used filling a tank, balloon, or bucket (rate of flow, can be subdivided to teach limits, and use weird shapes for teaching area under curve and interpolation en route to integrals), or the classic throwing a ball back and forth and trying to describe the shape, the distance it flies, peak speed vs peak height, figuring out how hard you are "actually" throwing instantaneously. Honestly as soon as you start thinking about bulk substances moving around (gravel piles! fuel tanks!) it's easier to find examples than you'd ever reckon. Rate of change is everywhere.

Seems like I start by asking "how do we know how much this tank holds?", or "how fast does this line go up on the side of the tank?" and curiosity goes from there usually.

jaggederest

vor 10 Stundendiscuss

I'm rooting for pure UTC never adjusted to actual sidereal days. Let me wake up at 1600 and go to sleep at 800, it's fine.

jaggederest

vor 10 Stundendiscuss

I love teaching kids and young adults calculus by socratic method. They get so mad when they figure out you were teaching them math, but they often admit it was pretty fun. Only had the chance to teach like that a few times but it's dynamite when it happens.

jaggederest

gesterndiscuss

Adversarially train the children, rlai works on human brains too?

jaggederest

vorgesterndiscuss

I mean that your code has to accomplish something in the real world that is verifiable on a human level. It has to let customers get something done, or trade resources via a market, or something. That requires that it have some basis in reality that provides a ground truth about whether the system is working or not, and that's what gives you feedback that drives your tests and design.

jaggederest

vorgesterndiscuss

Feed it into an AI and ask it to adversarially criticize it, doc for doc, send back 30 responses in a zip folder, wipe hands on pants, return to HN.

jaggederest

vorgesterndiscuss

Oh man, when MCP was still new and shiny I made an MCP that let the AI choose appropriate theme music for what it was doing and it was an absolute blast, I need to make a more modern one.

Peer Gynt Suite's "In the Hall of the Mountain King" made a prominent appearance, but so did Aqua's "Barbie Girl"

jaggederest

vorgesterndiscuss

It turns out that you don't actually need tests for your tests, because the code provides a baseline truth for the tests. You do, at some point, have to be epistemically sound enough to actually look for correctness in either the code, behavior, or tests. We unfortunately haven't fully unlocked completely solipsistic value generation yet.

This is also part of why I like end to end tests that use actual UI flow, so I can watch it go by in slow mode before letting it loose fully automated.

jaggederest

vorgesterndiscuss

I don't know about that, Fable is, when properly guided, a better engineer for those things than I am. Narrow breadth, weird priorities, myopic and ivory tower as hell, but superhuman. Maybe that says more about me, or maybe not, but certainly it's caught bugs I would not have, and point it at things like a fuzzer, woo buddy, it has been many years since I broke out valgrind and nailed down a memory leak, but it sure can.

jaggederest

vorgesterndiscuss

I want all the layers of the pyramid, eventually, but the top layers matter the most. I can't count the number of times my paranoid "make sure that customers can successfully pay us" end to end test suite has prevented the money faucet from being shut off. I install one perennially at any company I work at and they always pay for themselves surprisingly quickly.

jaggederest

vorgesterndiscuss

I actually dispute this, I read all the code, the core thing people have to give up is not "reading the code" per se, it's giving up on "that's not how I would have done it".

When you see a perfectly clear function or object that just isn't your style, you have to accept it and move on. Where there are concrete concerns, or it's unreadable, demand excellence, but treat it like a coworker, not an IDE.

jaggederest

vorgesterndiscuss

That's interesting because I would feed that benchmark back into the agent and loop over it, to see how much faster you could get it, and agents are really good at that kind of recursive optimization. And I would definitely add at least a simulated 1GiB write test, probably a real one honestly, if I was building something like that.

At least with agent-run tests I care about loop speed a lot, but I care about complete coverage more, so having the odd heavy weight full stack integration test is fine, I think.

jaggederest

vorgesterndiscuss

You have to iterate on the tests, review and validate them, just like any other code, and if you generate a whole project's tests all at once the quality is abysmal, of course. I've been using a lot of old school data-driven testing techniques, where the harness is just code I review, and the data itself is e.g. json files and drives the system.

I actually have a public (AGPL) example here: https://github.com/pgdogdev/pgdog/tree/main/integration/sql - pgdog is particularly testable since it is trying for complete transparency, so you have a perfect oracle in hand via base postgresql, but it demonstrates the concept at least.

jaggederest

vorgesterndiscuss

https://github.com/gastownhall/gascity is certainly a choice. I enjoyed playing with gas town but it was a little too nondeterministic for production code, I think.

Directionally if what you're doing is straightforward it's an amazing experience to be able to slap in an epic planning document and wake up the next day to it being "done", with a big asterisk that done-ness is directly proportional to how good of a spec and how good of a model you were using.

That being said, these days if you use Fable, slap in an epic planning document, and ask it to run a workflow (be sure to specify that subagents should use, say, Sonnet, or wave goodbye to your wallet), it's almost as good as gastown/gascity but far more predictable.

jaggederest

vorgesterndiscuss

I think having style guidance in your context is valuable for avoiding this kind of thing. Having to read awful, cliched text all day is even worse than having to read reams of useless code. I have some simple humanizing content in there that specifically calls out the rhetorical devices that AI loves, and it drastically improves the diffs and comments. It also makes the coding performance generally slightly worse, but ergonomics uber alles.

jaggederest

vorgesterndiscuss

> generate an uncharacteristically exhaustive suite of unit tests to validate every possible scenario.

This is what you want. You want comprehensive tests at every level, far more than is reasonable for a human to build or maintain, from unit, functional, to full end to end and beyond. Adversarial testing (both TDD-style "write tests to demonstrate this bug", and posthoc "prove this patch wrong with a new test") is the best way to keep AI on track and make those diffs you have to read clean and easy.

An even better way is to use a more strongly typed language and really lock it down, but you can use testing in any language. I feel like my background in TDD and "TATFT" has been secret sauce when working with AI

jaggederest

vorgesterndiscuss

I believe they confirmed, on twitter or somewhere I frustratingly can't find, that downgrades are charged at the correct opus rate, after a user asked and was told "either that's how it works or it's a bug"

jaggederest

vorgesterndiscuss

Bench Bench Pro Maxx Series S 360? The original Bench Bench Pro Maxx Series S had some quality issues, so that's the current followup. We've also released a higher order benchmark developed out of Bench Bench Pro Maxx Series S 360 One King Ranch edition, allowing future benchmark towers to be fully self-contained.