derefr · HackerLangs

True. The real trick, if you have a client-side agent framework to hand, is to prompt it once as "gently" as possible to "just solve the problem"; and then, after its response to that, automatically prompt it again, with a separate prompt, to summarize that response a certain way. That way, the second prompt isn't "in mind" during generation of the first prompt. (And ideally, you don't even present the intermediate result to the user.)

Sadly, you can't do things like this directly using ChatGPT's own "GPTs" abstraction. (For that feature to be useful, they really need some concept of server-side agents as stateful resident IO-stream-reducer actors.)

derefr

hierdiscuss

It would (and does), yes; but this takes a lot more output tokens than asking for a summary would. The summary approach is only helpful insofar as it can be cheaper than using the thinking model. (You're basically tricking the instant model into thinking, which it can do, after a fashion.)

But, unless your desired output is literally a document for others to read, at the point where you're having a model generate a full, lengthy output multiple times over with revisions, you may as well just turn off auto mode and have it always deliberate (i.e. choose the thinking model explicitly from the model selector.) Then it'll be as messy as it needs to be while deliberating, but give you exactly what you want as output.

(And if your desired output is literally a document for others to read, that you want to interactively draft and polish, then (in the case of ChatGPT specifically) you should not only be explicitly forcing the "thinking" model, but also should be asking it to activate the "canvas" feature from the start. My understanding is that revising a canvas document involves the model emitting something like editing gestures, rather than simply re-streaming the updated chunks of text. This saves a lot of output tokens on large documents.)

derefr

hierdiscuss

The "auto" mode is (AFAICT) a per-conversation-turn router. (Presumably via a preliminary pass through a very fast tiny model that spits out an number for how challenging it thinks the next response might be to compute.)

On high-challenge turns, the auto mode routes to the "thinking" model. But on low-challenge turns, it routes to the "instant" model.

And the "instant" model, by design, has no capacity for deliberation. (If it did, it couldn't guarantee that its responses would begin streaming "instantly.")

derefr

hierdiscuss

> Lead with conclusion.

I would presume (perhaps falsely?) that an instruction like this would lead to the model presenting a conclusion not supported by the evidence, and potentially backtracking as it then tries to justify said conclusion.

Yes, if deliberation happens, the model should figure out what it wants to say during that phase; but if you're using auto mode, the model is not going to be doing any deliberating half the time. In those cases, the output blathering is the model's only chance for deliberation. It "thinks as it talks", per se.

Given that, I would advise a different approach: let it blather, but then get it to write you a conclusion at the end that the model can guarantee will obviate the need to read any of the blathering.

I.e. advise the model to add an "executive summary" to the end of any non-trivial-in-length response. With some wording to carefully navigate the model between "the summary is itself too long" vs "the summary acts more like clickbait, leaving out necessary detail such that it requires actually reading the blather."

Not sure exactly what that wording would look like. I imagine something like "write your postscript executive summary as if you were a senior CIA intelligence analyst summarizing ground-level reports into a daily digest for the Joint Chiefs of Staff. Take up as little of their time as possible, but ensure that any detail critical to decision-making is retained." (But that phrasing might only be useful if the model is delivering a certain type of response, and actively counter-productive otherwise. This kind of thing is delicate.)

derefr

avant-hierdiscuss

I haven't stress-tested it, but I would imagine it approaches complex problems the same way a human with a phone in their pocket would — that is, by having a degree of awareness of the confidence it has in its own knowledge in some areas; where, when it "realizes that it doesn't know", it blocks the conversation with statements like "I don't know, let me check."

I say this because this is already how ChatGPT works internally when using its "auto" mode; the version of the "fast" model used in the "auto" mode does the same "notice your ignorance and bring in the heavy model" thing, just silently, rather than mentioning that it's doing it.

(If someone has actually run the experiment, please chime in!)

derefr

avant-hierdiscuss

Given the personality type common on HN, I imagine that the GP, even if unplugged from all technology on their walk, wouldn't be in a mindful state of enjoying their surroundings, but rather would be "lost in the clouds", stewing on the same ideas/thoughts/problems; but with those thoughts going more in circles, due to a lack of ability to verify anything.

derefr

il y a 7 joursdiscuss

I wonder if this is why so many fantasy narratives lean on the trope of a long-dead precursor civilization which had greater knowledge and capabilities than our own. To most people, real historical innovators are in effect members of some mysterious and magical "precursor civilization", rather than "just people" they could identify with, or feel any kind of continuity with.

derefr

il y a 7 joursdiscuss

There are plenty of open-weight models with no such guardrails. Such models are basically the default choice for "collaborative fiction-writing" and "role-playing" tasks.

derefr

il y a 7 joursdiscuss

> CO2 was measured with infrared but water also absorbed it, so you need to heat things up enough to not have water. It can be small, but not watch small.

Can't you just measure CO2 "naively"; but then also, separately, measure rH; and then use the rH value to grab a research-calibrated LUT to pass the raw CO2 value through?

(I presume this is why all the little standalone CO2-sensor boxes you can buy also have rH displays. They're measuring it anyway to normalize the CO2 value, so they may as well make it a feature and display it.)

derefr

il y a 7 joursdiscuss

The trade-off being (I'm guessing) that those threads would be pretty easy to re-tap by hand?

derefr

il y a 10 joursdiscuss

> as they seem to never restock most of their products

There is a product development strategy (I'm not sure if there's a formal name for it) where you're given a lead on a finite-but-large supply of parts you can acquire for absurdly cheap; so you buy the batch; develop and price a product around the part; market your product until you run out of the part; and then, rather than switching over to paying retail for the parts and pricing up your product, you just put your product on indefinite restock hiatus (only ever to be fulfilled if you happen to get another lead on a cheap supply of that same part.)

Usually, though, you get a lead on a cheap supply of a different part; and so the cycle begins again.

derefr

il y a 14 joursdiscuss

> There's been no public policy, executive order, legislation, or otherwise on this

I mean, insofar as you could frame each new model as its own patentable invention, "patent secrecy" would be an existing policy framework that clearly justifies what's been happening here.

1. Some private company or individual invents something.

2. But the state wants a monopoly on the new thing's military use-cases (against other states/militaries.) So the state forcibly classifies the invention at patent time.

3. But the individual/company still wants to make more money than the state is giving them as compensation for their own use of the patent. They want to sell to the private market. But, of course, the state doesn't want to permit this, as arbitrary private parties could in turn resell to foreign state actors.

4. So a compromise is struck: private use is now made deny-by-default. The state permits the individual or company to sell into the private market, given: a rigorous per-customer KYB background-checking process; strong supply-chain tracing; contractual stipulations prohibiting resale; and the customer use-case being transformative or cost-prohibitive to extract the original invention from.

5. As such, big established enterprise customers who want the invention for private use in their internal industrial processes, can somewhat-easily jump all those hoops to acquire access to it; but everyone else is now basically locked out.

Sound familiar?

derefr

il y a 15 joursdiscuss

> "how hot is this deal" (and thus red == best)

If that's the intuition you want, then wouldn't it be natural to use the blackbody radiation spectrum (a.k.a. "color temperature")? Red hot -> white hot -> blue hot.

derefr

il y a 15 joursdiscuss

There is; click on the "CPU DEALSCOPE" button on the page and it shows a legend, while also letting you change what the primary axis of color-coding is. (Though oddly, the x-axis itself doesn't change, so random dots end up colored after changing this.)

derefr

il y a 18 joursdiscuss

You can do that with JPEGs, just not the obvious way (i.e. by exporting the JPEG at the target resolution.)

Instead, you need to export the JPEG at a resampled resolution that's a multiple of your target resolution, such that each pre-transform source-image pixel gets mapped to its own entirely-independent JPEG color block.

Most obvious (though perhaps not optimal?) approach: nearest-neighbour upscale your image by 8x, and then save as JPEG with 100% quality (which will create 8x8 blocks with 4:4:4 subsampling.)

derefr

il y a 18 joursdiscuss

Despite being a web dev back in that era, I had totally forgotten about best practice back then being to add "click here for [...]" in link text. Because users didn't necessarily understand what links were!

derefr

il y a 25 joursdiscuss

The main trick governments use isn't to hide the knowledge of how to build the stuff. It's rather to ban the sale of precursor chemicals and specialized devices (think: industrial-scale centrifuges) except through a government-observable KYC/AML-like chain-of-custody tracking scheme, that assumes/requires each intermediary and final consumer to be an organization certified as meeting certain security requirements.

Individuals obviously need not apply. But regular companies need not apply, either. Think "checkpoints and sign-out sheets that ensure that your own company will notice if some of this stuff disappears." Picture the sort of thing your mind might conjure if you've watched enough forensics protocol dramas and I say "evidence locker" and "tamper-evident seals" — except crossed with hazardous-materials handling policies.

The thing is, this whole chain-of-custody system can be pretty easily circumvented. I won't go deep into how (I'll just say: 1. there are principal-agent problems in academia, and 2. this system wasn't designed to handle sudden organizational bankruptcies well.) But the point is that a grey market for these precursor chemicals and specialized devices exists.

The main place that "false positive" events come from, that the state has to look into, is from people who manage to acquire precursor chemicals/devices without being part of any known chain of custody. (Which, note, doesn't mean that they did anything illegal per se. If it turns out they're just, say, a chemistry-education content creator, then the intelligence body just adds them to their knowledge graph and otherwise leaves them be. But they do have to do some interviewing to determine that first.)

To minimize the number of such events, the "knowledge" that is being truly suppressed here, isn't actually the knowledge of how to do the work; it's the knowledge of how to circumvent the chain-of-custody system. In other words: the logistics.

Information about "how to make a nuke" is general and evergreen; you can just absorb the lesson once and be good. So that info is just "out there", irrevocably. But information about "how to acquire the stuff to make a nuke" is both at least somewhat local to the country you're trying to do it from/in, and also changes all the time, as each state chases up and shuts down existing grey-market channels, and then new ones spring up to replace them. Thus, suppressing logistical knowledge is actually both useful and tractable. And so that's what states mostly go after.

(Mind you, the knowledge of "how to do the thing" does often end up roped into this knowledge-suppression scheme by overzealous downstream regulators who don't understand the load-bearing assumptions of the system they're working under.)

---

The worry states have about LLMs, I think, is that simply by scraping the web into a training dataset, they'll end up stumbling onto the right conversations (that sometimes do indeed happen anonymously in public) to end up with fresh + local chain-of-custody circumvention-logistics knowledge. (And it'd be very hard to "unpick" data like that from the training data.)

Or, even if they don't ingest the data at training time, they'll ingest "the places where that kind of info might end up", and thereby get so good at being "runtime demand-driven searching-and-scraping engines" for this type of thing that they'll be able to surface fresh sources of such info anyway — basically cranking the logistical-pipeline "reconnection speed" after state disruption of a supply channel down to near-zero.

Prohibiting the LLMs from speaking on this subject generally, prevents them specifically from enabling this specific fast-turnaround circumvention-logistics research use-case.

derefr

il y a 28 joursdiscuss

I'd be curious to see a version prompted to recapitulate the style of a Windows 9x app.

Everyone these days seems to fondly recall win9x as the last era when there was an actual visual "system" that applications actually obeyed (...or rather, that every app was forced into obeying, since Windows just wasn't very extensible to performant custom third-party controls until DirectDraw came along. But I digress.) I wonder whether LLMs can build something that actually obeys those rules (i.e. composes everything out of a hierarchy of [simulacra of] first-party W95-era GDI controls — think "Minesweeper is a grid of buttons with icons on them", that kind of thing), rather than just vaguely looking like W95.

derefr

il y a 28 joursdiscuss

My hypothesis is that making the knowledge of how this stuff works accessible to the public results in a lot of false-positives (from people just playing around) that intelligence agencies have to then sift through / tune filters against; which creates a noise floor for real foreign nuke programs to hide in.

So governments ban anything that could result in false positives (since nobody needs to be doing any of that stuff outside of designated labs anyway), to lower that noise floor; to in turn make catching the foreign nuke programs tractable.

(It's a bit like how fancy mansions always have a completely flat and barren part of the property between an outer perimeter and the start of any gardens/outbuildings/water features/etc. That barren area is a killbox: since nothing is supposed to be there, anything at all that does appear there is a valid target for the manion's guards to shoot at [or otherwise engage with], without needing to get a clear identification and command approval first. This wouldn't work if the killbox was covered in vision-obscuring decorative features; nor if the mansion had employees, animals, etc. that had a valid reason to wander into the killbox. So such things are prevented, in order to make the problem of perimeter security tractable.)

derefr

il y a 28 joursdiscuss

> The promise of wasi components has not been fulfilled. The market wants to hotload and link artifacts dynamically. The wasi project requires insider wizardry to use it that way: the offering has been statically linking components before you ship. Defeating 99% of the use cases.

I think both of these points on the spectrum (on the one end, fully static linking of WASI components within a monolithic single-source project; and on the other end, dynamic-at-runtime compiling + linking + loading + "hot instantiating" of arbitrary black-box WASI-component artifacts, with dynamic [presumably reflection-based?] API discovery to drive interaction with those components) are strawmen. There are relatively few "real" use-cases for WASI on either of these ends.

Most of the stuff anyone is really interested in using WASI for (AFAICT, from the use-cases for it I've seen in the wild) involves something closer to the midpoint between these two points. Somewhat dynamic and modular, but with no JIT compilation / components-fetching-components / hot component instantiation / dynamic reflection stuff going on.

Specifically, the "point" of WASI (in my opinion, and in the opinion of most people I've spoken with about it), is to serve as a sort of meta-standard (with tooling) for concrete "pluggable runtime" systems to implement plugin-support SDKs in terms of.

In such "plugin ecosystems", every "plugin" (WASI component) is of the same shape (i.e. exposing the same endpoints, and expecting the same capabilities.) And so the host runtime, and each of its plugins, can be precompiled (separately, in separate projects!) against that shape. And the plugin host can load arbitrary wasm components into a pre-baked plugin "slot" at runtime, because there's no dynamism / introspection / reflection / component framework support required or involved. The plugin host isn't a component itself; it's just ordinary host runtime code, written once. The component framework doesn't load the component; the host runtime does. Etc.

In a sense, this is "custom binding" as you were talking about. But it's custom binding against a WASI-specced target; which is what enables different plugins to be runtime-fungible within the same plugin "slot" from the host's perspective. (While giving you sandboxing for free, unlike the traditional "a plugin is just a DLL that exports certain symbols" approach.) WASI does all the work plugin hosts want of it at component compile/link time: verifying that the plugin is of the expected ABI shape, and guaranteeing sandboxing (by a WASI component inherently being a thing developed to run as an isolate under an abstract machine.) The fact that a compiled+linked WASI-component artifact is a blob of WebAssembly built against certain WIT interfaces, isn't just something tagged onto it by external metadata; it's also something inherent to the structure of the resulting artifact, i.e. a property determinable via static analysis.

At runtime — or slightly earlier, at plugin "install" time — a plugin host might not even keep the component as a component. It might preprocess it into a DLL, or whatever its runtime's equivalent of a DLL is. (A Java class file, say.) The crucial thing was that the component was a component when it was handed off from the downstream developer to the plugin host. Because then any code generated to wrap the component into a host-runtime-native module, is code trusted by the host, rather than code controlled by the downstream developer.

---

If your goal is to compile some one-off blob of WebAssembly code into an artifact that you can then e.g. treat like an old-school ActiveX component from browser JavaScript, then yeah, you don't need WASI at all for that. You're not trying to create a plugin ecosystem. You don't need to spec out a standard "socket" for downstream devs to plug into. You're just "plugging X into a thing that expects only and exactly X". So skip WASI; just use a one-off custom binding. (Though I would note that the WASI work has acted as a forcing function for the WebAssembly-component ABI, enabling you to write much richer custom bindings than you would have been able to write before WASI.)

But if you're:

- developing a FaaS runtime like Cloudflare Workers

- developing a game engine that allows "mods"

- developing a cloud-hosted agent sandbox, where the toplevel is code (that invokes LLMs, that invoke capabilities)

- developing a modern replacement for Wordpress, with an aim to allow just as much extensibility but to not repeat Wordpress's endless vulnerabilities

(etc)

...or, in other words, if you are developing an application or service that O(N) downstream-developed things (workloads, plugins, mods, extensions, whatever you want to call them) all plug into; where you want these things to all plug into your host system in a very precise and controlled way, rather than being given free rein to touch anything they want; and where these interaction patterns can all be described in one of a few very specific shapes, with a precisely definable spec for 1. what API the runtime wants to call into on the component; and 2. what APIs the runtime wants to hand to the component, to enable the component to call those APIs...

...then WASI was developed precisely for you.

And, more specifically, WASI was created so that you could:

1. use WASI to define that API spec (as machine-readable WIT files); and then

2. give that spec, and those WIT files, to the developers in your ecosystem;

3. so that they could then use existing WebAssembly+WASI tooling to build WebAssembly components that target your API spec. (Most likely not by expecting them to independently bootstrap a WebAssembly+WASI dev environment; but rather, by you shipping an SDK that embeds WebAssembly+WASI tooling and your WIT files together.)

(I would also note that this — i.e. "the thing WASI solves for" — is actually a rather rare use-case on the whole. Your average dev isn't [and shouldn't be!] building an ecosystem for API-sandboxed plugins of their code. The few devs that do need to construct their own plugin ecosystems around their project, probably can therefore be expected to go quite deep on learning any required "insider wizardry." If that was even required. Which it generally isn't, when all you're trying to do is to load one of N unknown-until-runtime but statically-defined-ABI-shape plugin components; rather than trying to load arbitrary runtime-generated dynamically-defined-ABI-shape components, allowing those components to load or compile+exec further components, etc.)

comments