I've been building this "general problem solver" (will likely focus on math problems first) that uses a special kind of orchestrator to direct/structure the problem solving approach in accordance with how many 'rounds' remain and other aspects of problem context. It does this largely by influencing the behavior of specialists.
You initialize the system with an objective and a number of rounds to run for, and it loads the current config (orchestrator + specialist prompts and LLM configs) and begins working on it. You can manually step one round at a time or just let it run.
Rather than accumulating a single long work log/context, at each round specialists apply patches to a number of named 'artifacts' with different roles (e.g. uncertainties, dead ends, findings), which are injected into prompts during subsequent rounds.
The engine is written in rust and there's a web UI (and CLI). You can use the built in config editor to define specialists (and their prompts), what the artifact set is, orchestrator prompting etc.
Taking on a 'slow' software project with the kind of attention to quality (inside and out) that I had pre-AI. It's a tool I'll use myself, LLM-related, but not any kind of radical idea; it's main value is in careful UX design/efficiency, engineering quality, and aesthetics.
I've been shooting for the moon with one experimental idea after another (like many others) testing out LLM capabilities as they develop, for at least 2yrs now.
I'm still very excited about how these new tools are changing the nature of software development work, but it's easy to get into this frenetic mode with it, and I think the antidote is along the lines of 'slowing down'.
>but you could give me two black boxes that act the same externally, one written as a single line , single character variables, etc. etc. etc. and another written to be readable, and I wouldn't care so long as I wasn't expected to maintain it.
The reality of software products is that they are in nearly in all cases developed/maintained over time, though--and whenever that's the case, the black box metaphor fails. It's an idealization that only works for single moments of time, and yet software development typically extends through the entire period during which a product has users.
> I read OPs "good code" to mean "highly aesthetic code" (well laid out, good abstractions, good comments, etc. etc.)
The above is also why these properties you've mentioned shouldn't be considered aesthetic only: the software's likelihood of having tractable bugs, manageable performance concerns, or to adapt quickly to the demands of its users and the changing ecosystem it's embedded in are all affected by matters of abstraction selection, code organization, and documentation.
Interesting that compaction is done using an encrypted message that "preserves the model's latent understanding of the original conversation":
> Since then, the Responses API has evolved to support a special /responses/compact endpoint (opens in a new window) that performs compaction more efficiently. It returns a list of items (opens in a new window) that can be used in place of the previous input to continue the conversation while freeing up the context window. This list includes a special type=compaction item with an opaque encrypted_content item that preserves the model’s latent understanding of the original conversation. Now, Codex automatically uses this endpoint to compact the conversation when the auto_compact_limit (opens in a new window) is exceeded.
That depends on the content of the SVGs.. Of course you can write a script to do a very literally kind of conversion of regardless, but in practice a lot of interpretation would be required, and could be done by an LLM. Simple case is an SVG that's a static presentation of a button; the intended React component could handle hover and click states and change the cursor appropriately and set aria label etc. For anything but trivial cases a script isn't going to get you far.
That's about how it came across for me as well: ignoring my actual content and joking about generalizations related to key words.
Project is cool overall, love the xkcd-like comic idea—but prompting and/or model-selection could use some work. I'd like to take a crack at tuning it myself :)
It sounds more like you just made an overly simplistic interpretation of their statement, "everything works like I think it should," since it's clear from their post that they recognize the difference between some basic level of "working" and a well-engineered system.
Hopefully you aren't discouraged by this, observationist, pretty clear hansmayer is just taking potshots. Your first paragraph could very well have been written by a professional SWE who understood what level of robustness was required given the constraints of the specific scenario in which the software was being developed.
I've been on a break from coding for about a month but was last working on a new kind of "uncertainty reducing" hierarchical agent management system. I have a writeup of the project here: https://symbolflux.com/working-group-foundations.html
Ah interesting, I missed that possibility. Digging a little more though my understanding is that what's universal is a shared basis in weight space, and particular models of same architecture can express their specific weights via coefficients in a lower-dimensional subspace using that universal basis (so we get weight compression, simplified param search). But it also sounds like to what extent there will be gains during inference is in the air?
Key point being: the parameters might be picked off a lower dimensional manifold (in weight space), but this doesn't imply that lower-rank activation space operators will be found. So translation to inference-time isn't clear.
> So, they found an underlying commonality among the post-training structures in 50 LLaMA3-8B models, 177 GPT-2 models, and 8 Flan-T5 models; and, they demonstrated that the commonality could in every case be substituted for those in the original models with no loss of function; and noted that they seem to be the first to discover this.
Could someone clarify what this means in practice? If there is a 'commonality' why would substituting it do anything? Like if there's some subset of weights X found in all these models, how would substituting X with X be useful?
I see how this could be useful in principle (and obviously it's very interesting), but not clear on how it works in practice. Could you e.g. train new models with that weight subset initialized to this universal set? And how 'universal' is it? Just for like like models of certain sizes and architectures, or in some way more durable than that?
I think the idea is like: it took extra work 'cause Rust makes you be so explicit about allocations and types, but it's also probably faster/more reliable because that work was done.
Of course at the end of the day it's just marketing and doesn't necessarily mean anything. In my experience the average piece of Rust software does seem to be of higher quality though..
Doing math is not the same as calculating. LLMs can be very useful in doing math; for calculating they are the wrong tool (and even there they can be very useful, but you ask them to use calculating tools, not to do the calculations themselves—both Claude and ChatGPT are set up to do this).
If you're curious, check out how mathematicians like Robert Ghrist or Terence Tao are using LLMs for math research, both have written about it online repeatedly (along with an increasing number of other researchers).
Apart from assisting with research, their ability on e.g. math olympiad problems is periodically measured and objectively rapidly improving, so this isn't just a matter of opinion.
I'm an experienced generalist software engineer typically working with early-stage startups or with clients on relatively greenfield projects.
I have strong technical foundations, good product sense, and have gone deep with AI/LLM tech both in the sense of being able to use it effectively, and in exploring the design space for products/tools leveraging it.
I've done a lot of work in developer tools and data visualization of various kinds. Data-rich, non-traditional UIs with highly optimized UX, and rapid prototyping are my forte.
I've been basically on sabbatical for a while now, mostly taking the time to learn and build with AI. At some point in early 2024 I decided: okay, time to take this seriously, and have. After the long break, I'm ready to start something new!
This project was a Show HN and I had a few people reach out to me after, ended up taking a position as a founding engineer at a YC company in SF (2013).
I developed a habit of building these kinds of 'dashboards' from my time in game development, and ended up working on software to generalize the visualization and program data monitoring aspects of them so they could be more easily added to arbitrary programs (always focused on display though, not editing variables—main interest was creating 'ambient' feedback from evolving program data).
If anyone's curious to see this more concretely, I give a demo here of a small one I built for another app I was working on (you'll hear it referenced as "Disk Atlas"): https://youtu.be/TeIgXzvYdpY?t=40
The video also talks about some more sophisticated / general purpose software for doing this (with a focus on creating easily comprehensible displays of data structures evolving over time) called Lucidity (started in 2014, now abandoned).
I've recently come back to the idea, starting from scratch again, this time with less of a focus on showing algorithm execution and more on creating queryable dashboards of dynamic program state.
I threw together a quick demo vid to give a glimpse at the new one, named 'printeff' (you monitor bits of program state by calling `printeff(myData)`): https://www.youtube.com/watch?v=uH4l9n3DaZ4
curious if anyone has thoughts :) I'll put out a better video than this before long though.
Just posted a first early demo and sample orchestrator system prompt yesterday: https://x.com/Westoncb/status/2053429329233895857
You initialize the system with an objective and a number of rounds to run for, and it loads the current config (orchestrator + specialist prompts and LLM configs) and begins working on it. You can manually step one round at a time or just let it run.
Rather than accumulating a single long work log/context, at each round specialists apply patches to a number of named 'artifacts' with different roles (e.g. uncertainties, dead ends, findings), which are injected into prompts during subsequent rounds.
The engine is written in rust and there's a web UI (and CLI). You can use the built in config editor to define specialists (and their prompts), what the artifact set is, orchestrator prompting etc.