For large projects like this I think a hierarchical division of labor also helps.
If you first carefully define the overall architecture and thus individual high level components of the system, then you know which of those components are mission critical and which are commodity. Mission critical would be anything ensuring ACID, etc. That way, no matter what you farm out to LLMs, you can keep the majority of limited human focus on the far fewer mission critical components. If tests end up not being robust enough to catch all issues, at least they'll be isolated to commodity code where damage is limited to things like DoS, etc, and not code that could cause data loss.
I also think it's important to first define the _contracts_ on and between each of these components, and derive tests from those contracts. Partly because contracts more succinct and easier to reason about. And partly because Rust provides many tools to enforce contracts at compile time, reducing the need for tests (which themselves could end up subtly flawed). Contracts can be enforced through typing, private vs public APIs, etc. Newtypes are _incredibly_ powerful for both enforcing contracts and making footguns much less likely.
Not only that, they additionally ran an experiment with the training temperature turned way up (2.0) and truncation turned off such that the majority of SFT examples were incoherent (63% IIRC). Yet the model finetuned on these broken examples still improved over baseline.
> Is every new thing not just combinations of existing things?
If all ideas are recombinations of old ideas, where did the first ideas come from? And wouldn't the complexity of ideas be thus limited to the combined complexity of the "seed" ideas?
I think it's more fair to say that recombining ideas is an efficient way to quickly explore a very complex, hyperdimensional space. In some cases that's enough to land on new, useful ideas, but not always. A) the new, useful idea might be _near_ the area you land on, but not exactly at. B) there are whole classes of new, useful ideas that cannot be reached by any combination of existing "idea vectors".
Therefore there is still the necessity to explore the space manually, even if you're using these idea vectors to give you starting points to explore from.
All this to say: Every new thing is a combination of existing things + sweat and tears.
The question everyone has is, are current LLMs capable of the latter component. Historically the answer is _no_, because they had no real capacity to iterate. Without iteration you cannot explore. But now that they can reliably iterate, and to some extent plan their iterations, we are starting to see their first meaningful, fledgling attempts at the "sweat and tears" part of building new ideas.
As far as I can tell 5.2 is the stronger model on paper, but it's been optimized to think less and do less web searches. I daily drive Thinking variants, not Auto or Instant, and usually want the _right_ answer even if it takes a minute. 5.1 does a very good job of defensively web searching, which avoids almost all of its hallucinations and keeps docs/APIs/UIs/etc up-to-date. 5.2 will instead often not think at all, even in Thinking mode. I've gotten several completely wrong, hallucinated answers since 5.2 came out, whereas maybe a handful from 5.1. (Even with me using 5.2 far less!)
The same seems to persist in Codex CLI, where again 5.2 doesn't spend as much time thinking so its solutions never come out as nicely as 5.1's.
That said, 5.1 is obviously slower for these reasons. I'm fine with that trade off. Others might have lighter workloads and thus benefit more from 5.2's speed.
I wish they would keep 4.1 around for a bit longer. One of the downsides of the current reasoning based training regimens is a significant decrease in creativity. And chat trained AIs were already quite "meh" at creative writing to begin with. 4.1 was the last of its breed.
So we'll have to wait until "creativity" is solved.
Side note: I've been wondering lately about a way to bring creativity back to these thinking models. For creative writing tasks you could add the original, pretrained model as a tool call. So the thinking model could ask for its completions and/or query it and get back N variations. The pretrained model's completions will be much more creative and wild, though often incoherent (think back to the GPT-3 days). The thinking model can then review these and use them to synthesize a coherent, useful result. Essentially giving us the best of both worlds. All the benefits of a thinking model, while still giving it access to "contained" creativity.
Well yeah, because 5.2 is the default and there's no way to change the default. So every time you open up a new chat you either use 5.2 or go out of your way to select something else.
(I'm particularly annoyed by this UI choice because I always have to switch back to 5.1)
> FPGAs will never rival gpus or TPUs for inference. The main reason is that GPUs aren't really gpus anymore.
Yeah. Even for Bitcoin mining GPUs dominated FPGAs. I created the Bitcoin mining FPGA project(s), and they were only interesting for two reasons: 1) they were far more power efficient, which in the case of mining changes the equation significantly. 2) GPUs at the time had poor binary math support, which hampered their performance; whereas an FPGA is just one giant binary math machine.
I had to return my Vision Pro after trying it for a week. I'm one of those rare customers that genuinely wanted to keep it, because it's the only VR headset I could _actually_ get work done in thanks to its stellar resolution and overall screen quality. In spite of its many, many flaws. But I had to ditch the thing because: 1) it's stupidly heavy, and 2) it's the only headset that caused me eyestrain.
I was praying for a new revision, but ... this wasn't it. No mention of making the thing lighter. Seems like instead they _added_ weight to the band to compensate.
Guess I'll keep waiting and hoping someone else fills the space. Maybe, just maybe, there will be a real Quest Pro with the same screen quality as the AVP. The Quest 3 is almost perfect in every regard except for that, so I'd happily drop "stupid" money to grab one with an AVP level display in it. (With the usual caveats of it being an evil Meta product, etc, etc).
At least in the U.S. the equality of women in society (and in law) has slowly risen over the last 100 years. Over that same period the availability of pornographic images has also slowly risen (from magazines, to VHS, to the Internet, to streaming videos, to VR).
So if we're looking at correlation, doesn't the data imply that _more_ porn is associated with _more_ rights for women?
(Conversely, the vast majority of people calling for and enacting policies for more restrictions on pornography are also rolling back rights for women.)
There are a few different approaches. Meta documents at least one approach quite well in one of their llama papers.
The general gist is that you have some kind of adapter layers/model that can take an image and encode it into tokens. You then train the model on a dataset that has interleaved text and images. Could be webpages, where images occur in-between blocks of text, chat logs where people send text messages and images back and forth, etc.
The LLM gets trained more-or-less like normal, predicting next token probabilities with minor adjustments for the image tokens depending on the exact architecture. Some approaches have the image generation be a separate "path" through the LLM, where a lot of weights are shared but some image token specific weights are activated. Some approaches do just next token prediction, others have the LLM predict the entire image at once.
As for encoding-decoding, some research has used things as simple as Stable Diffusion's VAE to encode the image, split up the output, and do a simple projection into token space. Others have used raw pixels. But I think the more common approach is to have a dedicated model trained at the same time that learns to encode and decode images to and from token space.
For the latter approach, this can be a simple model, or it can be a diffusion model. For encoding you do something like a ViT. For decoding you train a diffusion model conditioned on the tokens, throughout the training of the LLM.
For the diffusion approach, you'd usually do post-training on the diffusion decoder to shrink down the number of diffusion steps needed.
The real crutch of these models is the dataset. Pretraining on the internet is not bad, since there's often good correlation between the text and the images. But there's not really good instruction datasets for this. Like, "here's an image, draw it like a comic book" type stuff. Given OpenAI's approach in the past, they may have just bruteforced the dataset using lots of human workers. That seems to be the most likely approach anyway, since no public vision models are quite good enough to do extensive RL against.
And as for OpenAI's architecture here, we can only speculate. The "loading from top to be from a blurry image" is either a direct result of their architecture or a gimmick to slow down requests. If the former, it means they are able to get a low resolution version of the image quickly, and then slowly generate the higher resolution "in order." Since it's top-to-bottom that implies token-by-token decoding. My _guess_ is that the LLM's image token predictions are only "good enough." So they have a small, quick decoder take those and generate a very low resolution base image. Then they run a stronger decoding model, likely a token-by-token diffusion model. It takes as condition the image tokens and the low resolution image, and diffuses the first patch of the image. Then it takes as condition the same plus the decoded patch, and diffuses the next patch. And so forth.
A mixture of approaches like that allows the LLM to be truly multi-modal without the image tokens being too expensive, and the token-by-token diffusion approach helps offset memory cost of diffusing the whole image.
I don't recall if I've seen token-by-token diffusion in a published paper, but it's feasible and is the best guess I have given the information we can see.
EDIT: I should note, I've been "fooled" in the past by OpenAI's API. When o* models first came out, they all behaved as if the output were generated "all at once." There was no streaming, and in the chat client the response would just show up once reasoning was done. This led me to believe they were doing an approach where the reasoning model would generate a response and refine it as it reasoned. But that's clearly not the case, since they enabled streaming :P So take my guesses with a huge grain of salt.
I think "reasoning" models will solve the joke issue (amongst other issues), but not because they're "reasoning". Rather because they help solve the exploration issue and the scaling issue.
Having worked with LLMs a lot for my JoyCaption project, I've got all these hypothesis floating around in my head. I guess the short version, specifically for jokes, is that we lack "joke reasoning" data. The solution, like mathematical problems, is to get the LLM to generate the data and then RL it into more optimal solutions.
Longer explanation:
Imagine we want an LLM to correctly answer "How many r's are in the word strawberry?". And imagine that language has been tokenized, and thus we can form a "token space". The question is a point in that space, point Q. There is a set of valid points, set A, that encompasses _any_ answer to this question which is correct. There are thus paths through token space from point Q to the points contained by set A.
A Generator LLM's job is, given a point, predict valid paths through token space. In fact, we can imagine the Generator starting at point Q and walking its way to (hopefully) some point in set A, along a myriad of inbetween points. Functionally, we have the model predict next token (and hence point in token space) probabilities, and we can use those probabilities to walk the path.
An Ideal Generator would output _all_ valid paths from point Q to set A. A Generator LLM is a lossy compression of that ideal model, so in reality the set of paths the Generator LLM will output might encompass some of those valid paths, but it might also encompass invalid paths.
One more important thing about these paths. Imagine that there is some critical junction. A specific point where, if the Generator goes "left", it goes into a beautiful flat, grassy plain where the sun is shining. That area is really easy to navigate, and the Generator LLM's predictions are all correct. Yay! But if it goes "right" it ends up in the Fire Swamp with many dangers that it is not equipped to handle. i.e. it isn't "smart" enough in that terrain and will frequently predict invalid paths.
Pretraining already taught the Generator LLM to avoid invalid paths to the best of its abilities, but again its abilities are limited.
To fix this, we use RL. A Judge LLM takes a completed path and determines if it landed in the set A or not. With an RL algorithm and that reward signal, we can train the Generator LLM to avoid the Fire Swamp, since it often gets low rewards there, and instead goes to the Plain since it often gets rewards there.
This results in a Generator LLM that is more _reliable_ and thus more useful. The RL encourages it to walk paths it's good at and capable of, avoid paths it struggles with, and of course encourages valid answers whenever possible.
But what if the Generator LLM needs to solve a really hard problem. It gets set down at point Q, and explores the space based on its pretraining. But that pretraining _always_ takes it through a mountain and it never succeeds. During RL the model never really learns a good path, so these tend to manifest as hallucinations or vapid responses that "look" correct.
Yet there are very easy, long paths _around_ the mountain that gets to set A. Those don't get reinforced because they never get explored. They never get explored because those paths weren't in the pretraining data, or are so rare that it would take an impractical amount of exploration for the PT model to output them.
Reasoning is one of those long, easy paths. Digestible small steps that a limited Generator LLM can handle and use to walk around the mountain. Those "reasoning" paths were always there, and were predicted by the Ideal Generator, but were not explored by our current models.
So "reasoning" research is fundamentally about expanding the exploration of the pretrained LLM. The judge gets tweaked slightly to encourage the LLM to explore those kinds of pathways, and/or the LLM gets SFT'd with reasoning data (which is very uncommon in its PT dataset).
I think this breakdown and stepping back is important so that we can see what we're really trying to do here: get a limited Generator LLM to find its way around areas it can't climb. It is likely true that there is _always_ some path from a given point Q and set A that a limited Generator LLM can safely traverse, even if that means those paths are very long.
It's not easy for researchers to know what paths the LLM can safely travel. So we can't just look at Q and A and build a nice dataset for it. It needs to generate the paths itself. And thus we arrive at Reasoning.
Reasoning allows us to take a limited, pretrained LLM, and turn it into a little path finding robot. Early during RL it will find really convoluted paths to the solution, but it _will_ find a solution, and once it does it gets a reward and, hopefully, as training progresses, it learns to find better and shorter paths that it can still navigate safely.
But the "reasoning" component is somewhat tangential. It's one approach, probably a very good approach. There are probably other approaches. We just want the best ways to increase exploration efficiently. And we're at the point where existing written data doesn't cover it, so we need to come up with various hacks to get the LLM to do it itself.
The same applies to jokes. Comedians don't really write down every single thought in their head as they come up with jokes. If we had that, we could SFT existing LLMs to get to a working solution TODAY, and then RL into something optimal. But as it stands PT LLMs aren't capable of _exploring_ the joke space, which means they never come out of the RL process with humor.
Addendum:
Final food for thought. There's kind of this debating going on about "inference scaling", with some believing that CoT, ToT, Reasoning, etc are all essentially just inference scaling. More output gives the model more compute so it can make better predictions. It's likely true that that's the case. In fact, if it _isn't_ the case we need to take a serious look at our training pipelines. But I think it's _also_ about exploring during RL. The extra tokens might give it a boost, sure, but the ability for the model to find more valid paths during RL enables it to express more of its capabilities and solve more problems. If the model is faced with a sheer cliff face it doesn't really matter how much inference compute you throw at it. Only the ability for it to walk around the cliff will help.
And, yeah, this all sounds very much like ... gradient descent :P and yes there have been papers on that connection. It very much seems like we're building a second layer of the same stuff here and it's going to be AdamW all the way down.
> The truth is like 10 or 20 thousand lines of game logic can make a lot of games, and that's really not much code to port to your own game engine compared to the rest of the engine.
I intuitively want to agree, but on the other hand I've also seen many, many hobby/indie/etc projects deadlock when they switched engines. Or even engine versions (Unreal 4 -> 5).
The trash talk isn't the issue. When I, a male, join a multiplayer session I'm largely ignored. If I'm talked to, yes it's usually to trash talk me.
But when a female joins a session, they become the center of attention. They can't simply play the game; everyone is watching them and wanting to interact with them. In addition to that, the conversations from other players isn't just trash talk, it usually includes a heavy mix of flirting and sexual advances. Not funny, trolly, trash talk sexual advances. But intentional advances.
Not to mention the stalking outside of that session that females can experience.
The vast majority of people joining a multiplayer session are there to relax and have fun. Not get hit on by strangers on the internet, be the center of attention, or be open to harassment even after the session.
> Why Bose doesn't let you pair them using the standard system is beyond me.
The app collects data on what music you listen to and sends it back to Bose. So obviously they want to encourage you to install and use the app.
Last I checked, at least. Also, there are _two_ Bose apps now, and which one you use depends on which Bose QuietComfort headphones you have. That I have no explanation for.
That line of reasoning just boils down to "it brings one further from God", and it has been applied to everything that the church wishes to demonize. But that reasoning is always applied backwards. People aren't starting from "What would bring us further from God?" and deducing their way to psychedelics and other things frowned upon by the church. It happens the other way around. The church says that certain things are bad, and then use lines of reasoning like "it brings one further from God" after the fact to justify the arbitrary decision.
So I suppose I take issue with it being called "reasoning" when really it's an after the fact justification with no bearing on reason whatsoever.
It's also interesting that a lot of religions use psychedelics to get _closer_ to God(s), and the "reasoning" there is equally as sound.
Here's a great youtube video which documents a case of methanol poisoning and how it was treated (it goes into a good amount of interesting details): https://www.youtube.com/watch?v=4DQUrg0Yhu4
> Remove those dangers, and remove most of the struggle (although we will always re-define what struggle is) and we're a bit like fish out of water. We're still trying to adapt.
I suppose this is a bit like that one theory of allergies and autoimmune disorders; they may be caused because we aren't exposed to parasites anymore. That part of the immune system thus never gets trained and can sometimes go haywire.
I hadn't worked with payment systems for a number of years, but recently added Stripe to a quick MVP I built (bitcoinvoice.com). It took two hours. I couldn't believe how easy it was, and how nice the UX of the whole thing was.
I imagined the usual song and dance of writing a bunch of back-end code to handle all their weird requests, worrying about double checking everything in the requests to make sure nobody can spoof them, etc, etc. Ya know, the dumb stuff that takes hours to write, hours to test, and days to wait for support to get back to you because their test servers are broken.
And then I imagined waiting days for them to approve my account for live transactions, requiring scanned copies of my driver's license with me flashing some sort of gang sign while reciting page 452 of the 1993 re-print of Moby Dick.
Nope. Turned out they have a synchronous API for processing payment so I don't need to handle callbacks on the backend. Turned out the front end code was, yeah about 7 lines (more with a few tweaks). And their documentation injects your keys into the sample code for you (wow)! On top of all that ... they had Go sample code!
And when I went to enable my live account, I'm pretty sure all they asked for was name and address, maybe TIN. Nothing else. No wait. Just ... bam, I was enabled and could start accepting live payments.
It was perhaps the most pleasant API integration I've ever done.
There are _some_ rough spots. They don't really explain Radar, their fraud protection very well. It wasn't clear to me if it was automatically included and handled. shrugs. And though they advertised support for Bitcoin payments, it turns out you have to use their async API to accept Bitcoin payments. I was willing to accept zero-conf payments, so I figured I could just keep using their synchronous API, but I guess not.
If you first carefully define the overall architecture and thus individual high level components of the system, then you know which of those components are mission critical and which are commodity. Mission critical would be anything ensuring ACID, etc. That way, no matter what you farm out to LLMs, you can keep the majority of limited human focus on the far fewer mission critical components. If tests end up not being robust enough to catch all issues, at least they'll be isolated to commodity code where damage is limited to things like DoS, etc, and not code that could cause data loss.
I also think it's important to first define the _contracts_ on and between each of these components, and derive tests from those contracts. Partly because contracts more succinct and easier to reason about. And partly because Rust provides many tools to enforce contracts at compile time, reducing the need for tests (which themselves could end up subtly flawed). Contracts can be enforced through typing, private vs public APIs, etc. Newtypes are _incredibly_ powerful for both enforcing contracts and making footguns much less likely.