[email protected]: Hydraulic Conveyor makes distributing desktop apps as easy as a web app. Use it to bring your mobile apps to the desktop and avoid the need to maintain a web version, to use your language and toolkits of choice or to do things that you can't do in the browser. Integrated support for Electron, Flutter, JVM and native apps.
https://www.hydraulic.dev
--
[email protected]: Personal email. I've worked on anti-automation, was an early Bitcoin developer and then designed the Corda enterprise blockchain platform and the Conclave toolkit for writing SGX enclaves.
Both EU referendums were about whether to leave. Labour took Britain in without a referendum and only held one on reversing it due to the outrage that caused.
The Internet Watch Foundation is one of these NGOs that makes lists of hashed child porn images and URLs. All the big tech companies subscribe so they can coordinate on blocking child porn, as they are legally required to do.
Unfortunately the IWF is also a "charity" and thus engages in political lobbying. Because like all such NGOs they have a single purpose, they lobby for making that purpose easier irregardless of other costs, which they view as out of scope. It's obviously easier to watch the internet if tech firms are forced to watch everything all the time, so that's what they're in favour of.
People actually working at tech firms on messaging systems don't want to do this, however. So they end up funding people who are undermining their own policies. This is very common whenever NGOs get involved e.g. governments funding NGOs that directly undermine the government's own efforts.
The fix would be for tech firms to leave the IWF and set up their own alternative organization that doesn't engage in lobbying activity. However, that would require a lot of cross-org agility that is difficult for big companies to achieve even internally, let alone across the industry, and the leadership is all thinking about AI anyway not EU stuff where they already just assume the EU is going to regulate them all the death anyway. So inertia carries the day.
It's not that weird. There are other cases like it. Jacques Baud is a Swiss guy and was sanctioned by the EU Commission for "spreading Russian propaganda". At the time he was living in Brussels and the EU became a prison that he's not allowed to leave.
The sanctions imposed on December 15 by the European Union against Jacques Baud and eleven other people include the freezing of their assets, a ban on doing business and bans on entering the EU.
“I don’t have the right to return to Switzerland, or even to travel within the EU. I’m essentially being held against my will,” says Jacques Baud.
Around ten years ago I watched some MEPs on YouTube talk about their jobs. A lot of them were scathing. The EP isn't a real parliament and the people in it don't have any real power, so it fills up with cheerleaders and hecklers.
Attendance isn't just low before the summer break, it's low all the time. MEPs miss votes for reasons as trivial as there being a football game on, because why not? It's not like their votes matter much anyway. Everything important is decided upon long before they get involved, they aren't allowed to actually write laws or pass them or repeal them, and so nobody with any political ambition goes there unless they're using it as a springboard to national politics.
> "Mr President, our parliament in the United Kingdom sat for 142 days last year and it was criticized widely for only sitting for that amount of time yet we in Brussels and Strasburg in the plenary and mini plenary, we sit for just five days a month and on three of those days it's only part of a day that we actually sit for. So we end up with debate in this chamber where microphones of speakers are cut off after 60 seconds, 90 seconds or 2 minutes, and we have no way of expanding on a point, we have no way of properly challenging a speaker, we have no means of proper scrutiny of proposed legislation that comes through. Surely it would make more sense to have sufficient time allocated to proper debate to reason debate and those who actually believe in the structures and institutions of this place should surely welcome that. I think my 60 seconds is up."
You can tell it's a joke institution because they regularly penalize MEPs for the content of their two minute speeches. Britain has the concept of parliamentary immunity but the EU does not, so you get "politicians" who are told what they can and cannot say by the leaders of other parties.
They do claim to be shipping new features to their acquired apps. Look at their website. It's got lists of such things.
The steelman case for this is something like, mature apps that found product market fit are often over-staffed and doing a lot of duplicated work. You could get five of them together and consolidate their infrastructure/code to reduce costs, and have generalist devs who can work on any of those codebases. Then you need fewer people.
So this isn't an irrational thing to do. It's commonly done by firms like Google or Meta where they buy a small company and then rewrite it onto their own infrastructure to reduce costs. Sometimes the engineers are reallocated to other projects, or things drift and there are eventually layoffs. Google bought DoubleClick and then laid off 50% of the staff! Twitter didn't consolidate products but was clearly overstaffed, nobody imagines that Twitter was unique.
So the bull case for this is that it's finding efficiencies. The apps may not be the shiniest hottest things anymore, but they can still live on and be maintained if they're run more efficiently as a business. And yes this may involve layoffs or price rises, as often software startups hopelessly misprice their product and prefer to burn VC money than lose users or colleagues. Managers who aren't emotionally attached to the product or company can correct this, putting it on a long term stable path. That may suck for the user but probably sucks less than the company being under, or being acquihired and the product totally shut down.
TPMs were moved into the Northbridge (I think) many years ago, at least on Intel systems.
PC hardware still isn't tamper resistant though. Memory may be encrypted but its contents can be tampered with in other ways. Only SGX made a serious attempt to be tamper resistant, although Intel eventually sacrificed that to boost performance.
A game engine has a vast space of possible outputs, all of which will be considered good enough.
Its primary output is pixels and sound. Those are hard to test in a reasonable way. Screenshot testing is useless in a codebase where most of the changes are about making the pixels prettier.
Ah, OK. I called her a liar not because of a dispute over what exactly transformers are doing inside, but because in TFA we see:
> What are the most common misconceptions about the “stochastic parrots” metaphor?
Bender: I think one of the biggest ones is, “Bender says AI is a stochastic parrot.”
But in the paper itself we see her say exactly that, several times. What she's trying to do now is wordsmith out of it by claiming that in her world LLMs are totally unrelated to AI, so when she said LLMs are stochastic parrots she wasn't making a claim about AI.
Nobody else defines AI to exclude LLMs, nor did they at the time, and it wasn't the core of the argument she made either. But then she admits that the paper does "generalize towards AI" at the end. So... whatever.
It's morally important to reject this stuff. When academics play word games it devalues all institutional output.
You probably don't need a geometric map. Just have someone wander around with a mobile app and feed the video into a more powerful model once, asking it to produce descriptions of the different areas of the office or building and how they connect. Now you have a "text adventure game" map you can use it with a small LLM to produce instructions for the robot to follow, assuming it knows where it currently is.
The advantage over traditional approaches is presumably flexibility. LIDAR isn't going to solve an instruction like "find the man with the pink shirt".
Interestingly the paper (I finished scan reading it now) does say that the models can move data from their 'automatic' circuits into the J-space if they need to reason on it reflectively. So in some sense LLMs actually can access their own vectors, at least some of the time.
It requires some level of semantic understanding, like what a paren is and what it means to balance them.
The issue in this discussion is that "predict the next token" is a problematically reductive description of what's going on. It's like saying compilers are programs that emit bytes or that humans are mammals that make sounds. It's not strictly false but it's not capturing the depth of what's happening either.
A simple way to see this is to ask: predicting the next token of what? The obvious answer - predicting the next token that would be found in the training set - isn't correct. If that's what it were doing then it would yield no prediction or random predictions for any prefix not found in that training set, but it isn't what happens. We see generalization and reasoning. They can answer questions never asked before. And once post-training kicks in the question of what it's predicting becomes even harder. It becomes more like predicting what this specific AI assistant would say next, which is a circular definition.
I think you're right that the idea of looping layers is unique to you, congrats and thanks for writing those great blog posts (I read them for the first time a few days ago!). But the idea that the thinking is happening in an abstract space via neural circuits in the middle layers I feel was one that I was reading about in 2024 at least, as Anthropic have been doing this kind of research for a long time. Maybe I'm misremembering though!
My impression from reading the literature is that there are a gazillion interesting ideas and findings published that nobody is picking up in production models. The big labs are researcher constrained, there just aren't enough hours in the day to keep up with the literature and integrate all the interesting ideas found there. So it's not surprising that your trick still works. It'd be even less surprising to discover nobody at these labs has read your blogs, or they have but never found time to experiment with them. Or, they tried, but there is no set of loops that improves some metrics without harming others - I would expect neural circuits to be misaligned across the middle layers so looping layers for one task would put a fault line in circuits for other tasks.
Then they have to trade off the extra GPU capacity needed to do the extra layers, and so on.
My interpretation of prior mechanistic interpretability research is that LLMs transform concepts in non-verbal spaces as well, which is why that was a surprising statement. For example showing how they do arithmetic on line lengths by rotating helical manifolds.
And the way they transform data isn't by transforming words. The layers transform high dimensional vectors - a format very alien to us. It's not obvious that these vectors must encode concepts from the vocabulary.
Edit: the paper claims that it's only J-space concepts that need to map to English words, other forms of cognition that are more 'practiced' and don't require so much reasoning bypass the J-space and can work in non-verbal subspaces. So that's the answer.
Very nice. These papers are always so great. If anyone from Anthropic is reading or really anyone with AI research background I'd love some input on these thoughts:
> The result serves as a corroboration of the workspace account, that the representations used for verbal report are the same ones that govern how the model silently reasons.
This sounds suspiciously saying the models must follow the strong Sapir-Whorf hypothesis. Can that really be true, given that humans don't?
Other misc observations:
• The slice explorer indicates Claude really likes Python to an overwhelming extent. Or at least it expects people who ask for help in programming to use Python. Given the prompt "Please help me understand this code: " at the colon its thoughts are completely dominated by Python and no other language. Does this say something about the training set, or about the fact it's popular with beginners?
• Claude also really loves Reddit. Its thoughts at many points include Reddit for no obvious reason. Again this must be due to the training set. Are documents presented to Claude with attribution during pre-training, leading to conversations being dominated by Redditness? If so this is kind of a scary alignment problem all by itself given how censored and extremist Reddit can be.
• The early layers almost always decode to the same set of religion related tokens, like "Biserica" (the Romanian word for church) and "Freguesias" (parishes in Portugal). What's up with that? I guess it's some sort of zero initialization that gets mapped to some arbitrary token space because in the early layers the J-space is empty?
• Now the J-space is interpretable, does this make "neuralese" or layer looping less dangerous? Will we see reasoning tokens and summaries disappear in favour of pure residual based thinking?
• Earlier papers have claimed that different languages map to a shared set of abstract concept vectors, but this paper says the Claude models think natively in English. What explains this disagreement?
They're definitely willing to consider it. Read the parts of the paper where they use J-space interpretations to train more ethical behavior into the model by interrupting it.
Nah, it's a cool blog post especially as it was real AI research done at home (albeit with a ridiculously expensive PC), but Anthropic and other labs have been investigating this kind of thing for years.
Even the original transformer architecture makes this clear. It had an explicit "encoder" phase and then a "decoder" phase. Modern LLMs collapse the two together, or are sometimes described rather confusingly as being decoder only. But what they're doing is more or less the same.
AWS isn't a utility because utilities sell commodities for which you can switch to a competitor easily, as you're buying the same thing. There are companies selling similar products to AWS but none that sell AWS itself.
ROI lags. If they laid off 90% of the Xbox division and left only enough to keep the lights on (they don't run their own factories I guess), then they'd still get nearly the same amount of income the next year with far lower costs. Of course revenue would start to sharply decline as the number of games produced was much lower, as the tech became obsolete. But it's not like revenue would evaporate instantly.
So they could do that and invest the income they still get into bonds, and then that would be where the returns come from.
They don't want to do that obviously, hence why Xbox needs reform.
> this line is directly cited from another paper (pay attention - it's from july 2020)
It's her own paper, she's citing herself. And the full sentence is "As we discuss in §5, LMs are not performing..." so she's actually just teeing up the same claim she's made previously for further discussion. Why are you trying to claim I'm misrepresenting her words?
> did you read the article after that line or was there a shortage of attention span??
I did. It's more of the same, so there's nothing to say about it.
[email protected]: Hydraulic Conveyor makes distributing desktop apps as easy as a web app. Use it to bring your mobile apps to the desktop and avoid the need to maintain a web version, to use your language and toolkits of choice or to do things that you can't do in the browser. Integrated support for Electron, Flutter, JVM and native apps.
https://www.hydraulic.dev
--
[email protected]: Personal email. I've worked on anti-automation, was an early Bitcoin developer and then designed the Corda enterprise blockchain platform and the Conclave toolkit for writing SGX enclaves.