Why don't we just ask AI to write assembler?
18 comments
Programming languages are not just for ergonomics. They are valuable abstractions that help us reason. And they also help LLMs reason in the same manner.
Two reasons.
First reason, LLMs are modeled from what humans have been doing, and the have been writing software that way recently so it's easier to mimick that to get straight to results. This reason might fade away in the future.
Second reason, something related to impedance (mis)match, a signal processing notion (when the interface between two media is not well-suited, it is difficult to have a signal pass through).
Going through intermediate levels makes a structured workflow where each steps follows the previous one "cheaply". On the contrary, straight generating something many layers away requires juggling with all the levels at once, hence more costly. So "cheaply" above both means "better use of a LLM context" but also use regular tools where they are good instead of paying the high price (hardware+computation+environment) of doing it via LLM.
Interestingly, AIs are used to generate sample-level audio and some video, which may look like it contradicts the point. Still they are costly (especially video).
First reason, LLMs are modeled from what humans have been doing, and the have been writing software that way recently so it's easier to mimick that to get straight to results. This reason might fade away in the future.
Second reason, something related to impedance (mis)match, a signal processing notion (when the interface between two media is not well-suited, it is difficult to have a signal pass through).
Going through intermediate levels makes a structured workflow where each steps follows the previous one "cheaply". On the contrary, straight generating something many layers away requires juggling with all the levels at once, hence more costly. So "cheaply" above both means "better use of a LLM context" but also use regular tools where they are good instead of paying the high price (hardware+computation+environment) of doing it via LLM.
Interestingly, AIs are used to generate sample-level audio and some video, which may look like it contradicts the point. Still they are costly (especially video).
Dave Plummer claims to have successfully generated working executable PE binaries using ChatGPT.
https://x.com/davepl1968/status/2044482592620351955
https://x.com/davepl1968/status/2044482592620351955
IF you don't check the code, you can't navigate correctly the UI. The AI is non-deterministic, and you can never be sure, that it will produce the same quality and use the same approach everytime. Even, with all the rules, lessons learned, documentation, a code review afterwards will find something. With that in mind, in order to write an assembler program, you need someone to teach the model how to do it. And in order to teach it, you need to be able to read what's generated ;)
If you are a senior developer [1] responsible for delivering projects where you have to delegate to mid level ticket takers, you have to deal with developers who are also non deterministic and you can never trust their quality.
Hell my coding is non deterministic with different degrees of quality depending on what else I have going on.
But just like a developer, an LLM can also reason over intent based on clearly named functions, modularity, etc.
[1] if someone is pulling well defined tickets off the board. They are a mid level developer regardless of title.
Hell my coding is non deterministic with different degrees of quality depending on what else I have going on.
But just like a developer, an LLM can also reason over intent based on clearly named functions, modularity, etc.
[1] if someone is pulling well defined tickets off the board. They are a mid level developer regardless of title.
LLM cannot reason about anything.
It can provide text that can be plausibly interpreted as reasoning by someone reading that text.
When human provides a plausible explanation then it means they either had someone else provide it to them or they actually understand the issue.
LLM cannot understand anything, it can only provide output based on the training data where similar input has likely to have produced similar output in the past.
Human can tell you they don't understand or don't know something but LLM is unlikely to have training that will produce that kind of output, it is more likely you will always get something that looks correct but it might not be.
LLM can automate a part of the process where human might take slightly but, ultimately, any output generated by LLM cannot be trusted and should be checked by human that understands the issue...and that is actually the hard part where humans will struggle so they won't actually do it.
When human is producing the output that human is performing the following actions: -analysing the issue -analysing the exiting process -building the understanding of the existing process -building the understanding of how issue affects the existing process -producing the output to address the issue in the existing process -checking the output as it is being produced -updating the understanding of the existing process with lessons learned from the above -checking the final product to ensure that it has solved the original issue and hasn't broken some other part of the system
LLM can help speed up one of those steps (producing the output) at the expense of slowing down the other parts (which were already slow) and reducing the understanding and reliability of the existing system which will make future iterations even slower.
LLM can be used to speed up the generation of examples but just like in the past you could not just copy the example from some random internet search result, you should not just copy the LLM output without understanding it...and that is the slow part where LLM might not help (and might actually make worse) for most people.
And when in the past you encountered comprehensive and well documented output you could assume human that put that amount of effort actually understood what they were doing and wouldn't have expended that much effort to generate garbage, you cannot make that same assumption now with LLMs.
LLM can automate a part of the process where human might take slightly but, ultimately, any output generated by LLM cannot be trusted and should be checked by human that understands the issue...and that is actually the hard part where humans will struggle so they won't actually do it.
When human is producing the output that human is performing the following actions: -analysing the issue -analysing the exiting process -building the understanding of the existing process -building the understanding of how issue affects the existing process -producing the output to address the issue in the existing process -checking the output as it is being produced -updating the understanding of the existing process with lessons learned from the above -checking the final product to ensure that it has solved the original issue and hasn't broken some other part of the system
LLM can help speed up one of those steps (producing the output) at the expense of slowing down the other parts (which were already slow) and reducing the understanding and reliability of the existing system which will make future iterations even slower.
LLM can be used to speed up the generation of examples but just like in the past you could not just copy the example from some random internet search result, you should not just copy the LLM output without understanding it...and that is the slow part where LLM might not help (and might actually make worse) for most people.
And when in the past you encountered comprehensive and well documented output you could assume human that put that amount of effort actually understood what they were doing and wouldn't have expended that much effort to generate garbage, you cannot make that same assumption now with LLMs.
Again this is not true with a real world recent example.
For context: for the project I’m about to describe, I did the 3 week discovery process where I iterated through the design. I designed the architecture from an empty AWS account with IAC and an empty git repo. I know every decision that was made and why.
An issue was reported while the client was testing - a duplicate message was displayed to the user.
I gave codex three pieces of information - the duplicate IDs and told it was duplicate.
Codex:
1. Created and ran a query in the Postgres database after finding the ARN to the credentials - you don’t have to pass credentials to the database in AWS, you pass the entry in Secrets Manager directly to the database as long as you have permission to both (Dev account). I didn’t tell it the database and queried where I was storing the event.
2. It found the lambda that stored the events in the database.
3. It looked at the CloudFormation template to figure out the Lambda was triggered by messages in an SQS queue
4. Looking at the same template it saw that the SQS message was described to an SNS topic
5. It found the code that sent the events - a 3000 line lambda
6. It was able to explain what the lambda did and find there wasn’t a bug in the logic
7. It saw that the flow was data driven and got the information from a DDB table defined by an environment variable.
8. It then looked at that CloudFormation template that deployed the Lambda
9. It ran a query on the DDB table after looking at that CloudFormation template to figure out the schema
It then told me that there was a duplicate entry in the database.
I knew the entire structure of the system - again I designed all of this myself. I wanted to see how codex would do.
Everything you are saying a modern LLM can do.
I won’t even go to how well it debugged a vibe coded internal website just by telling it to use Docker container with headless chromium and Playwright. It debugged it by taking screenshots while navigating and making changes.
For context: for the project I’m about to describe, I did the 3 week discovery process where I iterated through the design. I designed the architecture from an empty AWS account with IAC and an empty git repo. I know every decision that was made and why.
An issue was reported while the client was testing - a duplicate message was displayed to the user.
I gave codex three pieces of information - the duplicate IDs and told it was duplicate.
Codex:
1. Created and ran a query in the Postgres database after finding the ARN to the credentials - you don’t have to pass credentials to the database in AWS, you pass the entry in Secrets Manager directly to the database as long as you have permission to both (Dev account). I didn’t tell it the database and queried where I was storing the event.
2. It found the lambda that stored the events in the database.
3. It looked at the CloudFormation template to figure out the Lambda was triggered by messages in an SQS queue
4. Looking at the same template it saw that the SQS message was described to an SNS topic
5. It found the code that sent the events - a 3000 line lambda
6. It was able to explain what the lambda did and find there wasn’t a bug in the logic
7. It saw that the flow was data driven and got the information from a DDB table defined by an environment variable.
8. It then looked at that CloudFormation template that deployed the Lambda
9. It ran a query on the DDB table after looking at that CloudFormation template to figure out the schema
It then told me that there was a duplicate entry in the database.
I knew the entire structure of the system - again I designed all of this myself. I wanted to see how codex would do.
Everything you are saying a modern LLM can do.
I won’t even go to how well it debugged a vibe coded internal website just by telling it to use Docker container with headless chromium and Playwright. It debugged it by taking screenshots while navigating and making changes.
Layers of abstraction remain effective and valuable. Why reinvent state management, for example, with each application?
Runtime also matters; you can’t run assembly on the web.
Security mechanisms can also preclude assembly.
Etc.
FWIW, your question stopped short before the bottom turtle in the stack. Below assembly is machine code. So your question could rather be, why not emit machine code. Assembly is made for humans because we can understand it, but machine code is not really tractable for humans to engage with in a meaningful way.
Runtime also matters; you can’t run assembly on the web.
Security mechanisms can also preclude assembly.
Etc.
FWIW, your question stopped short before the bottom turtle in the stack. Below assembly is machine code. So your question could rather be, why not emit machine code. Assembly is made for humans because we can understand it, but machine code is not really tractable for humans to engage with in a meaningful way.
That would be quite expensive in terms of time and token use. It would need to be tested, and you’d have so many repetitive tests you might as well encode the behavior they expect in generators of blocks of assembly, i.e. higher level languages and compilers.
I wish I could test that and mass port from c++ to plain and simple C.
Any 'public' (rate limited) web API (using CURL) from current AI inferences services?
Any 'public' (rate limited) web API (using CURL) from current AI inferences services?
Too many tokens, and high-level languages have better semantics, making them more suitable for reasoning.
Because there’s not enough learning material? Like most of the code LLMs have stolen for training is highlevel code, not assembly
AI doesn't actually know anything, just predicts, and as such most training data is in high level languages.
[deleted]
Ai are are not smart enough for that, its not real ai neither to do it
> If we generate so much code using AI that no one is really looking or reading the code anymore, just verifying end functionality, we can really just skip all that and go straight to assembler, no?
We could also just autogenerate the content of our websites, emails, contracts.
And we do, resulting in mountains of slop, varying from soulless to wildly incorrect.
Code is a precise way to describe intent. Using LLMs make up some of the intent results in the author not knowing what the precise functionality of the resulting code is.
The companies selling LLM services present this as magic which will magically do what the author wants it to do, without even the author themselves knowing or defining it.
In reality it is simply ignorance and lies.
Sorry we can’t wishful think good working software into existence.
We could also just autogenerate the content of our websites, emails, contracts.
And we do, resulting in mountains of slop, varying from soulless to wildly incorrect.
Code is a precise way to describe intent. Using LLMs make up some of the intent results in the author not knowing what the precise functionality of the resulting code is.
The companies selling LLM services present this as magic which will magically do what the author wants it to do, without even the author themselves knowing or defining it.
In reality it is simply ignorance and lies.
Sorry we can’t wishful think good working software into existence.
If we generate so much code using AI that no one is really looking or reading the code anymore, just verifying end functionality, we can really just skip all that and go straight to assembler, no?
Sure, we could reuse some basic building blocks like implementations of the tcp/ip protocol, http, sockets etc but server frameworks like fastapi are just human friendly abstractions over all that.