HackerTrans
TopNewTrendsCommentsPastAskShowJobs

seddonm1

no profile record

Submissions

Show HN: Using LLMs and differential testing to convert code

reorchestrate.com
2 points·by seddonm1·5 miesięcy temu·0 comments

Show HN: Your binary is no longer safe

reorchestrate.com
3 points·by seddonm1·5 miesięcy temu·0 comments

Bringing a Warhammer to a Knife Fight

reorchestrate.com
2 points·by seddonm1·5 miesięcy temu·1 comments

comments

seddonm1
·w zeszłym miesiącu·discuss
Based on the now-deprecated Clear Linux it does seem that these optimizations add up [0] and so maybe we should be considering them more broadly?

[0] https://www.phoronix.com/review/clear-linux-48p-ubuntu/6
seddonm1
·w zeszłym miesiącu·discuss
I would be interested to know if there is a method similar to this one in Rust [0] that allows a single binary to support multiple optimization levels depending on the executing CPU? It feels wasteful to not enable these optimizations but I don't really want to force a user to choose between a complex feature matrix.

[0] https://github.com/ronnychevalier/cargo-multivers
seddonm1
·4 miesiące temu·discuss
I am getting closer and closer to a full verified rewrite in Rust. I have also moved to a much easier sqlite relational structure for the backend.

I actually sidestepped the annoying btrieve problem by exporting the data using a go binary [0] and I write it to a sqlite instance with raw byte arrays (blobs). btreive is weird because it has a dll but also a a service to interact with the files.

P.s. I have spent a lot of hours on this mainly to learn actual LLM capabilities that have improved a huge amount in the last year.

[0] https://github.com/barchart/go-btrieve
seddonm1
·4 miesiące temu·discuss
It’s a real problem. I threw it at an old MUD game just to see how hard it is [0] then used differential testing and LLMs to rewrite it [1]. Just seems to be time and money.

[0] https://reorchestrate.com/posts/your-binary-is-no-longer-saf...

[1] https://reorchestrate.com/posts/your-binary-is-no-longer-saf...
seddonm1
·5 miesięcy temu·discuss
No, I would need to find a binary to test on. I suspect it would produce horrible code at the decompiler layer but ultimately I would expect that function signatures are still relatively clean?

Its scary - once you get the differential testing harness set up it seems to be just a matter of time/tokens for it to stubbornly work through it.
seddonm1
·5 miesięcy temu·discuss
I have tried to post this here but it has not got traction.

I have a demonstrated process here on my blog (all hand written without AI).

This bit about how to brute force decompilation: https://reorchestrate.com/posts/your-binary-is-no-longer-saf...

And this about how to do the conversion and address the LLM hallucination problem: https://reorchestrate.com/posts/your-binary-is-no-longer-saf...

Yes, it is absolutely possible.
seddonm1
·5 miesięcy temu·discuss
I am applying differential/property based testing to all the side effects of functions (mutations) and return values. The rust code coverage is also used to steer the LLM as it finds discrepancies in side effects.

It is written up in my link - please bear in mind it is really hard to find the right level to communicate this level of detail at - so I'm happy to answer questions.
seddonm1
·5 miesięcy temu·discuss
My test harness loads up the original DLL then executes that in parallel against the converted code (differential testing). That closes the feedback loop the LLM needs to be able to find and fix discrepancies.

I'm also doing this on an old Win32 DLL so the task is probably much easier than a lot of code bases.
seddonm1
·5 miesięcy temu·discuss
I delivered a talk at Rust Sydney about this exact topic last week:

https://reorchestrate.com/posts/your-binary-is-no-longer-saf...

I am able to translate multi-thousand line c functions - and reproduce bug-for-bug implementation
seddonm1
·5 miesięcy temu·discuss
Hi Ben. I published an article about this problem this week (and did a talk at Rust Sydney).

What you need is differential, property testing. I’m sure it would work for you (you can skip the first half as you already have the source):

https://reorchestrate.com/posts/bringing-a-warhammer-to-a-kn...
seddonm1
·5 miesięcy temu·discuss
What about s3 stored in SQLite? https://github.com/seddonm1/s3ite

This was written to store many thousands of images for machine learning
seddonm1
·5 miesięcy temu·discuss
A process for using LLMs to do brute-force decompilation of binaries and conversion to another programming language - including testing to prove equality.

This process is targeting an old computer game but there is nothing preventing this being run on any binary.