HackerTrans
TopNewTrendsCommentsPastAskShowJobs

jakozaur

5,629 karmajoined il y a 18 ans
Email: jacek (at) migdal.pl

Submissions

Make Europe Cool Again

jacek.migdal.pl
5 points·by jakozaur·hier·0 comments

We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them

quesma.com
245 points·by jakozaur·il y a 5 mois·98 comments

OTelBench: Can AI instrument OpenTelemetry?

quesma.com
5 points·by jakozaur·il y a 6 mois·1 comments

Nano Banana Pro: raw intelligence with tool use

quesma.com
4 points·by jakozaur·il y a 8 mois·0 comments

A postmortem on our $2.5M database gateway: lessons from pilot purgatory

quesma.com
6 points·by jakozaur·il y a 8 mois·0 comments

The security paradox of local LLMs

quesma.com
160 points·by jakozaur·il y a 9 mois·87 comments

Local LLMs are worse for security

quesma.com
1 points·by jakozaur·il y a 9 mois·0 comments

CompileBench: Can AI Compile 22-year-old Code?

quesma.com
148 points·by jakozaur·il y a 10 mois·65 comments

Winners of OpenAI GPT-OSS-20B Red‑Teaming Challenge

kaggle.com
1 points·by jakozaur·il y a 10 mois·0 comments

comments

jakozaur
·il y a 15 jours·discuss
It's rather hard to do at the proxy level with agentic coding, such as Claude Code or similar. These are long-chained sessions of tool use that heavily rely on prompt caching. Changing mid-flight is costly.

It looks like much more context is required to decide on the best model (e.g., summarizing logs might use a cheap model, whereas you likely want Opus/Mythos/GPT 5.6 to debug multithreading logic). In an agentic system, a decision about the model may be embedded in the decision to orchestrate the model.
jakozaur
·il y a 2 mois·discuss
Poland's first partly free election was on 4 June 1989, preceded by the roundtable negotiations.

The protests in Czechoslovakia came later, called the Velvet Revolution, from 17 to 28 November 1989. In June 1990, Czechoslovakia held its first democratic elections, a year after Poland.

Poland paved the way for the whole of central and eastern Europe. The Round Table produced the negotiated-exit template that Hungary built on in its own talks that summer, and that Czechoslovakia, East Germany, and the Baltics drew on as their regimes fell within months.

And it did so from the deepest macroeconomic crisis of any of the satellite states: hyperinflation running into the hundreds of percent by late 1989, an unresolved sovereign default from 1981, and chronic shortages.

Since then Poland has converged fastest of any of them. From a low base it has climbed to the upper-middle of central and eastern Europe by GDP per capita PPP, overtaken Hungary, and is now closing on Czechia and Slovenia.
jakozaur
·il y a 2 mois·discuss
The story is longer: Poland was the first country to make a remarkable peaceful transition from a bankrupt, failed Soviet satellite state. The shock therapy, plus NATO and EU aspirations, paved the way.

It is a story of a country that made a lot of the right decisions along the way. Managed to keep consistent high growth, not a pony trick or boom/bust mode.

Poland should be a role model for many other countries.

Recommend a book: https://www.amazon.com/Europes-Growth-Champion-Insights-Econ...

And Noah's blog post: https://www.noahpinion.blog/p/the-polandmalaysia-model
jakozaur
·il y a 2 mois·discuss
A heat pump could win as the best HVAC technology, though a better drilling for ground-sourced ones. Just a shallow drilling (up to 100m) that works in retrofit mode, such as drilling from the basement, would be a great upgrade:

- No outdoor unit that looks awful in many settings

- works well, even in the coldest winter, without a spike in electricity usage, COP 5

- very reliable with long durability

- super quiet, no ambient noise

- 20% more efficient

Currently, drilling is very disruptive in retrofits, but there is progress in compact techniques that might change the equation.

Disclaimer: angel investor in https://www.flexdrill.at/
jakozaur
·il y a 4 mois·discuss
Build systems are tested by CompileBench (Quesma's benchmark).

Disclaimer: I'm the founder.
jakozaur
·il y a 5 mois·discuss
I can provide trajectories. Though probably we are not going to publish them this time. This would need some extra safeguards.

Email me. The address is in profile.
jakozaur
·il y a 5 mois·discuss
As I do eval and training data sets for living, in niche skills, you can find plenty of surprises.

The code is open-source; you can run it yourself using Harbor Framework:

git clone [email protected]:QuesmaOrg/BinaryAudit.git

export OPENROUTER_API_KEY=...

harbor run --path tasks --task-name lighttpd-* --agent terminus-2 --model openrouter/anthropic/claude-opus-4.6 --model openrouter/google/gemini-3-pro-preview --model openrouter/openai/gpt-5.2 --n-attempts 3

Please open PR if you find something interesting, though our domain experts spend fair amount of time looking at trajectories.
jakozaur
·il y a 5 mois·discuss
So many models refuse to do that due to alignment and safety concerns. So cross-model comparison doesn't make sense. We do, however, require proof (such as providing a location in binary) that is hard to game. So the model not only has to say there is a backdoor, but also point out the location.

Your approach, however, makes a lot of sense if you are ready to have your own custom or fine-tuned model.
jakozaur
·il y a 5 mois·discuss
Oh, nice find... We end up using PyGhidra, but the models waste some cycles because of bad ergonomics. Perhaps your cli would be easier.

Still, Ghidra's most painful limitation was extremely slow time with Go Lang. We had to exclude that example from the benchmark.
jakozaur
·il y a 5 mois·discuss
See direct benchmark link: https://quesma.com/benchmarks/binaryaudit/

Open-source GitHub: https://github.com/QuesmaOrg/BinaryAudit
jakozaur
·il y a 5 mois·discuss
Funny thing, AI is not that terrible at using Ghidra. We released a benchmark on that and hopefully models will improve: https://quesma.com/blog/introducing-binaryaudit/
jakozaur
·il y a 5 mois·discuss
Funny coincidence, I'm working on a benchmark showcasing AI capabilities in binary analysis.

Actually, AI has huge potential for superhuman capabilities in reverse engineering. This is an extremely tedious job with low productivity. Currently reserved, primarily when there is no other option (e.g., malware analysis). AI can make binary analysis go mainstream for proactive audits to secure against supply-chain attacks.
jakozaur
·il y a 5 mois·discuss
Great idea. Currently, people have to rely on client-side spans in OpenTelemetry. However, it would be awesome if we could get spans for slow SQL queries, along with explanations.
jakozaur
·il y a 5 mois·discuss
In this benchmark, micro-services are really small, ~300 lines, and sometimes just two of them. More realistic tasks (large codebases, more microservices) would have a lower success rate.
jakozaur
·il y a 6 mois·discuss
See x thread for rationale: https://x.com/mitchellh/status/2014433315261124760?s=46&t=FU...

“ Ultimately, I want to see full session transcripts, but we don't have enough tool support for that broadly.”

I have a side project, git-prompt-story to attach Claude Vode session in GitHub git notes. Though it is not that simple to do automatic (e.g. i need to redact credentials).
jakozaur
·il y a 7 mois·discuss
I wish Neal would do behind the scenes, how he built this art. I wonder whether LLM assistants like Claude Code make such an interactive show more feasible.

He previously did a game "Infinite Craft" which leveraged Llama models. However, I was only able to find an outdated blog from 2019.
jakozaur
·il y a 7 mois·discuss
Not sure if I get this: WASM lets you use any language in the browser, though it still works way better with languages without GC, such as Rust or a transpiling C engine. Java is unlikely to be the best choice.

In the era of LLM assistants like Claude Code, any engineer can write frontend code using popular stacks like React and TypeScript. This use case is when those tools shine.
jakozaur
·il y a 7 mois·discuss
LLVM IR is quite fun to play with from many programming languages. The Java example is rather educational, but there are several practical example,s such as in Go Lang:

https://github.com/llir/llvm
jakozaur
·il y a 7 mois·discuss
The effect of climate change may be highly uneven. Some regions will be fine with adaptation, while other places will hardly sustain cities.
jakozaur
·il y a 7 mois·discuss
It is even more true with startups and business. Super rushed is bad, but doing for too long decreases quality.