You can still download VS6 from Microsoft and clone a repo from there, and chances it’ll compile and run are higher than JS project that’s two weeks old
Not claude code specific, but I've been noticing this on Opus 4.6 models through Copilot and others as well. Whenever the phrase "simplest fix" appears, it's time to pull the emergency break. This has gotten much, much worse over the past few weeks. It will produce completely useless code, knowingly (because up to that phrase the reasoning was correct) breaking things.
Today another thing started happening which are phrases like "I've been burning too many tokens" or "this has taken too many turns". Which ironically takes more tokens of custom instructions to override.
I posted this and it picked up steam over night, so I thought I'd add how I'm using it:
I work on 3D/4D math in F#. As part of the testing strategy for algorithms, I've set up a custom agent with an F# script that instruments Roslyn to find FP and FP-in-loop hotspots across the codebase.
The agent then reasons through the implementation and writes core expressions into an FPCore file next to the existing tests, running several passes, refining the pres based on realistic caller input. This logs Herbie's proposed improvements as output FPCore transformations. The agent then reasons through solutions (which is required, Herbie doesn't know algorithm design intent, see e.g. this for a good case study: https://pavpanchekha.com/blog/herbie-rust.html), and once convinced of a gap, creates additional unit tests and property tests (FsCheck/QuickCheck) to prove impact. Then every once in a while I review a batch to see what's next.
Generally there are multiple types of issues that can be flagged:
a) Expression-level imprecision over realistic input ranges: this is Herbie's core strength. Usually this catches "just copied the textbook formula" instance of naive math. Cancellation, Inf/NaN propagation, etc. The fixes are consistently using fma for accumulation, biggest-factor scaling to prevent Inf, hypot use, etc.
b) Ill-conditioned algorithms. Sometimes the text books lie to you, and the algorithms themselves are unfit for purpose, especially in boundary regions. If there are multiple expressions that have a <60% precision and only a 1 to 2% improvement across seeds, it's a good sign the algo is bad - there's no form that adequately performs on target inputs.
c) Round-off, accumulation errors. This is more a consequence of agent reasoning, but often happens after an apparent "100% -> 100%" pass. The agent is able to, via failing tests, identify parts of an algorithm that can benefit from upgrading the context to e.g. double-word arithmetic for additional precision.
This guys factory is just across the lake from where I live and this is painful to watch. Both Alibaba and the general local industry (metal fabs, train shops, etc) have high degrees of expertise in supply chain verification. You can hire (heck even bribe) experts along the way to reduce fuck ups. The video contained no mention of any audits, any additional paperwork beyond some pictures.
I once had a company that procured very simple electronics (fingerprint readers) from Taiwan and due diligence included travelling there, meeting every single person in the engineering office in person, then touring the contract factory where this would be built, then negotiating shipping and even driver development details.
This took all of one week and the price of a few plane tickets. We didn’t have the cash for professional auditors. In the end we got a product that worked, and even at a lower price (negotiating at a distance is not effective).
No engine can be 100% perfect of course, the original comment is broadly accurate though. CodeQL builds a full semantic database including types and dataflow from source code, then runs queries against that. QL is fundamentally a logic programming language that is only concerned with the satisfiably of the given constraint.
If dataflow is not provably connected from source to sink, an alert is impossible. If a sanitization step interrupts the flow of potentially tainted data, the alert is similarly discarded.
The end-to-end precision of the detection depends on the queries executed, the models of the libraries used in the code (to e.g., recognize the correct sanitizers), and other parameters. All of this is customizable by users.
All that can be overwhelming though, so we aim to provide sane defaults. On GitHub, you can choose between a "Default" and "Extended" suite. Those are tuned for different levels of potential FN/FP based on the precision of the query and severity of the alert.
Severities are calculated based on the weaknesses the query covers, and the real CVE these have caused in prior disclosed vulnerabilities.
Heyo, I'm the Product Director for detection & remediation engines, including CodeQL.
I would love to hear what kind of local experience you're looking for and where CodeQL isn't working well today.
As a general overview:
The CodeQL CLI is developed as an open-source project and can run CodeQL basically anywhere. The engine is free to use for all open-source projects, and free for all security researchers.
Results are stored in standard formats and can be viewed and processed by any SARIF-compatible tool. We provide tools to run CodeQL against thousands of open-source repos for security research.
The repo linked above points to dozens of other useful projects (both from GitHub and the community around CodeQL).
Ive been using it (the original 15 tool version) for months now. It’s amazing. Any app's inner workings are suddenly transparent. I can track down bugs. Get a deeper understanding of any tool, and even write plug-ins or preload shims that mod any app. It’s like I finally actually _own_ the software I bought years ago.
For objective C heavy code, I also use Hopper Disassembler (which now has a built in MCP server).
They’re multi wavelength CT. Basically whenever you see a 4:3 box with a “smiths” logo over the belt it’s going to be a pretty painless process (take nothing out except analog film)
I work at GitHub but even then I often use OpenRouter models in the CLI and Copilot App