You read your chips' datasheets that go in detail on pinouts and protocols. You use logic analyzers that capture signals and programmatically decode a multitude of protocols from those signals. When you don't know what are the pinouts or protocols, you compare against similar enough known ones, or bruteforce them.
Some examples:
* Once you've learned a few chip pinouts, you can pretty much guess unknown pinouts just from identifying a few ground/control/address pins, as even chip-on-board globs follow similar layouts [1]. However, despite plenty of datasheet archives being publicly available, none of them allow you to actually search by pin function [2], so you potentially have to go through dozens of datasheets of similar model ids to find what you need.
* UART baud rates that are likely used are in the single digits, they can be easily bruteforced.
* JTAG pins you need to interface with can go up to a dozen or so, there are enumeration scripts you can run in an Arduino to identify which pin has which function. These scripts also identify the IDCODE which you can lookup against boundary scan files if you need so [3]. But in most cases, you will interface with JTAG without thinking of the state machine behind it.
* Reverse engineering memory maps is a matter of following data read/write patterns and inferring associated functionality. You will bump into several address cross-references that also hint at what are the base addresses of each map. It's a more general skill you develop as you go, and Ghidra's decompilation made it much more accessible in the last years. The author went with a elaborate linker script but a more bare-bones approach would be to link code as a distinct ELF object, then copy its text section over to offset 0x20A0000-0x2010000 in the firmware image, and patch the initializers.
* Soldering and associated skills can also be self-learned from tutorials, pick several videos and learn the tricks/mistakes each of them cover.
So, in practice? Each of these does not require a vast amount of knowledge for things to happen, even allowing one to skip required reading of huge bibles that are recommended to electronics beginners. This is how a lifetime gets reduced to a few months of non-working hours.
When getting into hardware hacking, what I felt was the main blocker is how a lot is described at a superficial level, without enough breadcrumbs one can follow to reproduce the same results. Sure, the pictures of spaghetti wires and decapped chips look awesome, but nobody learns from that. Unlike the software side where you are given the source and everything you need to lookup is in front of you.
It looks like the compilation database is only used for discovering source files, without any additional cross translation unit analysis [1]. Even if you don't plan on implementing data flow tracking or other passes, it can still be interesting to know on which translation unit a function declaration is defined on.
Also, it seems we always have to recompile each unit when switching between source files. Consider pre-compiling and caching so it's only done once. That should also enable users to search across files/ASTs.
> Fraud detection in transaction data is mostly SQL. Not machine learning, not graph databases, not whatever Gartner is hyping this year. SQL, run against the right tables, with the right joins, looking for the right shapes.
It's also not all program-integrity, which is the only work that could justify such blanket statements. Worse is better as long as it addresses the problem domain.
Fintech clients are generally interested in knowing whether a transaction happening _right now_ is fraud. They want to know that in a few milliseconds, for high-dimensional data. It's work done at a scale where relational databases cannot meet these real-time constraints, and instead find other uses like historical data loading. That's how you end up with in-memory databases, stream-processing engines, and yes, even machine learning.
Having said that, some of the author's points are valid, and I'm looking forward for their next writings, in particular dealing with noisy alerts is a general problem beyond performance engineering.
The author is referring to limitations in analysing banking:
> Ghidra supports the 8051 architecture but not code banking.
Usually in these ISAs an I/O port or a register sets the bank number, so any processor module should be able to resolve concrete banked references. But you still need to know what that register holds in various code paths, which are likely dynamically computing those values.
No tooling can give this out-of-the-box, as it relies on knowing the concrete initial state of the system (i.e. memory and register contents), and knowing what to return when hooking into I/O accesses.
Once these are known, we can leverage the built-in pcode emulator and run it with this state. It seems nowadays Ghidra has some built-in support for Z3, but I personally never used it, so I'm not sure how viable it is for symbolic execution. Regardless, with either approach, we would now have concrete banked code references being resolved, and could script some auto annotation of the disassembly with these references. These would be equivalent to what the author gathered from the logic analyzer trace.
A pure static analysis approach seems to suggest one would manually brute-force through all possible bank numbers at any given code path, which I guess is only viable if you have the time for that.
Which drives and parameters for the READ BUF SCSI command yielded the expected 2366 bytes per sector? I imagine that it was combined with seeks to each sector before reading from the buffer (as it would be harder to isolate multiple sectors data in cache?).
Nice work Dmitry, looking forward to read your next article.
The later model Pixter Multimedia had the full memory space accessible via JTAG, which is how some carts and even boot ROM got dumped a while ago [1], is it the same deal with Pixter Color?
That OpenOCD script was a bit flaky, and sometimes the boot ROM would be already unloaded before reading, maybe you have some insights in how to make it more robust.
btw, have you looked into the original Pixter? The cart connector seems to have a very narrow bus, so it doesn't look like those carts have code, and probably can only be dumped with a decap.
For other architectures, it feels like a missed opportunity to not have an independent WASM build of MAME's debugger, as the whole project could already be built in WASM (although I think the latest versions were broken, as that target isn't actively maintained): https://docs.mamedev.org/initialsetup/compilingmame.html#ems...
I found their ELF format specification to have a decent coverage, even if not completely exhaustive (e.g. some debug info isn't breakdown after a certain point, but it just might be incomplete rather than limitations).
Some examples:
* Once you've learned a few chip pinouts, you can pretty much guess unknown pinouts just from identifying a few ground/control/address pins, as even chip-on-board globs follow similar layouts [1]. However, despite plenty of datasheet archives being publicly available, none of them allow you to actually search by pin function [2], so you potentially have to go through dozens of datasheets of similar model ids to find what you need.
* UART baud rates that are likely used are in the single digits, they can be easily bruteforced.
* JTAG pins you need to interface with can go up to a dozen or so, there are enumeration scripts you can run in an Arduino to identify which pin has which function. These scripts also identify the IDCODE which you can lookup against boundary scan files if you need so [3]. But in most cases, you will interface with JTAG without thinking of the state machine behind it.
* Reverse engineering memory maps is a matter of following data read/write patterns and inferring associated functionality. You will bump into several address cross-references that also hint at what are the base addresses of each map. It's a more general skill you develop as you go, and Ghidra's decompilation made it much more accessible in the last years. The author went with a elaborate linker script but a more bare-bones approach would be to link code as a distinct ELF object, then copy its text section over to offset 0x20A0000-0x2010000 in the firmware image, and patch the initializers.
* Soldering and associated skills can also be self-learned from tutorials, pick several videos and learn the tricks/mistakes each of them cover.
So, in practice? Each of these does not require a vast amount of knowledge for things to happen, even allowing one to skip required reading of huge bibles that are recommended to electronics beginners. This is how a lifetime gets reduced to a few months of non-working hours.
When getting into hardware hacking, what I felt was the main blocker is how a lot is described at a superficial level, without enough breadcrumbs one can follow to reproduce the same results. Sure, the pictures of spaghetti wires and decapped chips look awesome, but nobody learns from that. Unlike the software side where you are given the source and everything you need to lookup is in front of you.
[1]: https://qufb.gitlab.io/writeups/mysteries
[2]: https://github.com/qufb/PinoutDB
[3]: https://bsdl.info/index.htm