A study of malicious code in PyPI ecosystem(about.honywen.com)
about.honywen.com
A study of malicious code in PyPI ecosystem
https://about.honywen.com/publication/2023ase/#
37 comments
This is why I'm so interested in running Python (and almost every other language) inside a WebAssembly sandbox.
It's 2023. I never want to run untrusted code on any of my devices outside of a sandbox ever again.
WebAssembly feels like it should be that sandbox. Every few months I check to see if it's easy enough to run regular Python/JavaScript/etc code inside a WebAssembly sandbox yet... we're not quite there but it feels really close.
More notes here: https://til.simonwillison.net/webassembly/python-in-a-wasm-s...
It's 2023. I never want to run untrusted code on any of my devices outside of a sandbox ever again.
WebAssembly feels like it should be that sandbox. Every few months I check to see if it's easy enough to run regular Python/JavaScript/etc code inside a WebAssembly sandbox yet... we're not quite there but it feels really close.
More notes here: https://til.simonwillison.net/webassembly/python-in-a-wasm-s...
WebAssembly as a platform is probably as good as any other, but I think as an ISA and compilation target they flubbed a lot of the basics. It's slightly too clever, slightly too awkward to target seamlessly etc
Curious to know why existing sandboxing/isolation technologies like VMs & Containers not enough here?
It's really just a matter of practicality. Developing within docker or a VM is kind of a pain.
Would love to know the specifics I'm missing. Based on my experience, Docker/FC just works out of the box and gives you a shell.
A big one that has bitten my team a lot is syncing files between local and container. There are even whole separate tools like https://docker-sync.readthedocs.io designed to try to mitigate the problem. The most annoying thing is it almost works almost all the time and then all of a sudden stop working for some people for no obvious reason. We gave up on Docker for development because we couldn't trust it to work consistently.
I must be missing something, but why not just mount your code into the container? That's what I usually do.
I have a directory with my files in it and mount that directory into the container. However when I make changes to the files outside the container, those changes aren't always reflected inside the container, causing all kinds of weird behaviour and mismatches. It almost always works as expected, for almost everybody on the team, but everybody with a Mac has experienced this at least once.
They're not lightweight and easy enough to use.
I have Docker on my Mac. I run things in it if I have to, but it's pretty resource heavy and I have to look at my notes every time I want to run "docker run".
Plus Docker isn't actually a fully secure sandbox!
There's Firecracker, which I believe is secure (AWS built it for Lambda after all) but good luck figuring out how to run that on a Mac.
I want something that's tiny, fast, ridiculously easy to use and that I can run dozens if not hundreds of instances.
WebAssembly is almost that thing.
I have Docker on my Mac. I run things in it if I have to, but it's pretty resource heavy and I have to look at my notes every time I want to run "docker run".
Plus Docker isn't actually a fully secure sandbox!
There's Firecracker, which I believe is secure (AWS built it for Lambda after all) but good luck figuring out how to run that on a Mac.
I want something that's tiny, fast, ridiculously easy to use and that I can run dozens if not hundreds of instances.
WebAssembly is almost that thing.
> They're not lightweight and easy enough to use.
```
$ alias SafeRun="bwrap --ro-bind / / --dev /dev --proc /proc --unshare-all --tmpfs ~ --bind ~/Untrusted ~/Untrusted"
$ SafeRun python3 ~/Untrusted/RandomScript.py
$ SafeRun ~/Untrusted/RandomExecutable
```
This is basically manually invoking what Flatpak does:
https://github.com/containers/bubblewrap
This is also useful for more than just security. E.G., you can test how your app would behave on a fresh install by masking your user configuration files. I personally also have a tool that uses it to basically bundle all dependencies from an entire Linux distribution in order to make highly portable AppImages— Been meaning to post that, will get around to it eventually maybe.
The flags above should hide your user data (`--tmpfs`), disable network access (`--unshare-all`), hide/virtualize devices and OS state (`--dev` and `--proc`), and make the rest of the root filesystem read-only (`--ro-bind`— Including the insecure X11 socket in `/tmp`, which you might want to expose for GUI apps).
Check them against `bwrap --help`; I might have omitted one or two more things you'd need.
```
$ alias SafeRun="bwrap --ro-bind / / --dev /dev --proc /proc --unshare-all --tmpfs ~ --bind ~/Untrusted ~/Untrusted"
$ SafeRun python3 ~/Untrusted/RandomScript.py
$ SafeRun ~/Untrusted/RandomExecutable
```
This is basically manually invoking what Flatpak does:
https://github.com/containers/bubblewrap
This is also useful for more than just security. E.G., you can test how your app would behave on a fresh install by masking your user configuration files. I personally also have a tool that uses it to basically bundle all dependencies from an entire Linux distribution in order to make highly portable AppImages— Been meaning to post that, will get around to it eventually maybe.
The flags above should hide your user data (`--tmpfs`), disable network access (`--unshare-all`), hide/virtualize devices and OS state (`--dev` and `--proc`), and make the rest of the root filesystem read-only (`--ro-bind`— Including the insecure X11 socket in `/tmp`, which you might want to expose for GUI apps).
Check them against `bwrap --help`; I might have omitted one or two more things you'd need.
Interestingly, Docker on Mac actually runs lightweight VM, which then runs a whole Linux OS inside it, to then run your containers.
From an isolation perspective, that's giving you hypervisor level isolation which is a step above most container-based solutions, which you point out rightly isn't a full sandbox. A hypervisor with CPU support is probably the gold standard for isolation, asides from running it on a separate physically isolated system.
From an isolation perspective, that's giving you hypervisor level isolation which is a step above most container-based solutions, which you point out rightly isn't a full sandbox. A hypervisor with CPU support is probably the gold standard for isolation, asides from running it on a separate physically isolated system.
> There's Firecracker, which I believe is secure (AWS built it for Lambda after all) but good luck figuring out how to run that on a Mac.
Then, how about using AWS Lambda for your use case?
Then, how about using AWS Lambda for your use case?
I want to run code on my own laptop and phone.
Makes sense. When I read your notes, it appeared that you wanted to run untrusted code on a backend server.
Yeah, I want to do that as well! And Lambda is definitely high on my list of potential solutions for that.
I'm still hoping for a solution to the run-on-personal-devices problem though.
I'm still hoping for a solution to the run-on-personal-devices problem though.
Agreed about Docker.
Personally, I use Google Colab for my convenient, lightweight Python sandbox. But I’d rather not depend on Google. It feels weird that this is my best option for now.
Personally, I use Google Colab for my convenient, lightweight Python sandbox. But I’d rather not depend on Google. It feels weird that this is my best option for now.
> Personally, I use Google Colab for my convenient, lightweight Python sandbox. But I’d rather not depend on Google. It feels weird that this is my best option for now.
`bwrap` is great for this on Linux. A one line alias should let you run any program in a lightweight and highly customizable sandbox safely-ish. See my reply to the parent comment for an example.
`bwrap` is great for this on Linux. A one line alias should let you run any program in a lightweight and highly customizable sandbox safely-ish. See my reply to the parent comment for an example.
Java and C# used to have this, but both removed it later on.
I'd love to read more about why these attempts were abandoned, and how they got the idea to try
The JEP to remove it chronicles the rise and fall of java.security: https://openjdk.org/jeps/411
One of the sources referenced in the paper is about the work the company I co-founded is doing (https://phylum.io). We've been working closely with PyPI to not only report issues related to malware, but are also helping provide guidance around on-going work to help curb additional malware in the ecosystem. I think this is a step in the right direction, and one I hope other ecosystems will eventually take.
Though, this is likely to be a cat and mouse game for the foreseeable future. Detection will get better, and attackers will change tactics.
In the meantime, we've been open-sourcing some tooling to help protect developers from these sorts of attacks. Namely, a sandbox that locks down network/disk/env [1] and our CLI [2] that allows you to perform a `pip` install in the sandbox, after checking our API for behaviors/issues with the package. For example:
1. https://github.com/phylum-dev/birdcage
2. https://github.com/phylum-dev/cli
Though, this is likely to be a cat and mouse game for the foreseeable future. Detection will get better, and attackers will change tactics.
In the meantime, we've been open-sourcing some tooling to help protect developers from these sorts of attacks. Namely, a sandbox that locks down network/disk/env [1] and our CLI [2] that allows you to perform a `pip` install in the sandbox, after checking our API for behaviors/issues with the package. For example:
phylum pip install <pkgName>
1. https://github.com/phylum-dev/birdcage
2. https://github.com/phylum-dev/cli
> Though, this is likely to be a cat and mouse game for the foreseeable future. Detection will get better, and attackers will change tactics.
How do you see detection ever getting better to the point where it isn't a game of cat and mouse? Isn't the whole point of these security threats that go undetected is that they are novel attacks? Or even popularized attacks with different parameters?
In these trust but verify models of a central registry, it will require some heavy lifting like AI security detection to even keep up with. That seems like a new problem in itself if introduced.
How do you see detection ever getting better to the point where it isn't a game of cat and mouse? Isn't the whole point of these security threats that go undetected is that they are novel attacks? Or even popularized attacks with different parameters?
In these trust but verify models of a central registry, it will require some heavy lifting like AI security detection to even keep up with. That seems like a new problem in itself if introduced.
"We conducted an empirical study to understand the characteristics and current state of the malicious code lifecycle in the PyPI ecosystem. We first built an automated data collection framework and collated a multi-source malicious code dataset containing 4,669 malicious package files. We preliminarily classified these malicious code into five categories based on malicious behaviour characteristics. Our research found that over 50% of malicious code exhibits multiple malicious behaviours, with information stealing and command execution being particularly prevalent. In addition, we observed several novel attack vectors and anti-detection techniques. Our analysis revealed that 74.81% of all malicious packages successfully entered end-user projects through source code installation, thereby increasing security risks. A real-world investigation showed that many reported malicious packages persist in PyPI mirror servers globally, with over 72% remaining for an extended period after being discovered. Finally, we sketched a portrait of the malicious code lifecycle in the PyPI ecosystem, effectively reflecting the characteristics of malicious code at different stages. We also present some suggested mitigations to improve the security of the Python open-source ecosystem."
Two interesting takeaways from the paper itself[1]: Chinese companies and universities appear to make extensive use of PyPI mirrors (references 23-26), and those mirrors consistently fail to purge malicious packages in a timely manner.
[1]: https://about.honywen.com/publication/2023ase/2023ASE.pdf
[1]: https://about.honywen.com/publication/2023ase/2023ASE.pdf
Interesting. I wonder if this is also true for available/existing commercial/open-source PyPI mirroring solutions [Edited].
Why would the Chinese have their own pypi mirror software? I think they are running the same software as western companies.
Probably because accessing PyPI directly is slow, because of the Great Firewall.
I think the GP is asking why Chinese engineers would have their own software for mirroring, not why they'd have their own mirrors.
(I believe most Chinese mirrors use bandersnatch[1], like most non-Chinese mirrors.)
[1]: https://github.com/pypa/bandersnatch
(I believe most Chinese mirrors use bandersnatch[1], like most non-Chinese mirrors.)
[1]: https://github.com/pypa/bandersnatch
During mirroring recently I noticed a new pypi package with such a long name that it caused "file name too long" errors on artifactory and with bandersnatch saving to local risk. In this case the code handled the situation correctly (if python throwing an error con be considered correct handling) but it could well have been an exploit mechanism.
Anything touching the internet where user provided content will be consumed should be hardened and preferably sandboxed in some way, stuck in an unprivileged DMZ, and monitored.
As for the pypi modules themselves, a web of trust system might actually work. For scientific packages especially. There's always the risk of an author getting their machine compromised and someone pushing an update with either a subtle bug which can pass unnoticed or a loud and quickly detected attack. Making a change in account 2FA tokens visible would be a great start, as would integrating signing keys.
(The long file name in this case was the digits of pi.)
Anything touching the internet where user provided content will be consumed should be hardened and preferably sandboxed in some way, stuck in an unprivileged DMZ, and monitored.
As for the pypi modules themselves, a web of trust system might actually work. For scientific packages especially. There's always the risk of an author getting their machine compromised and someone pushing an update with either a subtle bug which can pass unnoticed or a loud and quickly detected attack. Making a change in account 2FA tokens visible would be a great start, as would integrating signing keys.
(The long file name in this case was the digits of pi.)
Am I reading this correctly? There's only ~2k/470k packages containing malware on the PyPi registry?
That seems exceptional. The registry maintainers must be working extremely hard on these problems.
That seems exceptional. The registry maintainers must be working extremely hard on these problems.
There's a website to accompany this which is more readable than the paper: https://sites.google.com/view/pypiempircal
(At least I think it's to accompany this paper, I found it in a footnote and it's not entirely clear if it's officially related or not.)
(At least I think it's to accompany this paper, I found it in a footnote and it's not entirely clear if it's officially related or not.)
So, is there a way to check if any malicious packages installed in my conda environment?
I've been building Packj [1] to detect dummy, malicious, abandoned, typo-squatting, and other "risky" PyPI/NPM/Ruby/PHP/Maven/Rust packages. It carries out static/dynamic/metadata analysis and scans for 40+ attributes such as num funcs/files, spawning of shell, use of SSH keys, network communication, use of decode+eval, mismatch of GitHub code vs packaged code (provenance), change in APIs across versions, etc. to flag risky packages.
1. https://github.com/ossillate-inc/packj
1. https://github.com/ossillate-inc/packj
Cool project. How do you feel about projects like OpenSSF scorecards or even the checks that socket.dev do today on these packages to help determine risk?
https://github.com/ossillate-inc/packj/blob/main/.packj.yaml
Secondly, what about impersonation where attackers imitate a popular package and its respective metadata?
https://github.com/ossillate-inc/packj/blob/main/.packj.yaml
Secondly, what about impersonation where attackers imitate a popular package and its respective metadata?
Thanks! We need more such efforts to improve supply-chain security of open-source software.
Packj detects typo-squatting (impersonation) as well.
Packj detects typo-squatting (impersonation) as well.
This is interesting! Implementing it today itself for my open source package development pipeline!
The https://en.wikipedia.org/wiki/Principle_of_least_privilege as constructed by https://en.wikipedia.org/wiki/Capability-based_security is the solution for this, yet Python will probably never be capable of this kind of internal encapsulation, it's too much of a fundamental change - and even if some sort of sandboxing ability is accomplished, creating separate/recursive sandboxes (needed when importing more, separate libraries which in turn import libraries whose authority they want to limit) will probably require another interpreter instance for each sandbox (as with WebAssembly).
I hope current and future language designers will take this into account, and construct their compilers, virtual machines and interpreters accordingly. Python was created before the internet as we know it now existed, so perhaps its lack of security mechanisms shouldn't be surprising. But it and any new developments that fail to consider this aspect of computation will be fundamentally flawed from the beginning.
https://github.com/void4/notes/issues/41