Show HN: Memstop – A tool to prevent OOM errors in parallel builds(github.com)
github.com
Show HN: Memstop – A tool to prevent OOM errors in parallel builds
https://github.com/surban/memstop
1 comments
Congrats on shipping this—memstop looks incredibly useful for anyone diagnosing memory usage regressions in long-running Python apps or ML workflows. I especially appreciate the simplicity: one function call, no external agents, and integration with stdlib tracemalloc for context. Clean and focused.
Any plans to support tracking across async tasks or threads?
Any plans to support tracking across async tasks or threads?
I built a small tool to solve a problem on my build VM, which has 32 CPU cores but only 16GB of RAM. Running make -j32 would frequently fail when the OOM killer terminated a compiler process.
Throttling with a lower -j count felt inefficient, and make's load-average flag (-l) tracks CPU, not memory pressure.
So, I created Memstop. It's a simple LD_PRELOAD library that acts as a gatekeeper. Before each new process starts, it checks the available memory in /proc/meminfo. If memory is below a configurable percentage (default 10%), it simply pauses and waits for other processes to finish.
This allows my builds to use full parallelism, self-regulating by pausing when memory gets tight instead of crashing.
Usage is simple:
LD_PRELOAD=/path/to/memstop.so make -j32
You can control the threshold with the MEMSTOP_PERCENT environment variable.
The project is on GitHub (GPLv3), and I'm sharing it in case it's useful to others. I'd love to hear your thoughts!
Link: https://github.com/surban/memstop