Ask HN: How do you troubleshoot desktop Linux crashes/freezes?
4 comments
The way the Gentoo or the Funtoo community has done it in the past was to run SSH on their desktop machines. With some correct flags you can still login into a "frozen" system. Tuning vm.admin_reserve_kbytes and adjusting the OOMScore should help with that.
Another option is netconsole for kernel issues, but that requires a separate system to receive the UDP logs. This may help with suspend issues.
Regarding NVIDIA GPUs and suspend, my recommendation is to use an Intel-only system for a painless Linux experience, at least for the local system. Intel has had the best Linux drivers for an overall experience and I believe the most reliable suspend. It's possible the suspend problems are entirely due to the NVIDIA GPU. Isolating the problematic component should help you identify the problematic driver.
If I have to run a NVIDIA GPU it's usually in a separate chassis. Avoiding having to deal with NVIDIA GPUs on local systems leads to happiness.
I would recommend trying Fedora with nvidia akmods to see how your system performs with one of the latest kernels. Chances are there may be fixes for suspend and driver issues.
The zen solution is to avoid having to deal with bad hardware and drivers. Otherwise the only realistic solution is to start recompiling the kernel, file an issue with the hardware vendor, and/or stick to hardware that someone else supports to fully work with linux. I believe steam machine uses AMD GPUs, and they have had better support in the Linux kernel than NVIDIA.
Another option is netconsole for kernel issues, but that requires a separate system to receive the UDP logs. This may help with suspend issues.
Regarding NVIDIA GPUs and suspend, my recommendation is to use an Intel-only system for a painless Linux experience, at least for the local system. Intel has had the best Linux drivers for an overall experience and I believe the most reliable suspend. It's possible the suspend problems are entirely due to the NVIDIA GPU. Isolating the problematic component should help you identify the problematic driver.
If I have to run a NVIDIA GPU it's usually in a separate chassis. Avoiding having to deal with NVIDIA GPUs on local systems leads to happiness.
I would recommend trying Fedora with nvidia akmods to see how your system performs with one of the latest kernels. Chances are there may be fixes for suspend and driver issues.
The zen solution is to avoid having to deal with bad hardware and drivers. Otherwise the only realistic solution is to start recompiling the kernel, file an issue with the hardware vendor, and/or stick to hardware that someone else supports to fully work with linux. I believe steam machine uses AMD GPUs, and they have had better support in the Linux kernel than NVIDIA.
30 years ago, suspend was dicey in Windows.
Things have improved but you’re still pressing your luck…
What about SSH in to debug before rebooting? Or maybe log with the kernels’ net console feature (mainly wired Ethernet), assuming a physical serial port isn’t available
Things have improved but you’re still pressing your luck…
What about SSH in to debug before rebooting? Or maybe log with the kernels’ net console feature (mainly wired Ethernet), assuming a physical serial port isn’t available
Claude has me doing fancy stuff with dmesg and journalctl. Can’t pretend to understand a lot of it but it has saved my butt a few times.
My worst cases were all related with early boot problems, so the dracut shell was helpful.
I make extensive use of Ctrl+Alt+f3, and magic system request keys (Ctrl+Alt+PrtSc+f; Ctrl+Alt+PrtSc+reisub). But often the system is too far gone to drop to a virtual terminal. In these cases, I usually try to find clues in dmesg/syslog/journalctl.
The bulk of issues seem to be caused by Nvidia GPUs, and suspend.
What are your go-to commands for recovery? And, when that isn't an option, how do you post-mortem after rebooting?