ThinkPad X1 Extreme intermittant freezes on Pop!_OS

[jsborjesson]

This ephemeral bug has been driving me mad for months and is pretty much the only reason I'm not perfectly satisfied with my current setup. If anyone has even the slightest hint of how to fix this, or thinks they are experiencing the same problem - please do comment.

Environment

• Thinkpad X1 Extreme Gen 3
  • Product Name: 20TK000FMX (upgraded RAM, same problem both before and after though)
  • SKU Number: LENOVO_MT_20TK_BU_Think_FM_ThinkPad X1 Extreme Gen 3
• Pop!_OS 20.10
• nvidia-driver-460
• ThinkVision P27h-10 external USB-C monitor

Narrowing this environment even further, my previous setup which was very similar (ThinkPad T580, Pop!_OS 20.04, and the same screen) never experienced this problem.

Symptoms

• Screen sometimes hangs until the next keypress.
  • This seems to happen only on the external display.
  • The thing has happened, but the monitor will not update until you do something again, this triggers the screen to render, showing that it did in fact already do the thing, just without updating the display.
• Entire system appears to freeze for ~1-2 seconds.
  • This can happen in the middle of moving the mouse cursor or scrolling, and it does freeze the movement of the cursor/scroll (it doesn't continue unrendered like the symptom above, but resumes from where it froze after a couple of seconds even without new input).
  • This seems to happen only after the external display is disconnected - it doesn't happen on the internal screen when the external display is present.
  • This symptom seems to disappear after logging out and back in again with the external screen disconnected.
• Error log shows up when waking from sleep
  • Messages saying something along the lines of spurious response (This message has gone away, replaced with the one below)
  • [520600,910609] usb 1-14: device descriptor read/64, error -71
    • These are repeated a few times more every time the computer wakes from sleep with only the numbers in brackets changing.
• A few times the external monitor image freezes indefinitely (just the image, program keeps running and you can move the mouse over to the other screen which keeps working). The only solution to this I've found is to disconnect/reconnect the monitor or reboot.
  • The times this has happened I'm fairly confident it was running out of RAM, a memory-hungry process is eating all of it and causes the nvidia driver(??) to not have enough memory, which causes it to freeze from which it never recovers.

Things I've tried

I'm convinced it has to do with NVidia drivers and how the system deals with hybrid graphics.

• Disabling hybrid graphics in BIOS (running only the external NVidia card) - this seems to upset it somehow and the symptoms differ (still intermittant freezes, but only on external monitor, or only on built in monitor etc), though it does not solve the problem it is evidence it is at least related to these settings.
• Using only internal graphics has no symptoms the short time I tried it, but doesn't support the external monitor which is a dealbreaker for me.
• Downgrade the graphics drivers to 440 using gnome-software https://github.com/pop-os/nvidia-graphics-drivers/issues/65 - no effect
• Full (unattended) hardware diagnostics (by pressing F10 at startup) passed without a hitch in a mere 7 hours or so.
• Upgrading the RAM from 16GB to 32GB, this does seem to lower the frequency this occurs

Things to try

• Manually upgrade the BIOS to a newer version than available from LVFS
• Keep up with the quite frequent updates to Pop!_OS (which I really like and really do not want to switch away from), it may get solved eventually

Sources

• This one seems very related: System is freezing after last update pop-os/pop#569
• Maybe related Freeze pop-os/pop#959
• It also seems spookily simlar to some other ephemeral bugs reported on certain projects, like this one in alacritty (I do not use alacritty, but the symptoms match in Gnome Terminal) Redraw randomly hangs until a keypress alacritty/alacritty#824
• This thread also has some similarities: https://www.reddit.com/r/archlinux/comments/4gtacw/terminal_sometimes_does_not_render_output_until/

[jsborjesson]

Upgraded firmware yesterday - let's see if there's any difference:

Embedded Controller 0.1.5 -> 0.1.7
Lenovo System Firmware 0.1.7 -> 0.1.12

[jsborjesson]

Currently I'm coping with this issue by logging out and back in again whenever I unplug my screen, and simply ignoring the error logs that show up when waking from sleep.

So far this is 100% reproducible - it always happens after unplugging the screen, and it's always entirely fixed by relogging. This thankfully brings this from a major prohibitive issue to a minor annoyance for me.