Friday, March 16, 2018

systemd-udevd taking 100 percent cpu - solved by blacklisting cx88_alsa audio driver and cx88xx etc TV Card Driver

--------------------------------------------------
Problem
--------------------------------------------------
systemd-udevd taking 100% cpu.
Even failing to power off Ubuntu. (screen shut down, machine power stayed on. Has to long press power button to shuf down Ubuntu1604LTS.)

--------------------------------------------------
Investigating
--------------------------------------------------
Seeing this i2c related kernel error.
"i2c i2c-1: sendbytes: NAK bailout."
It's reported previously. https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675540
This bug still exists and looks like it was identified as part of the Nouveau video driver, experienced now with Ubuntu 16.10.


$dmesg indicated that it's when calling cx88_audio_pci_driver_init
  continuously time out due to i2c error.
      - pci_audio_driver loading hang.....
        [   15.080129] i2c i2c-13: sendbytes: error -110
        [   16.287687] ip_tables: (C) 2000-2006 Netfilter Core Team
        [   16.308070] ip6_tables: (C) 2000-2006 Netfilter Core Team
        [  244.171444] INFO: task systemd-udevd:325 blocked for more than 120 seconds.
        [  244.171451]       Not tainted 4.13.0-32-generic #35~16.04.1-Ubuntu
        [  244.171453] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
        [  244.171456] systemd-udevd   D    0   325    278 0x00000004
        [  244.171460] Call Trace:
        [  244.171470]  __schedule+0x3c2/0x890
        [  244.171474]  schedule+0x36/0x80
        [  244.171477]  schedule_preempt_disabled+0xe/0x10
        [  244.171480]  __mutex_lock.isra.2+0x2ae/0x4e0
        [  244.171487]  ? driver_probe_device+0x450/0x450
        [  244.171490]  __mutex_lock_slowpath+0x13/0x20
        [  244.171492]  ? __mutex_lock_slowpath+0x13/0x20
        [  244.171495]  mutex_lock+0x2f/0x40
        [  244.171497]  __driver_attach+0x5d/0xf0
        [  244.171500]  ? driver_probe_device+0x450/0x450
        [  244.171503]  bus_for_each_dev+0x6c/0xc0
        [  244.171506]  driver_attach+0x1e/0x20
        [  244.171508]  bus_add_driver+0x1f4/0x270
        [  244.171512]  ? 0xffffffffc00a6000
        [  244.171515]  driver_register+0x60/0xe0
        [  244.171517]  ? 0xffffffffc00a6000
        [  244.171522]  __pci_register_driver+0x4c/0x50
        [  244.171528]  cx88_audio_pci_driver_init+0x1e/0x1000 [cx88_alsa]       ..............
        [  244.171532]  do_one_initcall+0x53/0x1b0
        [  244.171536]  ? __vunmap+0x81/0xb0
        [  244.171540]  ? kmem_cache_alloc_trace+0x152/0x1b0
        [  244.171543]  ? kfree+0x162/0x170
        [  244.171548]  do_init_module+0x5f/0x209
        [  244.171551]  load_module+0x1961/0x1d60
        [  244.171556]  ? ima_post_read_file+0x7d/0xa0
        [  244.171561]  SYSC_finit_module+0xe5/0x120
        [  244.171564]  ? SYSC_finit_module+0xe5/0x120
        [  244.171568]  SyS_finit_module+0xe/0x10
        [  244.171572]  entry_SYSCALL_64_fastpath+0x33/0xa3
        [  244.171575] RIP: 0033:0x7ff05dcda4d9
        [  244.171576] RSP: 002b:00007ffe319d70c8 EFLAGS: 00000202 ORIG_RAX: 0000000000000139
        [  244.171579] RAX: ffffffffffffffda RBX: 00005627dc1ac890 RCX: 00007ff05dcda4d9
        [  244.171581] RDX: 0000000000000000 RSI: 00005627dc1b8450 RDI: 0000000000000007
        [  244.171583] RBP: 00005627dc1b8450 R08: 0000000000000000 R09: 0000000000000011
        [  244.171584] R10: 0000000000000007 R11: 0000000000000202 R12: 0000000000000000
        [  244.171586] R13: 00005627dc1cec00 R14: 0000000000020000 R15: 0000000000000000

Tried loading old kernel back to 4-Feb. Problem still exist.
        qyang@Ubuntu1604LTS:~$ ll /lib/modules/
        total 24K
        drwxr-xr-x  5 root root 4.0K Jun 20  2017 4.8.0-56-generic/
        drwxr-xr-x 23 root root 4.0K Feb  4 07:36 ../
        drwxr-xr-x  5 root root 4.0K Feb  4 07:40 4.13.0-32-generic/
        drwxr-xr-x  5 root root 4.0K Feb 25 09:50 4.13.0-36-generic/
        drwxr-xr-x  6 root root 4.0K Mar 15 22:19 ./
        drwxr-xr-x  5 root root 4.0K Mar 15 22:48 4.13.0-37-generic/

Check cx88 releated hardware drivers.
        qyang@Ubuntu1604LTS:~$ lsmod |grep cx88
        cx88_alsa              20480  1
        snd_pcm                98304  5 snd_hda_intel,snd_hda_codec,cx88_alsa,snd_hda_core,snd_hda_codec_hdmi
        cx8800                 40960  1
        cx8802                 20480  0
        cx88xx                 90112  3 cx88_alsa,cx8802,cx8800
        videobuf2_dma_sg       16384  1 cx8800
        tveeprom               24576  1 cx88xx
        v4l2_common            16384  3 tuner,cx88xx,cx8800
        videobuf2_v4l2         24576  1 cx8800
        videobuf2_core         40960  4 cx8802,cx88xx,videobuf2_v4l2,cx8800
        videodev              172032  7 tuner,v4l2_common,cx88_alsa,videobuf2_core,cx88xx,videobuf2_v4l2,cx8800
        snd                    81920  22 snd_hda_intel,snd_hwdep,snd_seq,snd_hda_codec,cx88_alsa,snd_timer,snd_rawmidi,snd_hda_codec_hdmi,snd_hda_codec_generic,snd_seq_device,snd_hda_codec_realtek,snd_pcm
        rc_core                36864  4 rc_winfast,ite_cir,cx88xx
        i2c_algo_bit           16384  2 cx88xx,nouveau


--------------------------------------------------
Fix Applied
Ref:
    https://unix.stackexchange.com/questions/28240/disable-detection-of-certain-hardware
--------------------------------------------------
Decided to disable driver load.

      - $touch /etc/modprobe.d/blacklist-cx88-alsa.conf and edit it.
        qyang@Ubuntu1604LTS:~$ cat /etc/modprobe.d/blacklist-cx88_alsa.conf
        blacklist cx88_alsa
        blacklist cx88xx
        blacklist cx8800
        blacklist cx8802
      - Problem solved after reboot.

No comments: