Dotaz: Debian na HP DL380 G7 - sending NMI to all CPUs

[BigWrigley]

Zdravim.

Na pomerne vytizenem DB serveru s MariaDB a Debianem 7.9, jadro 3.2.0 se mi objevila tahle nehezka vec. V serveru je 2x Xeon X5660, NMI se posila na vsechna CPU.

[31820932.809457] INFO: rcu_bh detected stall on CPU 11 (t=0 jiffies)
[31820932.809556] sending NMI to all CPUs:
[31820932.809564] NMI backtrace for cpu 0
[31820932.809568] CPU 0 
[31820932.809570] Modules linked in: iptable_filter ip_tables x_tables tcp_diag inet_diag nfnetlink_log nfnetlink dm_mod mptctl mptbase bonding ext3 mbcache jbd loop radeon snd_pcm snd_page_alloc ttm drm_kms_helper snd_timer drm snd soundcore power_supply i2c_algo_bit joydev coretemp crc32c_intel ghash_clmulni_intel aesni_intel i2c_core aes_x86_64 aes_generic cryptd psmouse iTCO_wdt hpwdt iTCO_vendor_support i7core_edac hpilo edac_core pcspkr serio_raw acpi_power_meter container processor button evdev xfs sr_mod cdrom ata_generic usb_storage usbhid hid uhci_hcd sd_mod sg crc_t10dif ehci_hcd ata_piix usbcore libata usb_common bnx2 hpsa thermal thermal_sys scsi_mod netxen_nic [last unloaded: scsi_wait_scan]
[31820932.809636] 
[31820932.809640] Pid: 0, comm: swapper/0 Not tainted 3.2.0-4-amd64 #1 Debian 3.2.63-2+deb7u1 HP ProLiant DL380 G7
[31820932.809644] RIP: 0010:[ffffffff811ef6b0]  [ffffffff811ef6b0] intel_idle+0xb9/0x119
[31820932.809654] RSP: 0018:ffffffff81601e88  EFLAGS: 00000046
[31820932.809656] RAX: 0000000000000010 RBX: 0000000000000004 RCX: 0000000000000001
[31820932.809659] RDX: 0000000000000000 RSI: ffffffff81601fd8 RDI: 141d0d2c111cf400
[31820932.809662] RBP: 0000000000000002 R08: 0000000000000225 R09: 0000000000000225
[31820932.809665] R10: 00000000000035e3 R11: 0000000000000000 R12: ffff88049fc19d70
[31820932.809668] R13: 0000000000000010 R14: 141d0d2c3b617220 R15: 0000000000000000
[31820932.809671] FS:  0000000000000000(0000) GS:ffff88049fc00000(0000) knlGS:0000000000000000
[31820932.809675] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[31820932.809678] CR2: 00007f5f266a8000 CR3: 0000000001605000 CR4: 00000000000006f0
[31820932.809681] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[31820932.809684] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[31820932.809687] Process swapper/0 (pid: 0, threadinfo ffffffff81600000, task ffffffff8160d020)
[31820932.809689] Stack:
[31820932.809724]  ffffffff810139e6 ffffffff812722fe 0000000000000000 000000000008635e
[31820932.809731]  0000000000000000 000000000008635e ffff88049fc19d70 ffffffff81646510
[31820932.809737]  00000000fffffff0 0000000000000002 ffffffff816465c0 ffffffff8127141d
[31820932.809743] Call Trace:
[31820932.809787]  [ffffffff810139e6] ? read_tsc+0x5/0x14
[31820932.809792]  [ffffffff812722fe] ? menu_select+0x198/0x2c1
[31820932.809797]  [ffffffff8127141d] ? cpuidle_idle_call+0xec/0x179
[31820932.809803]  [ffffffff8100d242] ? cpu_idle+0xa5/0xf2
[31820932.809810]  [ffffffff816abb3b] ? start_kernel+0x3bd/0x3c8
[31820932.809814]  [ffffffff816ab140] ? early_idt_handlers+0x140/0x140
[31820932.809819]  [ffffffff816ab3c4] ? x86_64_start_kernel+0x104/0x111
[31820932.809821] Code: 28 e0 ff ff 80 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 e8 f9 72 e7 ff 4c 29 f0 48 89 c7 e8 1f bd e5 ff 4c 69 e8 40 42    
[31820932.810371] Call Trace:
[31820932.810375]  [ffffffff810139e6] ? read_tsc+0x5/0x14
[31820932.810379]  [ffffffff812722fe] ? menu_select+0x198/0x2c1
[31820932.810383]  [ffffffff8127141d] ? cpuidle_idle_call+0xec/0x179
[31820932.810387]  [ffffffff8100d242] ? cpu_idle+0xa5/0xf2
[31820932.810392]  [ffffffff816abb3b] ? start_kernel+0x3bd/0x3c8
[31820932.810396]  [ffffffff816ab140] ? early_idt_handlers+0x140/0x140
[31820932.810401]  [ffffffff816ab3c4] ? x86_64_start_kernel+0x104/0x111
[31820932.810406] NMI backtrace for cpu 1
[31820932.810409] CPU 1 
[31820932.810411] Modules linked in: iptable_filter ip_tables x_tables tcp_diag inet_diag nfnetlink_log nfnetlink dm_mod mptctl mptbase bonding ext3 mbcache jbd loop radeon snd_pcm snd_page_alloc ttm drm_kms_helper snd_timer drm snd soundcore power_supply i2c_algo_bit joydev coretemp crc32c_intel ghash_clmulni_intel aesni_intel i2c_core aes_x86_64 aes_generic cryptd psmouse iTCO_wdt hpwdt iTCO_vendor_support i7core_edac hpilo edac_core pcspkr serio_raw acpi_power_meter container processor button evdev xfs sr_mod cdrom ata_generic usb_storage usbhid hid uhci_hcd sd_mod sg crc_t10dif ehci_hcd ata_piix usbcore libata usb_common bnx2 hpsa thermal thermal_sys scsi_mod netxen_nic [last unloaded: scsi_wait_scan]
[31820932.810471]

Nejaky napad?

A.

Odpovědi

[NN]

Bug#705618

[BigWrigley]

Diky, to vypada, ze jedinym resenim bude upgrade.

[Brad]

CPU11 se chvíli flákalo, načež bylo povoleno nemaskovatelné přerušení. Pod tím je výpis modulů kterých se to týká - co se děje dál, server spadne? Nějaké virtuály?

Zde to taky trochu probírají, vypadá to na nějakou chybku v jádře, pokud by to chování vadilo, tak teoreticky asi zkusit novější jádro (nevím jak moc je to reálné, podle popisu to řešíte na nějakém lepším serveru; za nic neručím).

[BigWrigley]

Server nespadne, nespadne nastesti ani ta MariaDB. A stalo se to zatim jen jednou. Po Vanocich zkusim bud upgrade Debianu nebo aspon novejsi jadro.