Dotaz: Nefunkcny systemd po instalacii balika

[GeorgeWH]

CentOS 7.5 (kvm VM), po nainstalovani balika certbot (+ jeho zavislosti a automatickom update 4 balikov libsepol, libselinux, audit-libs a libsemanage) ostal system v podstate nefunkcny.

# systemctl status httpd.service
Failed to get D-Bus connection: Connection refused

Log po installacii:

Feb 20 19:40:47 server systemd: Started Cleanup of Temporary Directories.                                                                                         
Feb 20 20:07:38 server systemd: Reexecuting.                                                                                                                      
Feb 20 20:07:38 server systemd: systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
Feb 20 20:07:38 server systemd: Detected virtualization kvm.
Feb 20 20:07:38 server systemd: Detected architecture x86-64.
Feb 20 20:07:38 server systemd: Failed to connect to system bus: Connection refused
Feb 20 20:07:38 server systemd: Failed to initialize D-Bus connection: Connection refused
Feb 20 20:07:38 server yum[3477]: Updated: libsepol-2.5-10.el7.x86_64
Feb 20 20:07:38 server yum[3477]: Updated: libselinux-2.5-14.1.el7.x86_64
Feb 20 20:07:38 server yum[3477]: Installed: python2-pyasn1-0.1.9-7.el7.noarch
Feb 20 20:07:38 server yum[3477]: Installed: python-ipaddress-1.0.16-2.el7.noarch
...
Feb 20 20:07:42 server yum[3477]: Installed: python-IPy-0.75-6.el7.noarch
Feb 20 20:07:42 server yum[3477]: Installed: checkpolicy-2.5-8.el7.x86_64
Feb 20 20:07:42 server dbus[456]: [system] Reloaded configuration
Feb 20 20:07:42 server yum[3477]: Installed: policycoreutils-python-2.5-33.el7.x86_64
Feb 20 20:07:42 server yum[3477]: Installed: python2-mock-1.0.1-10.el7.noarch
...
Feb 20 20:07:43 server yum[3477]: Installed: python2-certbot-1.0.0-1.el7.noarch
Feb 20 20:07:43 server yum[3477]: Installed: certbot-1.0.0-1.el7.noarch
Feb 20 20:16:38 server systemd: Started /dev/initctl Compatibility Daemon.
Feb 20 20:16:38 server systemd: Starting /dev/initctl Compatibility Daemon...
Feb 20 20:16:38 server systemd-initctl: Failed to get D-Bus connection: Connection refused
Feb 20 20:16:38 server systemd: systemd-initctl.service: main process exited, code=exited, status=1/FAILURE
Feb 20 20:16:38 server systemd: Unit systemd-initctl.service entered failed state.
Feb 20 20:16:38 server systemd: systemd-initctl.service failed.
Feb 20 20:16:38 server systemd: Started /dev/initctl Compatibility Daemon.
Feb 20 20:16:38 server systemd: Starting /dev/initctl Compatibility Daemon...
Feb 20 20:16:38 server systemd-initctl: Failed to get D-Bus connection: Connection refused
Feb 20 20:16:38 server systemd: systemd-initctl.service: main process exited, code=exited, status=1/FAILURE
Feb 20 20:16:38 server systemd: Unit systemd-initctl.service entered failed state.
Feb 20 20:16:38 server systemd: systemd-initctl.service failed.
Feb 20 20:16:38 server systemd: Started /dev/initctl Compatibility Daemon.
Feb 20 20:16:38 server systemd: Starting /dev/initctl Compatibility Daemon...
Feb 20 20:16:38 server systemd-initctl: Failed to get D-Bus connection: Connection refused
Feb 20 20:16:38 server systemd: systemd-initctl.service: main process exited, code=exited, status=1/FAILURE
Feb 20 20:16:38 server systemd: Unit systemd-initctl.service entered failed state.
Feb 20 20:16:38 server systemd: systemd-initctl.service failed.
Feb 20 20:16:38 server systemd: Started /dev/initctl Compatibility Daemon.
Feb 20 20:16:38 server systemd: Starting /dev/initctl Compatibility Daemon...
Feb 20 20:16:38 server systemd-initctl: Failed to get D-Bus connection: Connection refused
Feb 20 20:16:38 server systemd: systemd-initctl.service: main process exited, code=exited, status=1/FAILURE
Feb 20 20:16:38 server systemd: Unit systemd-initctl.service entered failed state.
Feb 20 20:16:38 server systemd: systemd-initctl.service failed.
Feb 20 20:16:38 server systemd: Started /dev/initctl Compatibility Daemon.
Feb 20 20:16:38 server systemd: Starting /dev/initctl Compatibility Daemon...
Feb 20 20:16:38 server systemd-initctl: Failed to get D-Bus connection: Connection refused
Feb 20 20:16:38 server systemd: systemd-initctl.service: main process exited, code=exited, status=1/FAILURE
Feb 20 20:16:38 server systemd: Unit systemd-initctl.service entered failed state.
Feb 20 20:16:38 server systemd: systemd-initctl.service failed.
Feb 20 20:16:38 server systemd: start request repeated too quickly for systemd-initctl.service
Feb 20 20:16:38 server systemd: Failed to start /dev/initctl Compatibility Daemon.
Feb 20 20:16:38 server systemd: Unit systemd-initctl.socket entered failed state.
Feb 20 20:16:38 server systemd: systemd-initctl.service failed.

Do vtedy ziadny problem.

# systemctl reboot
Failed to get D-Bus connection: Connection refused

Na nete nic konkretneho, resp. riesia podobne problemy, ale za inych okolnosti.

Co s tym?

Vdaka.

---

---

Odpovědi

[GeorgeWH]

# shutdown -r now
Failed to open /dev/initctl: No such device or address
Failed to talk to init daemon.

# ps aux | grep dbus
dbus       456  0.0  0.0  58088  1160 ?        Ss    2019   0:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation

# ps aux | grep systemd
root         1  0.0  0.0  51464  3188 ?        Ss    2019  44:08 /usr/lib/systemd/systemd --system --deserialize 14
root       338  0.0  0.3  47272 14088 ?        Ss    2019   4:33 /usr/lib/systemd/systemd-journald
root       377  0.0  0.0  44464   392 ?        Ss    2019   0:00 /usr/lib/systemd/systemd-udevd
dbus       456  0.0  0.0  58088  1160 ?        Ss    2019   0:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation

[GeorgeWH]

Tak "pomohlo" poslanie Ctrl+Alt+Del - sice ostal visiet pri dosiahnuti Shutdown targetu, ale disky myslim, odpojil, takze ok.

[k3dAR]

kdyz by priste nepomohlo ani to, tak SysRq

[GeorgeWH]

Ano na to som myslel, ale potom ma este napadol ten C+A+D.

[R]

echo b >/proc/sysrq-trigger a dufat, ze to potom nabootuje. Veriaci clovek sa moze aj pomodlit, ale najlepsie k Lennartovi.

[k3dAR]

to udela reboot naprasaka, doporucuju doplnit info o tom co udelat pred tim, viz

[Filip Jirsák]

Ten systém byl nainstalován rovnou ve verzi 7.5, nebo upgradován z některé předchozí verze? Je /var/run symlink na /run ?

Podle logu systemd neumřel, ale z nějakého důvodu nefunguje D-Bus. Zajímavé je, že se se systemd nedomluvil ani systemctl pod rootem. To totiž nepoužívá komunikaci přes systémový D-Bus, ale socket /run/systemd/private. V /run má svůj socket i systémový D-Bus, takže se asi něco špatného stalo právě s /run.

Když nejde systemctl, dá se systemd domluvit aspoň pomocí signálů (man systemd, sekce SIGNALS). Případně sync; reboot -f

[GeorgeWH]

Ano, zrejme to bude tym. Uz som nieco podobne davnejsie riesil, ale spravalo sa to inac. Starsie instalacie nemali /var/run symlink na /run (bol to aj tento pripad), ale preco to vyskocilo prave teraz? Neupgradoval sa ziadny systemd balik.

Otazka c.2 - to teraz bude takto blbnut kazdy "starsi" system po upgrade?

[Peter Golis]

Dobrá otázka. Ale skúsil si sa opýtať na fóre danej distribúcie ako majú riešené to zlinkovanie /var/run na /run pri upgrade balíkov (alebo pri štarte)?

[michich]

V 7.0 už /var/run jako symlink byl. Ten systém byl upgradován z verze 6? Postup pro upgrade RHEL 6 na 7 popisuje Migration Planning Guide. Jsou k tomu nástroje preupg a redhat-upgrade-tool. Pokud vím, v CentOSu ale nejsou. Jedna z kapitol uvádí:

• 2.3.1.1. Preparing your file system for upgrade

  If /var is on a separate partition, you must manually convert /var/run and /var/lock to symbolic links: [...]

• 2.3.1.2. Verifying a successful upgrade

  [...] Check that the following symbolic links exist:

  • /bin is a symbolic link to /usr/bin
  • /sbin is a symbolic link to /usr/sbin
  • /lib is a symbolic link to /usr/lib
  • /lib64 is a symbolic link to /usr/lib64
  • /var/run is a symbolic link to /run
  • /var/lock is a symbolic link to /run/lock

preco to vyskocilo prave teraz?

RPM skriptlet při upgrade balíčku libsepol zavolal reexec systemd. Hned v tu chvíli systemd zalogoval problém s připojením na D-Bus. Myslím, že něco bylo špatně už dříve, ale až s tím reexecem se to projevilo.

[michich]

Zákazníci používající RHEL a Insights se o chybně nastaveném /var/run dozvědí. V Insights je na to definované pravidlo.

[GeorgeWH]

Bola to instalacia z live cd centos 7 cez yum --installroot=..., a potom dodatocna konfiguracia cez chroot.

Ale vsimol som si, ze poslednu dobu uz ten symlink vytvara, takze sa nieco muselo zmenit v centos 7.