SOLVED SystemD error during slow CPanel Update

[The Old Man]

Hi,

I'm an end user with Root access using an Inmotion Hosting Cloud VPS (48 core, 6GB RAM, SSD, PHP7.4, Redis, Zend Opcache, BBC1, CENTOS 7.9 virtuozzo).

WHM CPanel Update to 92.0.3 ran from WHM seemed very slow, taking about 20-30mins. I saw the following errors whilst it seems to wait a long time, I've copied the relevant lines below each error message. Same errors when forcing a CPanel update afterwards to make sure everything is okay. SSH login seems to be slow especially waiting for the root command line to appear, but may be coincidental

Use of uninitialized value in string eq at /usr/local/cpanel/Cpanel/RestartSrv/Systemd.pm line 138.

return 0 if $info->{'LoadState'} eq 'not-found';

Use of uninitialized value in string ne at /usr/local/cpanel/Cpanel/Init/Enable/Systemd.pm line 128.

return 0 if $info->{'LoadState'} ne 'loaded';

Systemctl keeps showing as degraded.
Running systemctl daemon-reload usually times out, daemon-reexec works quite quickly though.



Current Time: Monday, 30-Nov-2020 23:55:56 GMTRestart Time: Monday, 30-Nov-2020 23:16:27 GMTParent Server Config. Generation: 2Parent Server MPM Generation: 1Server uptime: 39 minutes 29 seconds
Server load: 0.78 1.04 1.20Total accesses: 2800 - Total Traffic: 13.0 MB - Total Duration: 534765CPU Usage: u2.59 s45.84 cu4.79 cs5.18 - 2.47% CPU load1.18 requests/sec - 5.6 kB/second - 4876 B/request - 190.988 ms/request1 requests currently being processed, 124 idle workers

I'd like to see the errors go away and normal service resumed, if possible!

Many thanks.

 

[cPRex]

Hey there! The issues you're experiencing seem like they are at a lower level than the cPanel software, as we don't control systemd. Do you see anything out of the ordinary in /var/log/messages from around the time you experienced the "systemctl daemon-reload" timeouts?

 

[The Old Man]

Hi,

Thank you. I just tried daemon-reload again and it timed out, the log shows these messages:

Dec  2 09:46:17 vps12345 systemd: Reloading.
Dec  2 09:46:24 vps12345 systemd: [/usr/lib/systemd/system/netdata.service:72] Unknown lvalue 'ProtectControlGroups' in section 'Service'
Dec  2 09:46:24 vps12345 systemd: [/etc/systemd/system/redis2.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit'
Dec  2 09:46:25 vps12345 systemd: [/etc/systemd/system/redis7.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit'
Dec  2 09:46:25 vps12345 systemd: [/etc/systemd/system/redis4.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit'
Dec  2 09:46:25 vps12345 systemd: [/etc/systemd/system/redis.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit'
Dec  2 09:46:25 vps12345 systemd: [/etc/systemd/system/redis5.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit'
Dec  2 09:46:25 vps12345 systemd: [/etc/systemd/system/redis6.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit'
Dec  2 09:46:25 vps12345 systemd: [/etc/systemd/system/redis3.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit'
Dec  2 09:46:25 vps12345 systemd: [/usr/lib/systemd/system/vzfifo.service:19] Support for option SysVStartPriority= has been removed and it is ignored

systemctl reenable redis.service
Removed symlink /etc/systemd/system/multi-user.target.wants/redis.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/redis.service to /etc/systemd/system/redis.service.
Failed to execute operation: Connection timed out


root@vps12345 [~]# systemctl reenable netdata.service
Removed symlink /etc/systemd/system/multi-user.target.wants/netdata.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/netdata.service to /usr/lib/systemd/system/netdata.service.
Failed to execute operation: Connection timed out

root@vps12345 [~]# rpm -q systemd
systemd-219-78.el7_9.2.x86_64

Thanks.

 

[cPRex]

It looks like those startup files may not be configured properly. Can you check this guide and see if that applies to your situation?

systemd script for starting my app server - Unknown lvalue 'StartLimitIntervalSec' in section 'Unit'

I have a upstart configuration file as shown below which works fine in Ubuntu 14: #/etc/init/data_server.conf #sudo start data_server #sudo stop data_server #sudo status data_server start on runl...

serverfault.com

 

[The Old Man]

Thanks for the push in the right direction! I commented out the lines concerned, and after rebooting the server normal service is resumed.
I'm not sure where they came from but it seems my version of SystemD is not up to date, so it didn't recognise the commands. I did install Loggly last week so perhaps it was responsible. There are no available updates for my server so I guess it's best to wait until perhaps CentOS 7.9 supplies a later version. I read some advice stating it wasn't advisable to update it manually.

Thanks again for the pointer.

 

[cPRex]

I'm glad things are working well! Since that isn't a tool that's related to cPanel I can't so for sure on the *why* behind the issue or what the cause may have been, but at least we were able to make some good progress!

 

[The Old Man]

Absolutely, I just did a forced CPanel update and completed in 2 mins, not the 20 odd mins like yesterday.

 

[cPRex]

Much better :D