DNS zone disappear from cPanel DNSOnly cluster

[lethol]

I run 3 cPanel servers connected to a 3 cPanel DNSOnly Cluster using Write-only configuration.

Sample DNS Path Diagram from one of our cPanel servers to the cluster
server03.domain.net
↳ ns3.domain.com
↳ ns2.domain.com
↳ ns.domain.com

Every once in a while a domain hosted in server03.domain.net stops responding to its DNS queries. When I notice this, I manually test lookups to each DNSOnly server on the cluster and they will not return any of the values configured on the zone.

After applying any change to the zone using DNS Zone Manager on server03.domain.net the cluster starts responding correctly to its queries. It seems to me the problem arises on the background when reloading the zone but I have not found any proper way to debug. I have no incidents of downtime on the cPanel server or the cluster. Because the way DNS cache works its hard to track the exact time the zone stops responding and try to debug accordingly.

Sample zone record for the affected domain:
Serial: 2021083001
Mname: ns.domain.com
Retry: 1800
Refresh: 3600
Expire: 1209600
Rname: logs.domain.com

Any help or feedback would be greatly appreciated. Thanks!

 

[cPRex]

Hey there! The best thing to do would be to enable the additional logging options for the cluster in WHM >> Tweak Settings under the "Logging" tab. This would enable additional details in the /usr/local/cpanel/logs/dnsadmin_log log file that you could use the help troubleshoot the cluster, as those would show any sync issues between the systems.

 

[lethol]

I switched 'Enable verbose logging of DNS zone syncing' to ON. Thanks

I'll leave tail running to see if anything strange pops up.

Are this logs normal?

dnsadmin 4.0 entering server mode
[2021-08-31 01:14:08 -0500] info [dnsadmin] Recovering from dormant or doing hot restart listen fd=4
[2021-08-31 01:19:08 -0500] info [dnsadmin] Switching dnsadmin to dormant mode: inactive for 5 minutes

Edit: Nevermind, I can see those logs are normal looking at our other cPanel servers.

 

[cPRex]

Yes, it's normal to see the service go in and out of dormant mode throughout the day.

 

[lethol]

It just happened to another domain on the same server and nothing came up on /usr/local/cpanel/logs/dnsadmin_log

Is there any way to increase dnsadmin logging leven on the cPanel DNSOnly side?

Thanks

 

[cPRex]

I don't think there's much logging to be found on the DNSOnly side of things, as all the logs should be happening on the webserver.

If you have a way to reproduce the issue you're always welcome to submit a ticket so we can take a look.