Backup has to try several times to transfer files to external storage

[bonny3]

Goodmorning everyone,

I have a problem with the backup that is generated daily on my site.
At 2 am the backup starts and locally creates the .tar.gz file
this is about 300GB in size and takes an hour.
so around 3 in the morning
starts transferring the created file to my 2 TB external storage
The problem that I notice, which is unable to finish the transfer and restarts several times throughout the night, until arriving at 9 in the morning
which concludes the transfer

So I would like to understand what is happening in my transfer and why it tries several times to transfer the file but without success?

Thanks
Best Regards
Alessandro

 

[ZenHostingTravis]

Hi Alessandro,

Perhaps the connection is being lost so the transfer has to restart?

Is the transfer via FTP, FTPS or SFTP?

 

[HostNoc]

HI
Can you please make sure when backup initiated server had enough space to generate full backup or your vendor dont have any connectivity issue with server.
Regards

 

[bonny3]

HI
Can you please make sure when backup initiated server had enough space to generate full backup or your vendor dont have any connectivity issue with server.
Regards

the server has 1 tera of unused space available
the storege has 1.5 Tera
So more than enough memory

 

[cPRex]

Another common issue is server load - if the load is too high the backup system will pause and wait for a less busy time to try and complete.

You should be able to check /usr/local/cpanel/logs/cpbackup or /usr/local/cpanel/logs/cpbackup_transporter to get more details on the issue happening.

 

[bonny3]

Hi Alessandro,

Perhaps the connection is being lost so the transfer has to restart?

Is the transfer via FTP, FTPS or SFTP?

SFTP

 

[bonny3]

Another common issue is server load - if the load is too high the backup system will pause and wait for a less busy time to try and complete.

You should be able to check /usr/local/cpanel/logs/cpbackup or /usr/local/cpanel/logs/cpbackup_transporter to get more details on the issue happening.

this is what happens after creating the Backup File,
the problem is in the transfer

[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] Initializing log file
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] cpbackup_transporter - parent starting
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] cpbackup_transporter - child starting
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] cPanel Backup Transporter Queue Daemon started.
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] cpbackup_transporter - started
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] cpbackup_transporter - Checking queue for tasks
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] Instantiating Object
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] Starting a "copy" operation on the "mysite Storage" destination ID
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] Validating destination path 2021-07-09/accounts/
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] Making Path 2021-07-09/accounts/
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] Path creation successful
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] Uploading account backup /backup/2021-07-09/accounts/wwwmysiteit.tar.gz to 2021-07-09/accounts/wwwmysiteit.tar.gz (from 2021-07-09/accounts/wwwmysiteit.tar.gz)
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] Attempting to upload /backup/2021-07-09/accounts/wwwmysiteit.tar.gz to 2021-07-09/accounts/wwwmysiteit.tar.gz for destination: mysite Storage
[2021-07-09 03:20:09 +0200] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2021-07-09/accounts/wwwmysiteit.tar.gz to 2021-07-09/accounts/wwwmysiteit.tar.gz for destination: mysite Storage
[2021-07-09 05:20:09 +0200] warn [cpbackup_transporter] Upload attempt failed: Time out reached for upload attempt #1
at /usr/local/cpanel/Cpanel/LoggerAdapter.pm line 27.
Cpanel::LoggerAdapter::warn(Cpanel::LoggerAdapter=HASH(0x16b8be8), "Upload attempt failed: Time out reached for upload attempt #1\x{a}") called at /usr/local/cpanel/Cpanel/Backup/Queue.pm line 658
Cpanel::Backup::Queue::transport_backup::attempt_to_upload_file(Cpanel::Backup::Queue::transport_backup=HASH(0x149b6e8), Cpanel::Transport::Files::SFTP=HASH(0x24aaba8), "/backup/2021-07-09/accounts/wwwmysiteit.tar.gz", "2021-07-09/accounts/wwwmysiteit.tar.gz", Cpanel::LoggerAdapter=HASH(0x16b8be8)) called at /usr/local/cpanel/Cpanel/Backup/Queue.pm line 333
Cpanel::Backup::Queue::transport_backup::process_task(Cpanel::Backup::Queue::transport_backup=HASH(0x149b6e8), Cpanel::TaskQueue::Task=HASH(0x24aaa28), Cpanel::LoggerAdapter=HASH(0x16b8be8)) called at /usr/local/cpanel/Cpanel/TaskQueue.pm line 635
eval {...} called at /usr/local/cpanel/Cpanel/TaskQueue.pm line 638
Cpanel::TaskQueue::__ANON__() called at /usr/local/cpanel/Cpanel/StateFile.pm line 241
eval {...} called at /usr/local/cpanel/Cpanel/StateFile.pm line 241
Cpanel::StateFile::Guard::call_unlocked(Cpanel::StateFile::Guard=HASH(0x249b080), CODE(0x2341268)) called at /usr/local/cpanel/Cpanel/TaskQueue.pm line 643
Cpanel::TaskQueue::process_next_task(Cpanel::TaskQueue=HASH(0x249ad08)) called at /usr/local/cpanel/bin/cpbackup_transporter line 168
eval {...} called at /usr/local/cpanel/bin/cpbackup_transporter line 166
[2021-07-09 05:20:09 +0200] info [cpbackup_transporter] Upload attempt #2 starting for /backup/2021-07-09/accounts/wwwmysiteit.tar.gz to 2021-07-09/accounts/wwwmysiteit.tar.gz for destination: mysite Storage
[2021-07-09 07:20:09 +0200] warn [cpbackup_transporter] Upload attempt failed: Time out reached for upload attempt #2
at /usr/local/cpanel/Cpanel/LoggerAdapter.pm line 27.
Cpanel::LoggerAdapter::warn(Cpanel::LoggerAdapter=HASH(0x16b8be8), "Upload attempt failed: Time out reached for upload attempt #2\x{a}") called at /usr/local/cpanel/Cpanel/Backup/Queue.pm line 658
Cpanel::Backup::Queue::transport_backup::attempt_to_upload_file(Cpanel::Backup::Queue::transport_backup=HASH(0x149b6e8), Cpanel::Transport::Files::SFTP=HASH(0x24aaba8), "/backup/2021-07-09/accounts/wwwmysiteit.tar.gz", "2021-07-09/accounts/wwwmysiteit.tar.gz", Cpanel::LoggerAdapter=HASH(0x16b8be8)) called at /usr/local/cpanel/Cpanel/Backup/Queue.pm line 333
Cpanel::Backup::Queue::transport_backup::process_task(Cpanel::Backup::Queue::transport_backup=HASH(0x149b6e8), Cpanel::TaskQueue::Task=HASH(0x24aaa28), Cpanel::LoggerAdapter=HASH(0x16b8be8)) called at /usr/local/cpanel/Cpanel/TaskQueue.pm line 635
eval {...} called at /usr/local/cpanel/Cpanel/TaskQueue.pm line 638
Cpanel::TaskQueue::__ANON__() called at /usr/local/cpanel/Cpanel/StateFile.pm line 241
eval {...} called at /usr/local/cpanel/Cpanel/StateFile.pm line 241
Cpanel::StateFile::Guard::call_unlocked(Cpanel::StateFile::Guard=HASH(0x249b080), CODE(0x2341268)) called at /usr/local/cpanel/Cpanel/TaskQueue.pm line 643
Cpanel::TaskQueue::process_next_task(Cpanel::TaskQueue=HASH(0x249ad08)) called at /usr/local/cpanel/bin/cpbackup_transporter line 168
eval {...} called at /usr/local/cpanel/bin/cpbackup_transporter line 166
[2021-07-09 07:20:09 +0200] info [cpbackup_transporter] Upload attempt #3 starting for /backup/2021-07-09/accounts/wwwmysiteit.tar.gz to 2021-07-09/accounts/wwwmysiteit.tar.gz for destination: mysite Storage
[2021-07-09 09:20:09 +0200] warn [cpbackup_transporter] Upload attempt failed: Time out reached for upload attempt #3
at /usr/local/cpanel/Cpanel/LoggerAdapter.pm line 27.
Cpanel::LoggerAdapter::warn(Cpanel::LoggerAdapter=HASH(0x16b8be8), "Upload attempt failed: Time out reached for upload attempt #3\x{a}") called at /usr/local/cpanel/Cpanel/Backup/Queue.pm line 658
Cpanel::Backup::Queue::transport_backup::attempt_to_upload_file(Cpanel::Backup::Queue::transport_backup=HASH(0x149b6e8), Cpanel::Transport::Files::SFTP=HASH(0x24aaba8), "/backup/2021-07-09/accounts/wwwmysiteit.tar.gz", "2021-07-09/accounts/wwwmysiteit.tar.gz", Cpanel::LoggerAdapter=HASH(0x16b8be8)) called at /usr/local/cpanel/Cpanel/Backup/Queue.pm line 333
Cpanel::Backup::Queue::transport_backup::process_task(Cpanel::Backup::Queue::transport_backup=HASH(0x149b6e8), Cpanel::TaskQueue::Task=HASH(0x24aaa28), Cpanel::LoggerAdapter=HASH(0x16b8be8)) called at /usr/local/cpanel/Cpanel/TaskQueue.pm line 635
eval {...} called at /usr/local/cpanel/Cpanel/TaskQueue.pm line 638
Cpanel::TaskQueue::__ANON__() called at /usr/local/cpanel/Cpanel/StateFile.pm line 241
eval {...} called at /usr/local/cpanel/Cpanel/StateFile.pm line 241
Cpanel::StateFile::Guard::call_unlocked(Cpanel::StateFile::Guard=HASH(0x249b080), CODE(0x2341268)) called at /usr/local/cpanel/Cpanel/TaskQueue.pm line 643
Cpanel::TaskQueue::process_next_task(Cpanel::TaskQueue=HASH(0x249ad08)) called at /usr/local/cpanel/bin/cpbackup_transporter line 168
eval {...} called at /usr/local/cpanel/bin/cpbackup_transporter line 166
[2021-07-09 09:20:14 +0200] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2021-07-09 09:20:14 +0200] info [cpbackup_transporter] Instantiating Object
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] Starting a "copy_backup_metadata" operation on the "mysite Storage" destination ID
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] Uploading backup metadata file /backup/2021-07-09/accounts/.master.meta to mysite Storage
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] Validating destination path 2021-07-09/accounts/
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] Path exists
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] Uploading backup metadata /backup/2021-07-09/accounts/.master.meta to 2021-07-09/accounts/.master.meta (from 2021-07-09/accounts/.master.meta)
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] Attempting to upload /backup/2021-07-09/accounts/.master.meta to 2021-07-09/accounts/.master.meta for destination: mysite Storage
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2021-07-09/accounts/.master.meta to 2021-07-09/accounts/.master.meta for destination: mysite Storage
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] Successful transfer of /backup/2021-07-09/accounts/.master.meta to 2021-07-09/accounts/.master.meta for destination mysite Storage
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2021-07-09/accounts/.master.meta) since keeplocal (0) is disabled.
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] Instantiating Object
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] Starting a "prune" operation on the "mysite Storage" destination ID
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] Performing prune operation, retaining 2 items on: mysite Storage
[2021-07-09 09:20:15 +0200] info [cpbackup_transporter] Pruning backup directory: 2021-07-07, from mysite Storage
[2021-07-09 09:20:32 +0200] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2021-07-09 09:20:32 +0200] info [cpbackup_transporter] Removing backup staging directory: /backup/2021-07-09
[2021-07-09 09:20:43 +0200] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2021-07-09 09:21:12 +0200] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 270s for new tasks
[2021-07-09 09:21:42 +0200] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 240s for new tasks
[2021-07-09 09:22:12 +0200] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 210s for new tasks
[2021-07-09 09:22:42 +0200] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 180s for new tasks
[2021-07-09 09:23:12 +0200] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 150s for new tasks
[2021-07-09 09:23:42 +0200] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 120s for new tasks
[2021-07-09 09:24:12 +0200] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 90s for new tasks
[2021-07-09 09:24:42 +0200] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 60s for new tasks
[2021-07-09 09:25:12 +0200] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 30s for new tasks
[2021-07-09 09:25:42 +0200] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 0s for new tasks
[2021-07-09 09:25:43 +0200] info [cpbackup_transporter] cpbackup_transporter - Exiting - the queue has been emptied; no more work to do after waiting for 300s
[2021-07-09 09:25:43 +0200] info [cpbackup_transporter] cPanel Backup Transporter Queue Daemon is being stopped.

 

[cPRex]

Thanks for posting that. It definitely seems like a networking issue of some sort between the two systems. Are you able to manually connect to the remote location during the times you experience the failure?

 

[bonny3]

Thanks for posting that. It definitely seems like a networking issue of some sort between the two systems. Are you able to manually connect to the remote location during the times you experience the failure?

yes I can do it however
I notice that at a certain point the file restarts from 0 Kb
do you have any ideas to solve the problem?

 

[cPRex]

So you're saying that even with a manual transfer attempt to the remote location, the file has to restart? Is that correct? If so, you may want to reach out to your remote backup provider to see if they can get you more details about the issue.

 

[bonny3]

So you're saying that even with a manual transfer attempt to the remote location, the file has to restart? Is that correct? If so, you may want to reach out to your remote backup provider to see if they can get you more details about the issue.

on their side they found no problems ... do you have any idea?

 

[cPRex]

I don't have any other ideas on my end for this issue. You're always welcome to submit a support ticket to our team so we can check the server directly on our end. If you do that, please post the ticket number here so I can follow along and make sure this thread stays updated.