Restoring AWstats data from log backup question

[ca2236]

Hi @cPanelLauren,

This is a follow up from the conversation here:

Hello,

It is possible but it's a bit of a process. Here is how we do it:

extract the old log file copy the run the stats program manually against it, then move the resultant file to the correct flle name for that month:

Code:
cd /home/cPUser/tmp/awstats

cp /home/cPUser/logs/domainname.tld.bkup-Mon-20YY.gz

gzip -d domainname.tld.bkup-Mon-20YY.gz

/usr/local/cpanel/3rdparty/bin/awstats.pl -config=/home/cPUser/tmp/awstats/awstats.domainname.tld.conf -LogFile=/home/cPUser/tmp/awstats/domainnamed.tld.bkup-Mon-20YY

mv awstatsMM20YY.domainname.tld.txt awstatsMM20YY.domainname.tld.txt.bak

mv awstatsMM20YY.txt awstatsMM20YY.domainnamed.tld.txt

chown cPUser awstatsMM20YY.domainname.tld.txt
Above: replace MON with the Month (Jan, Feb, Mar, etc...), replace cPUser with the cPanel Username, domainname.tld with the domain name, MM with the month (01,02,03, etc...) and 20YY with the year.

Thank you,

I have the backup files coped over to the server, but I am not sure where the
step 1 comes into place. (for cd /home/CPUser/tmp/awstats)

and the second is just copy the domainname logs. Does that mean, just copy it anywhere?
and should I run awstats.pl on each of the log files from backup?

I apologize, I just don't want to mess up someone's stats

 

[cPanelLauren]

Hi @ca2236

You're going to copy (not move) the backup file (for the missing time) at /home/cPuser/logs/domainname.tld.bkup-Month-YYYY.gz to /home/cPUser/tmp/awstats.

The action + filename looks like this on my server:

[lauren@server awstats]# cp /home/lauren/logs/mydomain.com-Apr-2018.gz /home/lauren/tmp/awstats/

Once that's done you need to decompress the file:

[lauren@server awstats]# gzip -d /home/lauren/tmp/awstats/mydomain.com-Apr-2018.gz

Once that's complete you'll need to configure awstats to run against those new records:

[root@server awstats]# /usr/local/cpanel/3rdparty/bin/awstats.pl -config=/home/lauren/tmp/awstats/awstats.mydomain.com.conf -LogFile=/home/lauren/tmp/awstats/mydomain.com-Apr-2018
Create/Update database for config "/home/lauren/tmp/awstats/awstats.mydomain.com.conf" by AWStats version 7.6 (build 20161204)
From data in log file "/home/lauren/tmp/awstats/mydomain.com-Apr-2018"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 14043
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 14043 new qualified records.

That will create a couple new files which is why moving them is necessary hence the final steps:

mv awstatsMM20YY.domainname.tld.txt awstatsMM20YY.domainname.tld.txt.bak

mv awstatsMM20YY.txt awstatsMM20YY.domainnamed.tld.txt

chown cPUser awstatsMM20YY.domainname.tld.txt

Above: replace MON with the Month (Jan, Feb, Mar, etc...), replace cPUser with the cPanel Username, domainname.tld with the domain name, MM with the month (01,02,03, etc...) and 20YY with the year.

 

[ca2236]

Hi @cPanelLauren

I appreciate the detailed steps you have given me. One more question.

I copied the backup files from /var/log/apache2/domlogs/<domainname>/

which has two files

domainname.tld and domainname.tld-ssl_log


are these two files incorrect? They don't seem to match what you have given in example.

 

[cPanelLauren]

Hello,


Well the files in /var/log/apache2/domlogs/domainname/domainname.tld and domainname.tld-ssl_log are both files that are automatically added to awstats.

What I explained was how to extract data from the user's archived data in /home/$user/logs/ not what's in /var/log/apache2/domlogs/

Does your user you're doing this for have the missing months archived in /home/$user/logs/ ?

 

[ca2236]

His account was over quota. So I don't think he would have item in /home/$user/logs (based on the conversation in that other forum)

 

[ca2236]

I just checked, it doesn't appear that he has logs for the missing month. (or at least part of it)

is it possible to rebuild from the directory path I gave you?

 

[cPanelLauren]

Do you have the missing month in the directory you listed?

 

[ca2236]

Yes, I seem to.

 

[cPanelLauren]

If you run /scripts/runweblogs for that user does it automatically add in the user's data? If not, I believe it is possible to get the data as long as you have the weblogs but I would suggest opening a ticket using the link in my signature to ensure that is the case.

 

[ca2236]

Hi,

how do I use /scripts/runweblogs? It doesn't appear that obvious that I can copy files from a backup server, run it on those files and export them to the awstats folder

 

[cPanelLauren]

Hi @ca2236

The usage of runweblogs is just to run statistics for one user. It won't grab the data from another server if that's where it's stored. If that's the case you may want to copy the logs to the live server in /etc/apache2/domlogs/domain.tld and then use /scripts/runweblogs to update the data


Thank you,

 

[ca2236]

Thanks,

but I guess I was wondering what the syntax of the steps are.

for example, if I copy the files from the backup server to the cpanel server and put them in a directory, can I run the script or another script to generate files that awstats will read.

If so, can it be any directory? if I copy them to the directory they are supposed to be on and rename it to not overwrite the current file, will that program work?

 

[cPanelLauren]

Hello,

If the files are domlogs in the same format of the logs in /etc/apache2/domlogs/domain.tld I would suggest putting them there, then running stats.