Keeping logs of all your traffic
Although we’ve mentioned AWStats a number of times before, we’ve never really gotten into how to set it up. While this isn’t exceptionally hard, it’s nice to have someone go there before you if it’s your first time out. So, here’s a rundown of installing AWStats on OS X Server 10.4.
Take special note of step 16 which tells you to create a web realm for your AWStats install. Otherwise you stand a good chance of getting bitten by some of the AWStats exploits that crop up from time to time. The Apple documentation on this should make it quite easy for you to set up the realm. These instructions were created using 10.4.7 server which currently runs perl version 5.8.6.
1. Download the latest version of AWStats from http://awstats.sourceforge.net/– at the time of writing, this was 6.5. You can download either the .zip file or the .tar.gz.
2. Extract the awstats file and move the awstats folder into /usr/local/. Change the name of the folder to just awstats
sudo mv awstats-6.5 /usr/local/awstats
3. Change into the /usr/local/awstats/tools folder and run the following command :
sudo perl awstats_configure.pl
4. Enter “Y” when prompted to continue install from NON standard directory.
5. Enter “Y” when prompted to created a new config file. Give your web site name or profile name when prompted – ie SuperDupaHostaStats or your.website.here
6. Press enter to finish the initial config.
7. We now need to verify that the main parameters in your newly created configuration file match our needs. The newly created configuration file lives here /usr/local/awstats/wwwroot/cgi-bin/awstats.SuperDupaHostaStats.conf with whatever name you entered in step 5 substituted for SuperDupaHostaStats.
8. Change the LogFile line to:
9. Check that the SiteDomain variable is set to your domain you wish to generate stats for (by default set to the name entered in step 5).
10. We now need to change the location that the awstats_updateall.pl script looks for our config files. Edit the file /usr/local/awstats/tools/awstats_updateall.pl and change the DIRCONFIG variable from etc/awstats to /usr/local/awstats/wwwroot/cgi-bin.
11. Create the awstats database directory sudo mkdir /var/lib and sudo mkdir /var/lib/awstats. Assign ownership of this directory to the www user:
sudo chown www /var/lib/awstats
12. Change your web server logging to the combined format â†’ Using server admin, edit the logging section of your website and select “combined” as the logging format. This is probably the most important step, if this is not done, the awstats script that parses the access_log file will fail with errors like “Your log file /var/log/httpd/access_log must have a bad format or LogFormat parameter setup does not match this format.” Note that if you already have a access_log file with data in it, you will probably need to archive it and start the stats from the time you change the log format.
13. You should now be able to run the awstats_updateall.pl script to create the awstats database. Run the following command
sudo /usr/local/awstats/tools/awstats_updateall.pl now
If you receive errors about log format being incorrect and you have changed the log format in the web server module of Server Admin, and make sure:
Â Â a. You have restarted the web server and
Â Â b. That you don’t have the “old” log format logs still present in the access_log file. If you do, you will need to remove them.
14. If the awstats database was successfully created, you should see some output like:
Running '"/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -update -config=SuperDupaHostaStats -configdir="/usr/local/awstats/wwwroot/cgi-bin"' to update config SuperDupaHostaStats Update for config "/usr/local/awstats/wwwroot/cgi-bin/awstats.SuperDupaHostaStats.conf" With data in log file "/var/log/httpd/access_log"... Phase 1 : First bypass old records, searching new record... Searching new records from beginning of log file... Phase 2 : Now process new records (Flush history on disk after 20000 hosts)... Jumped lines in file: 0 Parsed lines in file: 58 Found 0 dropped records, Found 49 corrupted records, Found 0 old records, Found 9 new qualified records.
15. You can then check your awstats page by browsing to http://your.web.server/awstats/awstats.pl?config=SuperDupaHostaStats
Of course, replacing your.web.server with the name/ip address of your web server and SuperDupaHostaStats with the name you entered in step 5.
16. Once you have verified that awstats are working ok, you will need to create a realm so that the awstats data is at least password protected (very easy to do with server admin, so I won’t go into detail on how to do that). You will also want to setup a cron job so that the awstats database is updated every day at least.
17. Create cron job for awstats database. Enter the following command:
sudo crontab -e
This will open roots crontab entry using the vi text editor (unless you have changed the default crontab editor). Enter editing mode (by pressing “i”) and enter the following:
0 0 * * * /usr/local/awstats/tools/awstats_updateall.pl now
This will make awstats update the database once a day every day at 12:00 am. You can change the above settings according to your preferences – ie update awstats database every hour or whatever. Check here for a good quick reference to using crontab http://www.adminschoice.com/docs/crontab.htm
18. As noted in the original AFP548 article reviewing awstats https://www.afp548.com/articles/reviews/awstats.html you will need to disable performance cache for your web server.
Thats it!!! All things going to plan, you should have awstats running successfully on your machine.
As an added bonus, this setup can also be used to generate stats for virtual sites as well.
19. When you create your virtual site, specify an unique access and error log file for each site (using Server Admin), remember to change the logging format to “combined”.
20. For each virtual site, create your awstats config file:
sudo cp /usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf /usr/local/awstats/wwwroot/cgi-bin/awstats.your.virtual.site.conf
21. Modify the settings in this file as per steps 8 and 9 above, noting that the log file will be different than in step 8, it will match what you entered in server admin in step 19.
22. Thats it!! You can manually run sudo /usr/local/awstats/tools/awstats_updateall.pl now to generate the awstats database for your virtual site or just wait for the cron job you setup earlier to run its course. You can access the awstats for your virtual site at http://your.virtual.site/awstats/awstats.pl?config=your.virtual.site of course the config=your.virtual.site must match the name given to the new awstats config file created in step 20 (without the awstats and the conf).