Awstats分析Nginx日志

时间：2023-03-09 05:23:40
Awstats分析Nginx日志

1.nginx日志格式设定

log_format  access  '$remote_addr - $remote_user [$time_local] "$request" '

           '$status $body_bytes_sent "$http_referer" '

           '"$http_user_agent" $http_x_forwarded_for';

参数说明：

$remote_addr   与$http_x_forwarded_for 用以记录客户端的ip地址；
$remote_user   ：用来记录客户端用户名称；
$time_local  ： 用来记录访问时间与时区； 
$request  ：  用来记录请求的url与http协议；
$status     ：  用来记录请求状态；成功是200，
$body_bytes_s ent  ：记录发送给客户端文件主体内容大小；
$http_referer  ：用来记录从那个页面链接访问过来的；
$http_user_agent  ：记录客户毒啊浏览器的相关信息；

2.nginx日志切割脚本

#!/bin/bash

logs_path="/home/wwwlogs/"  #设置日志文件存放目录

pid_path="/usr/local/nginx/logs/nginx.pid" #设置pid文件

mv ${logs_path}access.log ${logs_path}access_$(date -d "yesterday" +"%Y%m%d").log  #重命名日志文件，将access.log切割为每天的日志文件如access_20130728.log

kill -USR1 `cat ${pid_path}`  #向nginx主进程发信号重新打开日志

添加定时任务，每晚00:00切割，crontab -e

  * * * bash /usr/local/nginx/nginx_log.sh

3.安装和配置awstats

wget http://awstats.sourceforge.net/files/awstats-7.0.tar.gz

tar -zxvf awstats-7.0.tar.gz

mv awstats-7.0 /usr/local/awstats

chown -R root:root /usr/local/awstats

chmod -R =rwX /usr/local/awstats

chmod +x /usr/local/awstats/tools/*.pl

chmod +x /usr/local/awstats/wwwroot/cgi-bin/*.pl

执行 tools 目录中的 awstats_configure.pl 配置向导，创建一个新的统计

cd /usr/local/awstats/tools

./awstats_configure.pl

将会有如下一些提示：

-----> Running OS detected: Linux, BSD or Unix   

-----> Check for web server install

Enter full config file path of your Web server.

Example: /etc/httpd/httpd.conf

Example: /usr/local/apache2/conf/httpd.conf

Example: c:\Program files\apache group\apache\conf\httpd.conf Config file path ('none' to skip web server setup):

>none #这里添none并回车，因为我们没有使用apache

回车之后下一个选项：

Your web server config file(s) could not be found.

You will need to setup your web server manually to declare AWStats

 script as a CGI, if you want to build reports dynamically. See AWStats setup documentation (file docs/index.html)

-----> Update model config file '/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf' File awstats.model.conf updated. 

-----> Need to create a new config file ? Do you want me to build a new AWStats config/profilefile (required if first install) [y/N] ?

这里选Y，创建一个新的配置文件

-----> Define config file name to create

 What is the name of your web site or profile analysis ?

 Example: www.mysite.com

 Example: demo

Your web site, virtual server or profile name:

>nginx  #这里输入你要分析的域名，或是随便一个你易记的配置名并回车

接下来要定义你的配置文件存放的路径，可用默认

-----> Define config file path In which directory do you plan to store your config file(s) ?

 Default: /etc/awstats

Directory path to store config file(s) (Enter for default):

 > #直接回车，使用默认路径/etc/awstats

回车后的提示

-----> Create config file '/etc/awstats/awstats.nginx.conf'

Config file /etc/awstats/awstats.nginx.conf created.

 -----> Add update process inside a scheduler Sorry, configure.pl does not support automatic add to cron yet.

 You can do it manually by adding the following command to your cron: /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=nginx

Or if you have several config files and prefer having only one command: /usr/local/awstats/tools/awstats_updateall.pl now

 Press ENTER to continue...#按回车继续

 A SIMPLE config file has been created: /etc/awstats/awstats.nginx.conf You should have a look inside to check and change manually main parameters.

You can then manually update your statistics for 'yuyuanchun.com' with command: > perl awstats.pl -update -config=nginx

You can also build static report pages for 'nginx' with command: > perl awstats.pl -output=pagetype -config=nginx

Press ENTER to finish... #回车完成配置文件的创建

默认会生成一个名为awstats.nginx.conf配置文件在/etc/awstats/目录下，修改该配置文件的日志位置

vi /etc/awstats/awstats.nginx.conf

LogFile="/home/wwwlogs/access_%YYYY-0%MM-0%DD-24.log"

日志说明

# "LogFile" contains the web, ftp or mail server log file to analyze.

# You can also use tags in this filename if you need a dynamic file name

# depending on date or time (Replacement is made by AWStats at the beginning

# of its execution). This is available tags :

#   %YYYY-n  is replaced with  digits year we were n hours ago

#   %YY-n    is replaced with  digits year we were n hours ago

#   %MM-n    is replaced with  digits month we were n hours ago

#   %MO-n    is replaced with  letters month we were n hours ago

#   %DD-n    is replaced with day we were n hours ago

#   %HH-n    is replaced with hour we were n hours ago

#   %NS-n    is replaced with number of seconds at : since

#   %WM-n    is replaced with the week number in month (-)

#   %Wm-n    is replaced with the week number in month (-)

#   %WY-n    is replaced with the week number in year (-)

#   %Wy-n    is replaced with the week number in year (-)

#   %DW-n    is replaced with the day number in week (-, =sunday)

#                              use n= if you need (-, =monday)

#   %Dw-n    is replaced with the day number in week (-, =sunday)

#                              use n= if you need (-, =monday)

#   Use  for n if you need current year, month, day, hour...

n表示时间间隔，我这里就是分析距离当前今天24小时前（昨日）的日志，改成48就是前两天的日志

创建一个awstats用于记录数据的目录

mkdir -p /var/lib/awstats

然后运行awstats的wwwroot目录中的awatsts.pl来测试一下

/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=nginx

如果看到类似下面的提示就说明配置文件都正确了

Create/Update database for config "/etc/awstats/awstats.nginx.conf" by AWStats version 7.0 (build 1.964) From data

 in log file "/home/wwwlogs/access.log_20130727"... Phase  : First bypass old records, searching new record...

Direct access after last parsed record (after line ) Jumped lines in file:   Found  already parsed records.

Parsed lines in file:

Found  dropped records,

Found  comments,

Found  blank records,

Found  corrupted records,

Found  old records,

Found  new qualified records

4.Nginx 对 Perl 支持并不好，所以这里利用 awstats 的工具将统计的结果生成静态文件

首先在 webroot 目录下创建一个文件夹。例：/home/www/awstats

mkdir -p /home/www/awstats/nginx

写一个脚本，定期执行让 Awstats 把静态页面生成到该目录中，vi /usr/local/nginx/sbin/awstats.sh

/usr/local/awstats/tools/awstats_buildstaticpages.pl -update

\ -config=nginx -lang=cn -dir=/home/www/awstats/nginx

\ -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl

/usr/local/awstats/tools/awstats_buildstaticpages.pl Awstats  静态页面生成工具
-update -config=nginx 更新配置项
-lang=cn 语言为中文
-dir=/home/www/awstats 统计结果输出目录
-awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl Awstats 日志更新程序路径

添加脚本执行权限

chmod +x /usr/local/nginx/sbin/awstats.sh

运行脚本如果出现生成一堆网页类似

Launch update process : "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=nginx -update -configdir= ......

Build keywords page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=nginx -staticlinks

-lang=cn -output=keywords Build errors404 page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl"

-config=nginx -staticlinks -lang=cn -output=errors404  files built.

 Main HTML page is 'awstats.nginx.html'.

则说明成功了，接下来就可以加入定时任务了crontab -e

  * * * /usr/lcoal/nginx/sbin/awstats.sh #每天凌晨1:00执行

5.修改nginx配置文件，添加日志分析结果站点

server

        {

                listen       ;

                server_name 192.168.75.28;

                index index.html index.htm index.php default.html default.htm default.php;

                root  /home/wwwroot/awstats;

                location ^~ /icon/ {

                 root   /usr/local/awstats/wwwroot;

                 index  index.html;

                access_log off;

                                }

                autoindex on;

                access_log off;

        }

这样访问http://192.168.75.28就可以看到统计结果静态页面了。

相关文章