一、简介
在centos7系统中，默认的日志系统是rsyslog，它是一类unix系统上使用的开源工具，用于在ip网络中转发日志信息，rsyslog采用模块化设计，是syslog的替代品。
1、rsyslog特点
实现了基本的syslog协议
直接兼容syslogd的syslog.conf配置文件
在同一台机器上支持多个rsyslogd进程，支持多线程
丰富的过滤功能，可以实现过滤日志信息中的任何部分，可将消息过滤后在转发
灵活的配置选项，配置文件中可以写简单的逻辑判断，自定义输出格式等
增加了重要的功能，如使用tcp进行消息传输，支持UDP,TCP,SSL,TLS,RELP
有现成的前端web展示程序
可以使用MySQL,PGSQL,Oralce实现日志存储
默认安装的rsyslog软件包提供的守护进程是rsyslog，它是一项系统的基础服务，应该设置开机运行
2、核心组成
facility(设施，收集管道)，priority（级别），target（路径）
facility：从功能或程序上对日志进行分类
auth：认证相关
authpriv：认证权限相关
cron：计划任务相关
daemon：守护进程相关
kern：内核相关
lpr：打印相关
mail：邮件相关
news：新闻相关
security：安全相关
syslog：自身日志
user：用户相关
uucp：unix to unix copy，早期系统文件共享服务
local0-local7：用户自定义facility
priority：日志级别也可以叫loglevel
debug：调试
info：除去debug外的所有信息
notice：注意
warn，warning：警告
err，error：错误信息
crit：蓝色警报
alert：橙色警报
emerg，panic：红色警报
指定级别的方式：
*：所有级别
none：没有级别
priority：比此级别高的（包含）所有级别的日志信息都会记录
=priority：仅记录指定级别
target：路径
文件路径：将日志记录于指定的文件中；在文件路径之前使用”-“，表示异步写入；
用户：将日志信息通知给文件
*：所有用户
日志服务器：@SERVER
管道：| COMMAND
3、配置文件明细
vim /etc/rsyslog.conf
# rsyslog configuration file # For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html #### MODULES #### #模块部分
# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock #支持本地系统 provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
#$ModLoad imklog # reads kernel messages (the same are read from journald)
#$ModLoad immark # provides --MARK-- message capability # Provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun # Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun #### GLOBAL DIRECTIVES #### #全局指定
# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog # Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on # Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf # Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
$OmitLocalLogging on # File to store the position in the journal
$IMJournalStateFile imjournal.state #### RULES #### #规则部分
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
# 本地物理终端，比如启动引导的时候，打印在屏幕上的日志，可以用dmesg看
#kern.* /dev/console # Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
#可以使用分号隔开，”*.info“所有的info，排除mail ，authpriv，cron
*.info;mail.none;authpriv.none;cron.none /var/log/messages #如果想给发给日志服务器只需要指定服务器
#*.info;mail.none;authpriv.none;cron.none @192.168.216.53 # The authpriv file has restricted access.
#authpriv.* ，authpriv的任何级别
authpriv.* /var/log/secure # Log all the mail messages in one place.
#任意的mail，-/var/log/maillog 减号是异步的意思，因为不是特别关键，所以异步，节省效率
mail.* -/var/log/maillog # Log cron stuff
cron.* /var/log/cron # Everybody gets emergency messages
#所有登陆到系统上的用户的信息
*.emerg :omusrmsg:* # Save news errors of level crit and higher in a special file.
#”，“代表uucp和news都使用一个级别crit警告
uucp,news.crit /var/log/spooler # Save boot messages also to boot.log
#自定义日志，比如说添加一个local2,对应修改sshd配置文件
local7.* /var/log/boot.log
#local2.* /var/log/sshd.log
# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList # run asynchronously
#$ActionResumeRetryCount -1 # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###
2、loganalyzer是一款syslog日志和其他网络时间数据的web前端，它提供了对日志的简单浏览、搜索、基本分析和一些图表报告的功能。数据可以从数据库或一般syslog文本文件中获取，所有loganalyzer不需要改变现有的记录架构。基于当前的日志数据，它可以处理syslog日志消息windows事件日志记录，支持故障排除，使用户能快速查找日志数据中看出问题的解决方案
3、loganalyzer获取客户端日志会有两种保存模式，一种是直接读取客户端/var/log目录下的日志并保存到服务端该目录下，一种是读取后保存日志服务器数据库中，这里测试的是mariadb数据库
4、loganalyzer采用php开发，所以服务需要php的运行环境，这里用的lnmp
5、程序环境
192.168.216.51 web
192.168.216.52 mysql
192.168.216.53 rsyslog-mysql
所有软件版本
[root@web1 loganalyzer]# cat /etc/centos-release
CentOS Linux release 7.6.1810 (Core)
[root@web1 loganalyzer]# rpm -qa |grep nginx
nginx-filesystem-1.12.-.el7.noarch
nginx-1.12.2-2.el7.x86_64
nginx-mod-http-xslt-filter-1.12.-.el7.x86_64
nginx-mod-http-geoip-1.12.-.el7.x86_64
nginx-mod-http-perl-1.12.-.el7.x86_64
nginx-all-modules-1.12.-.el7.noarch
nginx-mod-mail-1.12.-.el7.x86_64
nginx-mod-http-image-filter-1.12.-.el7.x86_64
nginx-mod-stream-1.12.-.el7.x86_64
[root@web1 loganalyzer]# rpm -qa |grep php-fpm
php-fpm-5.4.16-46.el7.x86_64
[root@web1 loganalyzer]# rpm -qa |grep rsyslog
rsyslog-8.24.0-34.el7.x86_64
rsyslog-mysql-8.24.-.el7.x86_64 loganalyzer-4.1.5
[root@web3 ~]# rpm -qa |grep mariadb
2 mariadb-5.5.60-1.el7_5.x86_64
mariadb-server-5.5.-.el7_5.x86_64
mariadb-libs-5.5.-.el7_5.x86_64
[root@web3 ~]# rpm -qa |grep rsyslog
6 rsyslog-8.24.0-34.el7.x86_64
[root@web3 ~]#
二、部署
1、环境
lnmp环境用的Centos7+lnmp+zabbix4+分离mysql实验，这篇一样的环境
2、实验1：部署web的rsyslog-client，messages日志发送到rsyslog服务器
1）、51-web1服务器
vim /etc/rsyslog.conf
#*.info;mail.none;authpriv.none;cron.none /var/log/messages
2 *.info;mail.none;authpriv.none;cron.none @192.168.216.53 #指定rsyslog服务器，很简单
3 $ModLoad imudp
4 $UDPServerRun 514 #可以指定一个udp端口，默认也是udp的，看需求安排
重启服务
systemctl restart rsyslog
systemctl enable rsyslog
2）、53-web3服务器，rsyslog服务器
vim /etc/rsyslog.conf
指定端口
$ModLoad imudp
$UDPServerRun 514
systemctl restart rsyslog
systemctl enable rsyslog
3）、测试效果
51
[root@web1 web]# !system
systemctl restart nginx
53
[root@web3 ~]# tail /var/log/messages
Mar :: web3 avahi-daemon[]: Registering new address record for fe80:::e73d:1ef:2e1 on ens33.*.
Mar :: web3 avahi-daemon[]: Registering new address record for fe80::39cb:d8d1:a78b:9be1 on ens33.*.
Mar :: web3 avahi-daemon[]: Registering new address record for 192.168.216.53 on ens33.IPv4.
Mar :: web3 avahi-daemon[]: Registering HINFO record with values 'X86_64'/'LINUX'.
6 Mar 6 12:14:15 web1 systemd: Stopping The nginx HTTP and reverse proxy server...
7 Mar 6 12:14:15 web1 systemd: Stopped The nginx HTTP and reverse proxy server.
8 Mar 6 12:14:15 web1 systemd: Starting The nginx HTTP and reverse proxy server...
9 Mar 6 12:14:15 web1 nginx: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
10 Mar 6 12:14:15 web1 nginx: nginx: configuration file /etc/nginx/nginx.conf test is successful
11 Mar 6 12:14:15 web1 systemd: Started The nginx HTTP and reverse proxy server.
12 [root@web3 ~]#
3、实验2：配置基于mysql存储日志信息
1 )、51上安装数据库插件
yum install rsyslog-mysql
[root@web1 web]# rpm -ql rsyslog-mysql
/usr/lib64/rsyslog/ommysql.so
/usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql
2）、 53上安装mysql数据库，并配置
yum install mariadb-server mariadb
systemctl start mariadb
systemctl enable mariadb
[root@web3 ~]# mysql #进入mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is
Server version: 5.5.-MariaDB MariaDB Server Copyright (c) , , Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. to your MariaDB server version for the right syntax to use near 'usem mysql' at line
MariaDB [(none)]> use mysql; #修改root密码1
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A Database changed
MariaDB [mysql]> update user set password=password('admin123') where user='root'; #改root密码2
Query OK, rows affected (0.00 sec)
Rows matched: Changed: Warnings: MariaDB [mysql]> GRANT ALL ON Syslog.* TO 'Syslog'@'192.168.%.%'IDENTIFIED BY 'admin123'; #授权syslog
Query OK, rows affected (0.00 sec) MariaDB [mysql]> FLUSH PRIVILEGES; #刷新
Query OK, rows affected (0.01 sec) MariaDB [mysql]>
上面的数据库名称和syslog数据库用户，可以通过51上的/usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql文件查看
[root@web1 web]# less /usr/share/doc/rsyslog-8.24./mysql-createDB.sql
CREATE DATABASE Syslog;
USE Syslog;
CREATE TABLE SystemEvents
3）、测试，51上测试
[root@web1 web]# mysql -usyslog -h192.168.216. -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is
Server version: 5.5.-MariaDB MariaDB Server Copyright (c) , , Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]>
MariaDB [(none)]>
4）、数据库优化
vim /etc/my.cnf
skip_name_resolve=on #这个参数是禁止域名解析
innodb_file_per_table=on #共享表空间转化为独立表空间
#需要重启mariadb
systemctl restart mariasdb
5）、重定向导入数据库
51-web1
mysql -uSyslog -h192.168.216.53 -p < /usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql
6）、数据库有了，修改配置文件
51-web1
vim /etc/rsyslog.conf
1 #### MODULES #### $ModLoad ommysql
#### RULES ####
#*.info;mail.none;authpriv.none;cron.none /var/log/messages
#*.info;mail.none;authpriv.none;cron.none @192.168.216.53
*.info;mail.none;authpriv.none;cron.none :ommysql:192.168.216.53,Syslog,Syslog,admin123
systemctl restart rsyslog
7）、测试看看
51上
systemctl restart nginx
53上数据库看一下有没有写入
mysql -uroo -p
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| Syslog |
| mysql |
| performance_schema |
| syslog |
| test |
+--------------------+
rows in set (0.01 sec) MariaDB [(none)]> use Syslog;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A Database changed
MariaDB [Syslog]> show tables;
+------------------------+
| Tables_in_Syslog |
+------------------------+
| SystemEvents |
| SystemEventsProperties |
+------------------------+
rows in set (0.00 sec)
MariaDB [Syslog]> select count(*) from SystemEvents;
+----------+
| count(*) |
+----------+
| |
+----------+
row in set (0.01 sec)
MariaDB [Syslog]> select *from SystemEvents;
+----+------------+---------------------+---------------------+----------+----------+----------+------------------------------------------------------------------------------------------------------------------------------+------------+------------+-------------+-----------+---------------+---------+-----------------+--------------+-----------+----------+----------+------------+-----------+--------------+-----------------+----------+
| ID | CustomerID | ReceivedAt | DeviceReportedTime | Facility | Priority | FromHost | Message | NTSeverity | Importance | EventSource | EventUser | EventCategory | EventID | EventBinaryData | MaxAvailable | CurrUsage | MinUsage | MaxUsage | InfoUnitID | SysLogTag | EventLogType | GenericFileName | SystemID |
+----+------------+---------------------+---------------------+----------+----------+----------+------------------------------------------------------------------------------------------------------------------------------+------------+------------+-------------+-----------+---------------+---------+-----------------+--------------+-----------+----------+----------+------------+-----------+--------------+-----------------+----------+
| | NULL | -- :: | -- :: | | | web1 | Stopping System Logging Service... | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | | systemd: | NULL | NULL | NULL |
| | NULL | -- :: | -- :: | | | web1 | [origin software="rsyslogd" swVersion="8.24.0-34.el7" x-pid="" x-info="http://www.rsyslog.com"] exiting on signal . | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | | rsyslogd: | NULL | NULL | NULL |
| | NULL | -- :: | -- :: | | | web1 | Stopped System Logging Service. | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | | systemd: | NULL | NULL | NULL |
| | NULL | -- :: | -- :: | | | web1 | Starting System Logging Service... | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | | systemd: | NULL | NULL | NULL |
| | NULL | -- :: | -- :: | | | web1 | [origin software="rsyslogd" swVersion="8.24.0-34.el7" x-pid="" x-info="http://www.rsyslog.com"] start | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | | rsyslogd: | NULL | NULL | NULL |
| | NULL | -- :: | -- :: | | | web1 | Started System Logging Service. | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | | systemd: | NULL | NULL | NULL |
| | NULL | -- :: | -- :: | | | web1 | Stopping The nginx HTTP and reverse proxy server... | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | | systemd: | NULL | NULL | NULL |
| | NULL | -- :: | -- :: | | | web1 | Stopped The nginx HTTP and reverse proxy server. | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | | systemd: | NULL | NULL | NULL |
| | NULL | -- :: | -- :: | | | web1 | Starting The nginx HTTP and reverse proxy server... | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | | systemd: | NULL | NULL | NULL |
| | NULL | -- :: | -- :: | | | web1 | nginx: the configuration file /etc/nginx/nginx.conf syntax is ok | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | | nginx: | NULL | NULL | NULL |
| | NULL | -- :: | -- :: | | | web1 | nginx: configuration file /etc/nginx/nginx.conf test is successful | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | | nginx: | NULL | NULL | NULL |
| | NULL | -- :: | -- :: | | | web1 | Started The nginx HTTP and reverse proxy server. | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | | systemd: | NULL | NULL | NULL |
+----+------------+---------------------+---------------------+----------+----------+----------+------------------------------------------------------------------------------------------------------------------------------+------------+------------+-------------+-----------+---------------+---------+-----------------+--------------+-----------+----------+----------+------------+-----------+--------------+-----------------+----------+
rows in set (0.00 sec) MariaDB [Syslog]> select *from SystemEvents\G #竖排显示
4、前端显示logAnalyzer
51上配置
这里测试的为4.1.5，请自行下载loganalyzer-4.1.5.tar.gz，这里lnmp已经配置好，只需要修改nginx配置文件添加端口方式的虚拟主机
1）、环境
2）、配置loganalyzer
mkdir /loganalyzer
cp -r /loganalyzer-4.1.5/src/* /loganalyzer
cp loganalyzer-4.1.5/contri/* /loganalyzer
cd /loganalyzer
chmod +x *.sh
./configure.sh
./secure.sh
chmod 666 config.php
3）、修改nginx配置文件，添加虚拟主机
[root@web1 loganalyzer]# cat /etc/nginx/conf.d/default.conf
server {
listen ;
server_name 192.168.216.51;
index index.html index.htm index.php;
charset utf-;
root /loganalyzer;
location / {
#max_fails=;
#fail_timeout=;
root /loganalyzer;
} location ~.*\.php$ {
fastcgi_connect_timeout ;
root /loganalyzer;
fastcgi_send_timeout ;
fastcgi_read_timeout ;
#max_fails=;
#fail_timeout=;
fastcgi_buffers 16k;
fastcgi_buffer_size 32k;
#fastcgi_pass unix:/dev/shm/php-fpm.sock;
fastcgi_pass 127.0.0.1:;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
try_files $uri = ;
} location ~.*\.(gif|jpg|jpeg|png|bmp|swf|flv|mp3|wma)$ {
expires 30d;
}
location ~.*\.(js|css)$ {
expires 12h;
}
} server {
listen ;
server_name 192.168.216.51;
index index.html index.htm index.php;
charset utf-;
root /usr/share/zabbix;
location / {
#max_fails=;
#fail_timeout=;
root /usr/share/zabbix;
} location ~.*\.php$ {
fastcgi_connect_timeout ;
root /usr/share/zabbix;
fastcgi_send_timeout ;
fastcgi_read_timeout ;
#max_fails=;
#fail_timeout=;
fastcgi_buffers 16k;
fastcgi_buffer_size 32k;
#fastcgi_pass unix:/dev/shm/php-fpm.sock;
fastcgi_pass 127.0.0.1:;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
try_files $uri = ;
} location ~.*\.(gif|jpg|jpeg|png|bmp|swf|flv|mp3|wma)$ {
expires 30d;
}
location ~.*\.(js|css)$ {
expires 12h;
}
}
[root@web1 loganalyzer]# systemctl restart nginx
systemctl restart php-fpm
5、正式开始web端配置
1）开始访问，默认访问的是index.php，安装需要install.php,直接点here即可
2）、安装需求，点击下一步即可
3）检查配置，config.php没有加执行权限就会包如下错误，修改权限后请看下一步
添加了权限后，点击下一步
4）、一些基础配置，下一步即可
5）、创建第一个源，填写数据库信息后，下一步
6）、点击finish，完成
7）、展示页面
6、安全加固
config.php文件保持了数据库账号密码ip等信息，安装的时候需要加666权限，现在不在需要读权限了，改为644即可
chmod 644 /loganalyzer/config.php
7、中文语言包
1）、解压后将整个目录方至/loganalyzer/lang目录下
2）、chmod 655 -R /loganalyzer/lang/zh
3）、再次访问右上角就可以选中文了
本文参考了https://www.linuxidc.com/Linux/2017-10/147693.htm，有需要下载loganalyzer程序的还有中文语言包的可以去看看
转载请注明出处：https://www.cnblogs.com/zhangxingeng/p/10451828.html