ELK安装和配置及常用插件安装

时间：2023-03-08 21:05:15

环境

CentOS 7.3

root 用户

JDK 版本：1.8（最低要求），主推：JDK 1.8.0_121 以上

关闭 firewall

systemctl stop firewalld.service #停止firewall

systemctl disable firewalld.service #禁止firewall开机启动

关闭selinux

　

安装 Elasticsearch

elasticsearch运行需要使用普通用户

修改 /etc/security/limits.conf

*    soft              nofile     600000

*    hard              nofile     600000

*    soft              nproc      60000

*    hard              nproc      60000

jt_app soft memlock unlimited

jt_app hard memlock unlimited

修改/etc/sysctl.conf

net.ipv4.ip_forward = 1

net.ipv4.conf.default.rp_filter = 1

net.ipv4.conf.default.accept_source_route = 0

kernel.sysrq = 0

vm.swappiness = 0

kernel.core_uses_pid = 1

net.ipv4.tcp_syncookies = 1

kernel.msgmnb = 65536

kernel.msgmax = 65536

kernel.shmmax = 68719476736

kernel.shmall = 4294967296

net.core.somaxconn = 16384

vm.max_map_count = 262144

修改配置文件：

　生产环境主要配置：

#grep -v '^#' elasticsearch.yml|grep -v '^$'

cluster.name: prod_es_cluster

node.name: elk-log-srv01

node.master: true

node.data: true

path.data: /opt/es_data/data

path.logs: /opt/elasticsearch/logs

bootstrap.memory_lock: false

network.host: elk-log-srv01

http.port: 9200

transport.tcp.port: 9300

discovery.zen.ping_timeout: 3s

discovery.zen.fd.ping_timeout: 60s

discovery.zen.fd.ping_interval: 10s

discovery.zen.ping.unicast.hosts: ["elk-log-srv01", "elk-log-srv02","elk-log-srv03"]

discovery.zen.minimum_master_nodes: 2

gateway.recover_after_nodes: 3

gateway.expected_nodes: 3

gateway.recover_after_time: 5m

indices.query.bool.max_clause_count: 10240

http.cors.enabled: true

http.cors.allow-origin: "*"

http.cors.allow-credentials: true

search.max_buckets: 1000000

　　

　启动：

./bin/elasticsearch -d

　启动脚本：

[root@elk-log-srv01 elasticsearch]# cat /usr/lib/systemd/system/elasticsearch.service

[Unit]

Description=Elasticsearch

Documentation=http://www.elastic.co

Wants=network-online.target

After=network-online.target

[Service]

RuntimeDirectory=elasticsearch

Environment=ES_HOME=/opt/elasticsearch

Environment=ES_PATH_CONF=/opt/elasticsearch/config

Environment=PID_DIR=/opt/elasticsearch

#EnvironmentFile=-/etc/sysconfig/elasticsearch

#Environment=JAVA_HOME=/opt/jdk

WorkingDirectory=/opt/elasticsearch

User=jt_app

Group=jt_app

ExecStart=/opt/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet

# StandardOutput is configured to redirect to journalctl since

# some error messages may be logged in standard output before

# elasticsearch logging system is initialized. Elasticsearch

# stores its logs in /var/log/elasticsearch and does not use

# journalctl by default. If you also want to enable journalctl

# logging, you can simply remove the "quiet" option from ExecStart.

StandardOutput=journal

StandardError=inherit

# Specifies the maximum file descriptor number that can be opened by this process

LimitNOFILE=65536

# Specifies the maximum number of processes

LimitNPROC=4096

# Specifies the maximum size of virtual memory

LimitAS=infinity

# Specifies the maximum file size

LimitFSIZE=infinity

#

LimitMEMLOCK=infinity

# Disable timeout logic and wait until process is stopped

TimeoutStopSec=0

# SIGTERM signal is used to stop the Java process

KillSignal=SIGTERM

# Send the signal only to the JVM rather than its control group

KillMode=process

# Java process is never killed

SendSIGKILL=no

# When a JVM receives a SIGTERM signal it exits with code 143

SuccessExitStatus=143

[Install]

WantedBy=multi-user.target

# Built for packages-6.3.2 (packages)

[root@elk-log-srv01 elasticsearch]#

　

安装 Kibana

选择一台节点安装即可
进入安装目录修改配置文件：

    config/kibana.yml

server.port: 5601                                  #端口

server.host: "elk-log-srv01"                       #访问ip地址

elasticsearch.url: "http://elk-log-srv01:9200"      #连接elastic

kibana.index: ".kibana"                            #在elastic中添加.kibana索引

pid.file: /opt/kibana/kibana.pid

logging.dest: /opt/kibana/kibana.log

　启动：

nohup ./bin/kibana &

logstash安装

elasticsearch 常用插件安装

只是版本不一样，方法是一样的，替换成自己的版本即可

采用离线安装插件的方法

1、sql插件

### 项目地址

https://github.com/NLPchina/elasticsearch-sql

历史版本：

https://github.com/NLPchina/elasticsearch-sql/releases

### 下载sql插件

下载

wget https://github.com/NLPchina/elasticsearch-sql/releases/download/5.5.1.0/elasticsearch-sql-5.5.1.0.zip

安装

./bin/elasticsearch-plugin install file:///opt/elasticsearch-sql-5.5.1.0.zip

安装web访问

wget https://github.com/NLPchina/elasticsearch-sql/releases/download/5.4.1.0/es-sql-site-standalone.zip

unzip ./es-sql-site-standalone.zip

cd site-server

npm install express --save

node node-server.js & #后台启动

默认端口：8080

cd _site

vim controllers.js

修改链接es地址

url = "http://localhost:9200"

　2 分词器

项目地址：

https://github.com/medcl/elasticsearch-analysis-ik/

下载地址：

wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v5.5.1/elasticsearch-analysis-ik-5.5.1.zip

安装插件

./bin/elasticsearch-plugin install file:///opt/elasticsearch-analysis-ik-5.5.1.zip

　　

elasticsearch5.x安装head插件

5.0以上版本中不支持直接安装head插件，需要启动一个服务。

由于head插件本质上还是一个nodejs的工程，因此需要安装node，使用npm来安装依赖的包。（npm可以理解为maven）

#安装git

yum -y install git

#下载源码

git clone git://github.com/mobz/elasticsearch-head.git

安装 nodejs，修改环境变量

node -v

2、安装npm

3、使用npm安装grunt

由于 npm 是国外的源，下载速度比较慢，推荐使用国内淘宝镜像

npm install -g cnpm --registry=https://registry.npm.taobao.org

下面开始修改 head 插件的配置

地址：

https://github.com/mobz/elasticsearch-head

　　

cerebro插件安装

以单独进程启动

下载

wget https://github.com/lmenezes/cerebro/releases/download/v0.6.6/cerebro-0.6.6.zip

解压

unzip cerebro-0.6.6.zip

启动：

bin/cerebro -Dhttp.port=1234 -Dhttp.address=0.0.0.0 &

------------------------------------------------

其他配置

-Dconfig.file=/some/other/dir/alternate.conf

##项目地址

https://github.com/lmenezes/cerebro

　

kibana安装x-pack插件

先下载x-pack-5.5.1.zip

https://artifacts.elastic.co/downloads/packs

在线安装

bin/kibana-plugin install x-pack

离线安装

./bin/kibana-plugin install file:///opt/x-pack-5.5.1.zip

elasticsearch安装此插件一样

　　

相关文章