ELK应用之一:ELK平台搭建部署

时间：2023-03-09 04:03:15
ELK应用之一:ELK平台搭建部署

一、获取安装包:

EKL官网:https://www.elastic.co/products

二、安装Elasticsearch

2.1、解压安装包，在安装目录创建data和logs目录，并在配置文件中指定路径:

elasticsearch.yml:

path.data: /data/local/elasticsearch/data

path.logs: /data/local/elasticsearch/logs

bootstrap.memory_lock: false

bootstrap.system_call_filter: false

network.host: 0.0.0.0

http.port: 9200

http.enabled: true

http.cors.enabled: true
http.cors.allow-origin: "*"

elasticsearch默认不允许root用户运行，创建elasticsearch用户，并将elasticsearch安装目录拥有着修改为elasticsearch。

修改几个文件:

2.2、/etc/security/limits.conf:

*               soft    nproc           2048

*               hard    nproc           4096

*               soft    nofile          65536

*               hard    nofile          131072

2.3、/etc/sysctl.conf，添加如下:

vm.max_map_count = 655360

2.4、/etc/security/limits.d/20-nproc.conf:

*          soft    nproc     4096

发生的错误:

ERROR: [1] bootstrap checks failed

禁止bootstrap检测:

修改elasticsearch.yml，添加如下:

bootstrap.memory_lock: false

bootstrap.system_call_filter: false

以上修改后，若还提示错误，需要重启系统。

启动elasticsearch：

切换到elasticsearch用户，在安装目录/bin下直接运行:

./elasticsearch &

可以看到启动日志:

[2017-08-03T15:09:45,481][INFO ][o.e.p.PluginsService     ] [QaS1DC7] loaded module [aggs-matrix-stats]

[2017-08-03T15:09:45,481][INFO ][o.e.p.PluginsService     ] [QaS1DC7] loaded module [ingest-common]

[2017-08-03T15:09:45,481][INFO ][o.e.p.PluginsService     ] [QaS1DC7] loaded module [lang-expression]

[2017-08-03T15:09:45,481][INFO ][o.e.p.PluginsService     ] [QaS1DC7] loaded module [lang-groovy]

[2017-08-03T15:09:45,481][INFO ][o.e.p.PluginsService     ] [QaS1DC7] loaded module [lang-mustache]

[2017-08-03T15:09:45,482][INFO ][o.e.p.PluginsService     ] [QaS1DC7] loaded module [lang-painless]

[2017-08-03T15:09:45,482][INFO ][o.e.p.PluginsService     ] [QaS1DC7] loaded module [parent-join]

[2017-08-03T15:09:45,482][INFO ][o.e.p.PluginsService     ] [QaS1DC7] loaded module [percolator]

[2017-08-03T15:09:45,482][INFO ][o.e.p.PluginsService     ] [QaS1DC7] loaded module [reindex]

[2017-08-03T15:09:45,482][INFO ][o.e.p.PluginsService     ] [QaS1DC7] loaded module [transport-netty3]

[2017-08-03T15:09:45,482][INFO ][o.e.p.PluginsService     ] [QaS1DC7] loaded module [transport-netty4]

[2017-08-03T15:09:45,483][INFO ][o.e.p.PluginsService     ] [QaS1DC7] no plugins loaded

[2017-08-03T15:09:47,328][INFO ][o.e.d.DiscoveryModule    ] [QaS1DC7] using discovery type [zen]

[2017-08-03T15:09:47,878][INFO ][o.e.n.Node               ] initialized

[2017-08-03T15:09:47,879][INFO ][o.e.n.Node               ] [QaS1DC7] starting ...

[2017-08-03T15:09:48,051][INFO ][o.e.t.TransportService   ] [QaS1DC7] publish_address {192.168.1.23:9300}, bound_addresses {[::]:9300}

[2017-08-03T15:09:48,061][INFO ][o.e.b.BootstrapChecks    ] [QaS1DC7] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks

[2017-08-03T15:09:51,113][INFO ][o.e.c.s.ClusterService   ] [QaS1DC7] new_master {QaS1DC7}{QaS1DC7FQy6uyuZv4RCoEw}{9u0bZc88TCKmbcT1tOwdlw}{192.168.1.23}{192.168.1.23:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)

[2017-08-03T15:09:51,132][INFO ][o.e.h.n.Netty4HttpServerTransport] [QaS1DC7] publish_address {192.168.1.23:9200}, bound_addresses {[::]:9200}

[2017-08-03T15:09:51,132][INFO ][o.e.n.Node               ] [QaS1DC7] started

[2017-08-03T15:09:51,146][INFO ][o.e.g.GatewayService     ] [QaS1DC7] recovered [0] indices into cluster_state

2.5、安装elasticsearch-head：

elasticsearch-head是干啥的？

答:elasticsearch-head是一个可视化的管理elasticsearch集群的工具。

获取软件包:

git clone git://github.com/mobz/elasticsearch-head.git

到elasticsearch-head目录下：

npm install

注：前提需要安装nodjs。

tar zxvf node-v6.10.3.tar.gz

cd node

./configure

make

make install

编辑Gruntfile.js文件，connect修改如下:

connect: {

                        server: {

                                options: {

                                        hostname: '＊',

                                        port: 9100,

                                        base: '.',

                                        keepalive: true

                                }

                        }

                }

启动head:

/elasticsearch-head/node_modules/grunt/bin/grunt server

打开浏览器，输入IP:9100就可以看到Elasticsearch的状态了：

ELK应用之一:ELK平台搭建部署

三、安装Logstash

logstash的安装也比较简单，下载安装包后，直接解压就可以了。

测试logstash:

在解压后的config目录下创建用于测试的配置文件:

#vim test.conf

input {

    stdin {}

}

output {

    stdout {

        codec => rubydebug {}

    }

}

配置文件的说明:

文件中定义了输入和输出两部分，输入为标准输入，输出格式为codec => rubydebug {}

启动logstash:

./bin/logstash -f config/test.conf

-f参数，指定配置文件。

终端中开启交互模式，输入hello后，会被标准输出:

hello

{

    "@timestamp" => 2017-07-14T05:32:04.765Z,

      "@version" => "",

          "host" => "localhost",

       "message" => "hello"

}

配置logstash的输入和输出:

logstash支持多种输入类型

1、从普通日志文件输入:

input {

  file {

    path => "/var/log/messages"

    type => "syslog"

  }

input输入为文件时，可以将所有文件放到列表中:

path => [ "/var/log/messages", "/var/log/*.log" ]

2、从beats库输入:

beats库是一系列采集数据的插件，可以替代logstash，候问介绍。

input {

     beats {

        port => ""     ＃beats库已运行于5044端口

    }

}

3、其他输入：

logstash还支持TCP/IP、Syslog等输入，这里不详细介绍。

四、安装Kibana

kibana的安装同logstash，解压安装包后，直接运行bin下的kibana就可以启动。

配置文件:

config/kibana.yml:

erver.port: 5601     #监听的端口

server.host: "0.0.0.0"   #允许其他远程客户端访问

elasticsearch.url: "http://localhost:9200"    #连接的Elasticsearch地址

启动kibana后，就可以通过页面访问:

ELK应用之一:ELK平台搭建部署

Beats库的介绍:







相关文章