一个logstash很容易通过http打断成两个logstash实现跨服务器或者跨平台间数据同步,比如原来的流程是
logstash: nginx log -> kafka
打断成两个是
logstash1: nginx log -> http out
logstash2: http in ->kafka
具体如下
http out
filter {
ruby {
code => "event.cancel if not event.get('message').include?'something'"
}
}
output {
http {
url => "http://test.server:10000"
codec => "plain"
format => "json"
content_type => "application/json"
http_method => "post"
}
}
可以通过filter跳过不需要的记录
http in
input {
http {
host => "0.0.0.0"
port => 10000
additional_codecs => {"application/json"=>"json"}
codec => "plain"
threads => 4
ssl => false
}
}
http in
参考:https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http.html
http out
参考:https://www.elastic.co/guide/en/logstash/current/plugins-outputs-http.html