filebeat收集日志到es

---

前言

Filebeat版本：8.18
配置参考：https://www.elastic.co/guide/en/beats/filebeat/8.18/multiline-examples.html
下载地址：https://www.elastic.co/downloads/beats/filebeat

---

一、下载filebeat

打开：https://www.elastic.co/downloads/beats/filebeat
下载对应系统、对于版本的filebeat

二、修改配置

在/root/filebeat-8.18.2-linux-x86_64下新增bztcFilebeat.yml

vim bztcFilebeat.yml

将以下配置粘贴到yml中：

filebeat.inputs:- type: filestreamid: bztc-gatewayenabled: truepaths:- /root/bztc-gateway/bztc-log/bztc-gateway/all.logparsers:- multiline:type: patternpattern: '^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}'negate: truematch: afterfields:service_name: "gateway"fields_under_root: trueignore_older: 72hprocessors:- add_host_metadata:netinfo.enabled: true- script:lang: javascriptid: extract_logtimesource: >function process(event) {var str = event.Get("message");if (str != null) {var time = str.split(" ").slice(0, 2).join(" ");event.Put("logtime", time);}}- timestamp:field: logtimetimezone: Asia/Shanghailayouts:- '2006-01-02 15:04:05.000'- '2006-01-02 15:04:05'- type: filestreamid: bztc-notifyenabled: truepaths:- /root/bztc-notify/bztc-log/bztc-notify/all.logparsers:- multiline:type: patternpattern: '^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}'negate: truematch: afterfields:service_name: "notify"fields_under_root: trueignore_older: 72hprocessors:- add_host_metadata:netinfo.enabled: true- script:lang: javascriptid: extract_logtimesource: >function process(event) {var str = event.Get("message");if (str != null) {var time = str.split(" ").slice(0, 2).join(" ");event.Put("logtime", time);}}- timestamp:field: logtimetimezone: Asia/Shanghailayouts:- '2006-01-02 15:04:05.000'- '2006-01-02 15:04:05'output.elasticsearch:hosts: ["https://192.168.0.133:9200", "https://192.168.0.134:9200", "https://192.168.0.135:9200"]username: elasticpassword: elasticssl.certificate_authorities: ["./certs/ca.crt"]index: "bztc-log-%{[service_name]}"setup.ilm.enabled: falsesetup.template.enabled: true
setup.template.name: "bztc-log-template"
setup.template.pattern: "bztc-log-*"
setup.template.overwrite: true
setup.template.settings:index.number_of_shards: 1index.number_of_replicas: 1logging.level: info
logging.to_files: true
logging.files:path: /var/log/filebeatname: filebeatkeepfiles: 7permissions: 0640

其中的配置说明参考：https://www.elastic.co/guide/en/beats/filebeat/8.18/multiline-examples.html

三、运行

运行以下命令：

cd /root/filebeat-8.18.2-linux-x86_64
./filebeat -c bztcFilebeat.yml

运行成功后截图：

如有报错，需具体问题具体分析。

四、kibana中查看

索引：

数据流：

索引模板：

查看日志：

---

总结

filebeat收集日志到es