Hello_Filebeat

5791 ワード
Filebeat Elasticsearch Kibana Elasticsearch テキストリンク

0．About filebeat

  • Filebeat is a lightweight shipper for forwarding and centralizing log data. Installed as an agent on your servers, Filebeat monitors the log files or locations that you specify, collects log events, and forwards them to either to Elasticsearch or Logstash for indexing.
  • To configure Filebeat, you edit the configuration file. The default configuration file is called filebeat.yml.
  • Filebeat has a lot of module, which is quite easy installation for operator.

１．objective

Install Filebeat into EC2 instance(virtual machine on AWS) and create pre-defined visualization

２．Condition

  • you have Instance on AWS which has installed the following instance
  1. elasticsearch
  2. Kibana
  3. apache2

３．Command

・Install filebeat

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.0.0-darwin-x86_64.tar.gz
tar xzvf filebeat-7.0.0-darwin-x86_64.tar.gz
cd filebeat-7.0.0-darwin-x86_64/

・run filebeat

systemctl start filebeat

・confirm the filebeat status

systemctl status filebeat

active means that your filebeat runs corectly

filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
   Loaded: loaded (/usr/lib/systemd/system/filebeat.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2019-08-30 03:02:53 UTC; 2h 34min ago
     Docs: https://www.elastic.co/products/beats/filebeat

Modify filebeat.yml(/etc/filebeat) to set the connection information:

#============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:

  # Kibana Host
  # Scheme and port can be left out and will be set to the default (http and 5601)
  # In case you specify and additional path, the scheme is required: http://localhost:5601/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  host: "localhost:5601"

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["localhost:9200"]
  • activate apache module
filebeat modules enable apache

filebeat has a lot of default modules. you can just remove .disable when using module(or run the above command).

apache.yml                  googlecloud.yml.disabled  kibana.yml.disabled    netflow.yml.disabled     redis.yml.disabled
auditd.yml.disabled         haproxy.yml.disabled      logstash.yml.disabled  nginx.yml.disabled       santa.yml.disabled
cisco.yml.disabled          icinga.yml.disabled       mongodb.yml.disabled   osquery.yml.disabled     suricata.yml.disabled
coredns.yml.disabled        iis.yml.disabled          mssql.yml.disabled     panw.yml.disabled        system.yml
elasticsearch.yml.disabled  iptables.yml.disabled     mysql.yml              postgresql.yml.disabled  traefik.yml.disabled
envoyproxy.yml.disabled     kafka.yml.disabled        nats.yml.disabled      rabbitmq.yml.disabled    zeek.yml.disabled

The setup command loads the Kibana dashboards. If the dashboards are already set up, omit this command.

./filebeat setup
./filebeat -e

restart filebeat

systemctl restart filebeat

then you can find some visualization setting installed via your command

The following is the one of visualization example

and you can also confirm log message which is received to Elasticsearch.

４．Link