Filebeat+Logstash+Elasticsearch+Kibanaインストール(Ubuntu 16.04 LTS)
61618 ワード
Filebeatインストール
FilebeatはGo言語を使用し、java実行環境をインストールする必要はありません.
Filebeatのインストール
公式サイトからインストールパッケージをダウンロードし、解凍してインストールしたいディレクトリに入れればいいです.
うんてん
プロファイルの変更
Logstashインストール
前期準備
JDKバージョンが1.8以上であることを確認
To check your Java version, run the following command:
apt-transport-httpsのインストール
OracleJava JDKのインストール
バージョンの表示
実行結果:
インストールlogstash
公式サイトからインストールパッケージをダウンロードし、解凍してインストールしたいディレクトリに入れればいいです.
正常にインストールされたかどうかを確認
インストールが成功したかどうかをテストし、インストールディレクトリに入り、次のコマンドを実行します.
-eパラメータは、後の文を実行することを示します.フラグは、コマンドラインから直接構成を指定できます.コマンドラインで構成を指定すると、反復間でファイルを編集することなく、構成をすばやくテストできます.例のパイプは、標準入力stdinから入力を取得し、構造化された形式で標準出力stdoutに入力を移動します.
helloworldと入力し、実行結果は次のとおりです.
Logstashプロファイルを使用して実行
上記は、コマンドラインで実行され、下にはプロファイルで実行される完全な例があります.
プロファイルlogstash.conf
上記のコマンドと同じです.実行結果は同じです.
fは、どのプロファイルを使用して実行するかを指定します.
Elasticsearchインストール
前期準備
logstashと同様に、ElasticsearchはJDKバージョンが1.8以上であることを確認する必要があります.
To check your Java version, run the following command:
Elasticsearchのインストール
Elasticsearchの取得
解凍
ディレクトリをインストールディレクトリにコピーすると、インストールが完了します.
うんてん
elasticsearch binディレクトリへ
インストールに成功しました.
Kibanaインストール
Kibanaを取得
インストールディレクトリに直接解凍するとインストールが完了します
構成の変更
ここでは
Kibanaの実行:
Kibanaを訪問
参照ドキュメント:Filebeat Reference[6.5]»Getting Started With Filebeat»Step 1:Install Filebeat Elasticsearch Reference[6.5]
FilebeatはGo言語を使用し、java実行環境をインストールする必要はありません.
Filebeatのインストール
公式サイトからインストールパッケージをダウンロードし、解凍してインストールしたいディレクトリに入れればいいです.
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.5.1-linux-x86_64.tar.gz
tar xzvf filebeat-6.5.1-linux-x86_64.tar.gz
うんてん
プロファイルの変更
filebeat.yml起動./filebeat -c filebeat.yml(テスト学習フェーズはこのように起動可能)Logstashインストール
前期準備
JDKバージョンが1.8以上であることを確認
To check your Java version, run the following command:
java -version
apt-transport-httpsのインストール
sudo apt-get install apt-transport-https
OracleJava JDKのインストール
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer
バージョンの表示
java -version
実行結果:
cai@xin-x790:~$ java -version
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
インストールlogstash
公式サイトからインストールパッケージをダウンロードし、解凍してインストールしたいディレクトリに入れればいいです.
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.5.1.tar.gz
tar -zxvf logstash-6.5.1.tar.gz
正常にインストールされたかどうかを確認
インストールが成功したかどうかをテストし、インストールディレクトリに入り、次のコマンドを実行します.
bin/logstash -e 'input { stdin { } } output { stdout {} }'
-eパラメータは、後の文を実行することを示します.フラグは、コマンドラインから直接構成を指定できます.コマンドラインで構成を指定すると、反復間でファイルを編集することなく、構成をすばやくテストできます.例のパイプは、標準入力stdinから入力を取得し、構造化された形式で標準出力stdoutに入力を移動します.
helloworldと入力し、実行結果は次のとおりです.
sl@Li:/usr/local/logstash$ bin/logstash -e 'input { stdin { } } output { stdout {} }'
helloworldSending Logstash logs to /usr/local/logstash/logs which is now configured via log4j2.properties
[2018-11-30T10:32:27,261][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-11-30T10:32:27,273][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.5.0"}
[2018-11-30T10:32:29,257][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-11-30T10:32:29,394][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<0x5c08e158 run="">"}
The stdin plugin is now waiting for input:
[2018-11-30T10:32:29,525][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-11-30T10:32:29,755][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
{
"message" => "helloworld",
"@version" => "1",
"@timestamp" => 2018-11-30T02:32:47.739Z,
"host" => "Li"
}
Logstashプロファイルを使用して実行
上記は、コマンドラインで実行され、下にはプロファイルで実行される完全な例があります.
プロファイルlogstash.conf
input {
stdin {
}
}
output {
stdout {
}
}
~
上記のコマンドと同じです.実行結果は同じです.
bin/logstash -f logstash.conf
fは、どのプロファイルを使用して実行するかを指定します.
Elasticsearchインストール
前期準備
logstashと同様に、ElasticsearchはJDKバージョンが1.8以上であることを確認する必要があります.
To check your Java version, run the following command:
java -version
Elasticsearchのインストール
Elasticsearchの取得
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.5.1.tar.gz
解凍
tar -xvf elasticsearch-6.5.1.tar.gz
ディレクトリをインストールディレクトリにコピーすると、インストールが完了します.
うんてん
elasticsearch binディレクトリへ
cd elasticsearch-6.5.1/bin
./elasticsearchを実行し、間違いなければ、実行結果は以下の通りです.sl@Li:/usr/local/elasticsearch/bin$ ./elasticsearch
[2018-12-03T14:13:45,169][INFO ][o.e.e.NodeEnvironment ] [fDPpOQe] using [1] data paths, mounts [[/ (/dev/sda1)]], net usable_space [259.5gb], net total_space [310.9gb], types [ext4]
[2018-12-03T14:13:45,172][INFO ][o.e.e.NodeEnvironment ] [fDPpOQe] heap size [990.7mb], compressed ordinary object pointers [true]
[2018-12-03T14:13:45,174][INFO ][o.e.n.Node ] [fDPpOQe] node name derived from node ID [fDPpOQeySbqIdXW3AJL6fQ]; set [node.name] to override
[2018-12-03T14:13:45,174][INFO ][o.e.n.Node ] [fDPpOQe] version[6.5.1], pid[29569], build[default/tar/8c58350/2018-11-16T02:22:42.182257Z], OS[Linux/4.15.0-39-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_191/25.191-b12]
[2018-12-03T14:13:45,174][INFO ][o.e.n.Node ] [fDPpOQe] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.ZpLFrZb5, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/usr/local/elasticsearch, -Des.path.conf=/usr/local/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[2018-12-03T14:13:46,637][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [aggs-matrix-stats]
[2018-12-03T14:13:46,637][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [analysis-common]
[2018-12-03T14:13:46,637][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [ingest-common]
[2018-12-03T14:13:46,637][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [lang-expression]
[2018-12-03T14:13:46,637][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [lang-mustache]
[2018-12-03T14:13:46,637][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [lang-painless]
[2018-12-03T14:13:46,638][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [mapper-extras]
[2018-12-03T14:13:46,638][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [parent-join]
[2018-12-03T14:13:46,638][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [percolator]
[2018-12-03T14:13:46,638][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [rank-eval]
[2018-12-03T14:13:46,638][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [reindex]
[2018-12-03T14:13:46,638][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [repository-url]
[2018-12-03T14:13:46,638][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [transport-netty4]
[2018-12-03T14:13:46,638][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [tribe]
[2018-12-03T14:13:46,638][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [x-pack-ccr]
[2018-12-03T14:13:46,638][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [x-pack-core]
[2018-12-03T14:13:46,638][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [x-pack-deprecation]
[2018-12-03T14:13:46,639][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [x-pack-graph]
[2018-12-03T14:13:46,639][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [x-pack-logstash]
[2018-12-03T14:13:46,639][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [x-pack-ml]
[2018-12-03T14:13:46,639][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [x-pack-monitoring]
[2018-12-03T14:13:46,639][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [x-pack-rollup]
[2018-12-03T14:13:46,639][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [x-pack-security]
[2018-12-03T14:13:46,639][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [x-pack-sql]
[2018-12-03T14:13:46,639][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [x-pack-upgrade]
[2018-12-03T14:13:46,639][INFO ][o.e.p.PluginsService ] [fDPpOQe] loaded module [x-pack-watcher]
[2018-12-03T14:13:46,640][INFO ][o.e.p.PluginsService ] [fDPpOQe] no plugins loaded
[2018-12-03T14:13:49,306][INFO ][o.e.x.s.a.s.FileRolesStore] [fDPpOQe] parsed [0] roles from file [/usr/local/elasticsearch/config/roles.yml]
[2018-12-03T14:13:49,791][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [fDPpOQe] [controller/29677] [Main.cc@109] controller (64 bit): Version 6.5.1 (Build 1c5fe241dd9aea) Copyright (c) 2018 Elasticsearch BV
[2018-12-03T14:13:50,249][DEBUG][o.e.a.ActionModule ] [fDPpOQe] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2018-12-03T14:13:50,441][INFO ][o.e.d.DiscoveryModule ] [fDPpOQe] using discovery type [zen] and host providers [settings]
[2018-12-03T14:13:51,143][INFO ][o.e.n.Node ] [fDPpOQe] initialized
[2018-12-03T14:13:51,143][INFO ][o.e.n.Node ] [fDPpOQe] starting ...
[2018-12-03T14:13:51,323][INFO ][o.e.t.TransportService ] [fDPpOQe] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2018-12-03T14:13:51,342][WARN ][o.e.b.BootstrapChecks ] [fDPpOQe] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2018-12-03T14:13:54,404][INFO ][o.e.c.s.MasterService ] [fDPpOQe] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {fDPpOQe}{fDPpOQeySbqIdXW3AJL6fQ}{1pXLTM5-SfOGx5EVsx1v8g}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=9420918784, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2018-12-03T14:13:54,407][INFO ][o.e.c.s.ClusterApplierService] [fDPpOQe] new_master {fDPpOQe}{fDPpOQeySbqIdXW3AJL6fQ}{1pXLTM5-SfOGx5EVsx1v8g}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=9420918784, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {fDPpOQe}{fDPpOQeySbqIdXW3AJL6fQ}{1pXLTM5-SfOGx5EVsx1v8g}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=9420918784, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2018-12-03T14:13:54,463][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [fDPpOQe] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2018-12-03T14:13:54,475][INFO ][o.e.n.Node ] [fDPpOQe] started
インストールに成功しました.
Kibanaインストール
Kibanaを取得
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.5.1-linux-x86_64.tar.gz
インストールディレクトリに直接解凍するとインストールが完了します
tar -xzf kibana-6.5.1-linux-x86_64.tar.gz
構成の変更
ここでは
server.hostのみが0.0.0.0に変更されます.elasticsearchはkibanaと同じマシンに配備されているため、elasticsearch.urlは構成を変更しなくてもよいからです.その他の構成項目では、デフォルト値が使用されます.# Kibana is served by a back end server. This setting specifies the port to use.
#server.port: 5601
# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
#server.host: "localhost"
# The URL of the Elasticsearch instance to use for all your queries.
#elasticsearch.url: "http://localhost:9200"
Kibanaの実行:
sl@Li:/usr/local/kibana/bin$ ./kibana
log [05:36:39.933] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [05:36:39.969] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:39.972] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:39.977] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:39.980] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:40.024] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:40.025] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:40.038] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [05:36:40.041] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:40.049] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:40.157] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [05:36:40.158] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:40.165] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [05:36:40.168] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:40.175] [warning][security] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml
log [05:36:40.178] [warning][security] Session cookies will be transmitted over insecure connections. This is not recommended.
log [05:36:40.183] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:40.205] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:40.208] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [05:36:40.211] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:40.225] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:40.247] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [05:36:40.436] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [05:36:40.442] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [05:36:40.443] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [05:36:40.446] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [05:36:40.477] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [05:36:40.481] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [05:36:40.492] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.713] [warning][reporting] Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml
log [05:36:40.716] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [05:36:40.850] [info][license][xpack] Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active
log [05:36:40.853] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.855] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.855] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.856] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.857] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.857] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.858] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.858] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.859] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.860] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.860] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.860] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.862] [info][kibana-monitoring][monitoring-ui] Starting monitoring stats collection
log [05:36:40.870] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
log [05:36:40.923] [info][license][xpack] Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active
log [05:36:41.426] [info][listening] Server running at http://0.0.0.0:5601
log [05:36:41.434] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
Kibanaを訪問
http://localhost:5601にログインしてアクセスできます.参照ドキュメント:Filebeat Reference[6.5]»Getting Started With Filebeat»Step 1:Install Filebeat Elasticsearch Reference[6.5]