Redmineのproduction.logを監視する
目的
Redmineのパフォーマンスダウン、エラーなどを監視する。
前提条件
Redmineサーバ
- オンプレ
- Redmine 3.4.6
- nginx+unicornのリバースプロキシ構成のため、クライアントの解析は他のコンテンツも含めてnginxのログで行う。
- logstash-5.6.13
Elasticsearch/Kibana
- AmazonES
- Elasticsearch のバージョン6.3
- アクセスポリシー:「特定のIPからのドメインへのアクセスを許可」として会社のゲートウェイIPを指定
監視対象
- Redmine 3.4.6
- nginx+unicornのリバースプロキシ構成のため、クライアントの解析は他のコンテンツも含めてnginxのログで行う。
- Elasticsearch のバージョン6.3
- アクセスポリシー:「特定のIPからのドメインへのアクセスを許可」として会社のゲートウェイIPを指定
/var/log/nginx/access.log
/usr/local/redmine/log/production.log
production.logの解析対象は処理時間、ステータスコードを含む3パターンとした。
Rendered issues/index.html.erb within layouts/base (54.0ms)
Completed 200 OK in 31ms (Views: 1.2ms | ActiveRecord: 8.1ms)
Completed 302 Found in 487ms (ActiveRecord: 465.0ms)
Logstashの設定
/etc/logstash/conf.d/input.conf
input {
file {
path => ["/var/log/nginx/access.log*"]
start_position => beginning
type => "nginx"
exclude => ["*.gz"]
}
file {
path => ["/usr/local/redmine/log/production.log"]
start_position => beginning
type => "redmine"
}
}
/etc/logstash/conf.d/filter.conf
filter {
if [type] == "nginx" {
grok {
match => [ "message" , "%{COMBINEDAPACHELOG}+%{GREEDYDATA:forwarder} %{NUMBER:responsetime}"]
overwrite => [ "message" ]
}
date {
match=>["timestamp","dd/MMM/yyyy:HH:mm:ss Z"]
locale => us
remove_field => [ "timestamp" ]
}
mutate {
convert => ["response", "integer"]
convert => ["bytes", "integer"]
convert => ["responsetime", "float"]
gsub => ["referrer","\"",""]
gsub => ["agent","\"",""]
gsub => ["forwarder","\"",""]
}
grok {
match => { "request" => "^/%{WORD:first_path}/%{GREEDYDATA}$" }
tag_on_failure => ["_request_parse_failure"]
}
if "_request_parse_failure" in [tags] {
mutate {
replace => {"first_path" => "/"}
}
} else {
mutate {
replace => {"first_path" => "/%{first_path}" }
}
}
useragent {
source => "agent"
}
}
if [type] == "redmine" {
grok {
match => {
"message" => [
" Rendered %{GREEDYDATA:access_path} .*\(%{NUMBER:rendering_time}ms\)",
"Completed %{NUMBER:response}.*in %{NUMBER:total_time}ms \(Views: %{NUMBER:view_time}ms \| ActiveRecord: %{NUMBER:active_record_time}ms\)",
"Completed %{NUMBER:response}.*in %{NUMBER:total_time}ms \(ActiveRecord: %{NUMBER:active_record_time}ms\)"
]
}
}
mutate {
convert => ["rendering_time", "float"]
convert => ["total_time", "float"]
convert => ["view_time", "float"]
convert => ["active_record_time", "float"]
}
}
}
/etc/logstash/conf.d/output.conf
output {
if "_grokparsefailure" not in [tags] {
if [type] == "nginx" {
elasticsearch {
hosts => ["search-XXXXXXXXXXXX.ap-northeast-1.es.amazonaws.com:443"]
ssl => true
index => "nginx-%{+YYYY.MM.dd}"
}
}
if [type] == "redmine" {
elasticsearch {
hosts => ["search-XXXXXXXXXXXX.ap-northeast-1.es.amazonaws.com:443"]
ssl => true
index => "redmine-%{+YYYY.MM.dd}"
}
}
}
}
/etc/logstash/conf.d/input.conf
input {
file {
path => ["/var/log/nginx/access.log*"]
start_position => beginning
type => "nginx"
exclude => ["*.gz"]
}
file {
path => ["/usr/local/redmine/log/production.log"]
start_position => beginning
type => "redmine"
}
}
/etc/logstash/conf.d/filter.conf
filter {
if [type] == "nginx" {
grok {
match => [ "message" , "%{COMBINEDAPACHELOG}+%{GREEDYDATA:forwarder} %{NUMBER:responsetime}"]
overwrite => [ "message" ]
}
date {
match=>["timestamp","dd/MMM/yyyy:HH:mm:ss Z"]
locale => us
remove_field => [ "timestamp" ]
}
mutate {
convert => ["response", "integer"]
convert => ["bytes", "integer"]
convert => ["responsetime", "float"]
gsub => ["referrer","\"",""]
gsub => ["agent","\"",""]
gsub => ["forwarder","\"",""]
}
grok {
match => { "request" => "^/%{WORD:first_path}/%{GREEDYDATA}$" }
tag_on_failure => ["_request_parse_failure"]
}
if "_request_parse_failure" in [tags] {
mutate {
replace => {"first_path" => "/"}
}
} else {
mutate {
replace => {"first_path" => "/%{first_path}" }
}
}
useragent {
source => "agent"
}
}
if [type] == "redmine" {
grok {
match => {
"message" => [
" Rendered %{GREEDYDATA:access_path} .*\(%{NUMBER:rendering_time}ms\)",
"Completed %{NUMBER:response}.*in %{NUMBER:total_time}ms \(Views: %{NUMBER:view_time}ms \| ActiveRecord: %{NUMBER:active_record_time}ms\)",
"Completed %{NUMBER:response}.*in %{NUMBER:total_time}ms \(ActiveRecord: %{NUMBER:active_record_time}ms\)"
]
}
}
mutate {
convert => ["rendering_time", "float"]
convert => ["total_time", "float"]
convert => ["view_time", "float"]
convert => ["active_record_time", "float"]
}
}
}
/etc/logstash/conf.d/output.conf
output {
if "_grokparsefailure" not in [tags] {
if [type] == "nginx" {
elasticsearch {
hosts => ["search-XXXXXXXXXXXX.ap-northeast-1.es.amazonaws.com:443"]
ssl => true
index => "nginx-%{+YYYY.MM.dd}"
}
}
if [type] == "redmine" {
elasticsearch {
hosts => ["search-XXXXXXXXXXXX.ap-northeast-1.es.amazonaws.com:443"]
ssl => true
index => "redmine-%{+YYYY.MM.dd}"
}
}
}
}
既存の流用のため、AmazonESプラグインは使用しなかった。
ElasticsearchエンドポイントのFQDN+ポート443、sslの使用を設定する。
Kibanaの設定
Redmine_millisec
- Index:redmine-*
- Type:Line
- Y-Axis
- Aggregation:Max
- Field:total_time
- Y-Axis
- Aggregation:Max
- Field:active_record_time
- Y-Axis
- Aggregation:Max
- Field:rendering_time
- Y-Axis
- Aggregation:Max
- Field:view_time
- X-Axis
- Aggregation:Date Histogram
- Field: @timestamp
- Interval:Hourly
Redmine_err_count
※あらかじめ、「response.keyword:/(4|5).*/」をSearchとして登録
- Index:redmine-*
- Type:Area
- Y-Axis
- Aggregation:Count
- X-Axis
- Aggregation:Date Histogram
- Field: @timestamp
- Interval:Hourly
- Split Series
- Sub Aggregation:Terms
- Field:response.keyword
- Order By:Term
- Order:Ascending
- Size:10
Author And Source
この問題について(Redmineのproduction.logを監視する), 我々は、より多くの情報をここで見つけました https://qiita.com/ryouma_nagare/items/69e5efb4fbd5fcccc5eb著者帰属:元の著者の情報は、元のURLに含まれています。著作権は原作者に属する。
Content is automatically searched and collected through network algorithms . If there is a violation . Please contact us . We will adjust (correct author information ,or delete content ) as soon as possible .