Cygwinにfluentdをインストール～動作確認までのメモ

19218 ワード

Cygwin Fluentd Fluentd テキストリンク

暇つぶしに、Cygwin上にfluentdを入れて動作させるまでのメモ

apacheはvirtualbox上
cygwin側にrubyとfluentdを入れて動作させる(virtualboxの開発環境をあまり変更したくないため)
ruby関連まったく知りません

参考にさせていただいたサイト

Cygwin側の作業

cygwinのsetup.exeでrubyを入れる

結果

$ ruby --version
ruby 1.9.3p448 (2013-06-27) [i386-cygwin]

gemとかいうのをupdateしてfluentdをインストール

update結果

$ gem update --http-proxy http://proxy.xxx.yyy.zzz:8080
Updating installed gems
Updating bigdecimal
Fetching: bigdecimal-1.2.5.gem (100%)
Building native extensions.  This could take a while...
Successfully installed bigdecimal-1.2.5
Updating io-console
Fetching: io-console-0.4.2.gem (100%)
Building native extensions.  This could take a while...
Successfully installed io-console-0.4.2
Updating json
Fetching: json-1.8.1.gem (100%)
Building native extensions.  This could take a while...
Successfully installed json-1.8.1
Updating minitest
Fetching: minitest-5.2.2.gem (100%)
Successfully installed minitest-5.2.2
Updating rake
Fetching: rake-10.1.1.gem (100%)
Successfully installed rake-10.1.1
Updating rdoc
Fetching: rdoc-4.1.1.gem (100%)
Depending on your version of ruby, you may need to install ruby rdoc/ri data:

<= 1.8.6 : unsupported
 = 1.8.7 : gem install rdoc-data; rdoc-data --install
 = 1.9.1 : gem install rdoc-data; rdoc-data --install
>= 1.9.2 : nothing to do! Yay!
Successfully installed rdoc-4.1.1
Gems updated: bigdecimal, io-console, json, minitest, rake, rdoc
Installing ri documentation for bigdecimal-1.2.5...
Installing ri documentation for io-console-0.4.2...
Installing ri documentation for json-1.8.1...
Installing ri documentation for minitest-5.2.2...
Installing ri documentation for rake-10.1.1...
Installing ri documentation for rdoc-4.1.1...
Installing RDoc documentation for bigdecimal-1.2.5...
Installing RDoc documentation for io-console-0.4.2...
Installing RDoc documentation for json-1.8.1...
Installing RDoc documentation for minitest-5.2.2...
Installing RDoc documentation for rake-10.1.1...
Installing RDoc documentation for rdoc-4.1.1...

install結果

$ gem install fluentd --no-ri --no-rdoc --http-proxy http://proxy.xxx.yyy.zzz:8080
Fetching: msgpack-0.5.8.gem (100%)
Building native extensions.  This could take a while...
Fetching: yajl-ruby-1.2.0.gem (100%)
Building native extensions.  This could take a while...
Fetching: iobuffer-1.1.2.gem (100%)
Building native extensions.  This could take a while...
Fetching: cool.io-1.1.1.gem (100%)
Building native extensions.  This could take a while...
Fetching: http_parser.rb-0.5.3.gem (100%)
Building native extensions.  This could take a while...
Fetching: fluentd-0.10.42.gem (100%)
Successfully installed msgpack-0.5.8
Successfully installed yajl-ruby-1.2.0
Successfully installed iobuffer-1.1.2
Successfully installed cool.io-1.1.1
Successfully installed http_parser.rb-0.5.3
Successfully installed fluentd-0.10.42
6 gems installed

$ fluentd --version
fluentd 0.10.42

fluentdのセットアップ（任意のディレクトリ指定で）

$ fluentd --setup ./fluentd/fluent
Installed ./fluentd/fluent/fluent.conf.

fluentdの起動＆停止

起動コマンド（confファイル指定で）

fluentd -c ./fluentd/fluent/fluent.conf -vv &

起動結果

$ fluentd -c ./fluentd/fluent/fluent.conf -vv &
[1] 6880

$ 2013-12-03 17:51:18 +0900 [info]: fluent/supervisor.rb:171:supervise: starting fluentd-0.10.42
2013-12-03 17:51:18 +0900 [info]: fluent/supervisor.rb:273:read_config: reading config file path="./fluentd/fluent/fluent.conf"
2013-12-03 17:51:18 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered buffer plugin 'file'
2013-12-03 17:51:18 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered buffer plugin 'memory'
2013-12-03 17:51:18 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'debug_agent'
2013-12-03 17:51:18 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'exec'
2013-12-03 17:51:18 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'forward'
2013-12-03 17:51:18 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'gc_stat'
2013-12-03 17:51:18 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'http'
2013-12-03 17:51:18 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'monitor_agent'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'object_space'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'status'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'tcp'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'unix'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'syslog'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'tail'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered output plugin 'copy'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered output plugin 'exec'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered output plugin 'exec_filter'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered output plugin 'file'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered output plugin 'forward'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered output plugin 'null'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered output plugin 'roundrobin'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered output plugin 'stdout'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered output plugin 'tcp'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered output plugin 'unix'
2013-12-03 17:51:19 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered output plugin 'test'
2013-12-03 17:51:19 +0900 [info]: fluent/engine.rb:85:block in configure: gem 'fluentd' version '0.10.42'
2013-12-03 17:51:19 +0900 [info]: fluent/engine.rb:89:configure: using configuration file: <ROOT>
  <source>
    type forward
  </source>
  <source>
    type http
    port 8888
  </source>
  <source>
    type monitor_agent
    port 24220
  </source>
  <source>
    type debug_agent
    port 24230
  </source>
  <match debug.**>
    type stdout
  </match>
  <match system.**>
    type forward
    host 192.168.0.11
    <secondary>
      host 192.168.0.12
    </secondary>
  </match>
</ROOT>
2013-12-03 17:51:19 +0900 [info]: fluent/engine.rb:99:block in configure: adding source type="forward"
2013-12-03 17:51:19 +0900 [info]: fluent/engine.rb:99:block in configure: adding source type="http"
2013-12-03 17:51:19 +0900 [info]: fluent/engine.rb:99:block in configure: adding source type="monitor_agent"
2013-12-03 17:51:19 +0900 [info]: fluent/engine.rb:99:block in configure: adding source type="debug_agent"
2013-12-03 17:51:19 +0900 [info]: fluent/engine.rb:115:block in configure: adding match pattern="debug.**" type="stdout"
2013-12-03 17:51:19 +0900 [info]: fluent/engine.rb:115:block in configure: adding match pattern="system.**" type="forward"
2013-12-03 17:51:19 +0900 [warn]: plugin/out_forward.rb:57:configure: 'host' option in forward output is obsoleted. Use '<server> host xxx </server>' instead.
2013-12-03 17:51:19 +0900 [info]: plugin/out_forward.rb:87:block in configure: adding forwarding server '192.168.0.12:24224' host="192.168.0.12" port=24224 weight=60
2013-12-03 17:51:19 +0900 [warn]: plugin/out_forward.rb:57:configure: 'host' option in forward output is obsoleted. Use '<server> host xxx </server>' instead.
2013-12-03 17:51:19 +0900 [info]: plugin/out_forward.rb:87:block in configure: adding forwarding server '192.168.0.11:24224' host="192.168.0.11" port=24224 weight=60
2013-12-03 17:51:19 +0900 [debug]: plugin/out_forward.rb:170:rebuild_weight_array: rebuilding weight array lost_weight=0
2013-12-03 17:51:19 +0900 [debug]: plugin/out_forward.rb:170:rebuild_weight_array: rebuilding weight array lost_weight=0
2013-12-03 17:51:19 +0900 [info]: plugin/in_forward.rb:64:listen: listening fluent socket on 0.0.0.0:24224
2013-12-03 17:51:19 +0900 [debug]: plugin/in_http.rb:73:start: listening http on 0.0.0.0:8888
2013-12-03 17:51:19 +0900 [debug]: plugin/in_monitor_agent.rb:157:start: listening monitoring http server on http://0.0.0.0:24220/api/plugins
2013-12-03 17:51:19 +0900 [info]: plugin/in_debug_agent.rb:44:start: listening dRuby uri="druby://0.0.0.0:24230" object="Engine"
2013-12-03 17:51:56 +0900 [warn]: plugin/out_forward.rb:406:tick: detached forwarding server '192.168.0.12:24224' host="192.168.0.12" port=24224 phi=16.405017259743943
2013-12-03 17:51:56 +0900 [debug]: plugin/out_forward.rb:170:rebuild_weight_array: rebuilding weight array lost_weight=60
2013-12-03 17:51:56 +0900 [warn]: plugin/out_forward.rb:406:tick: detached forwarding server '192.168.0.11:24224' host="192.168.0.11" port=24224 phi=16.408709530721485
2013-12-03 17:51:56 +0900 [debug]: plugin/out_forward.rb:170:rebuild_weight_array: rebuilding weight array lost_weight=60

Ctrl+cで終了させて再度起動してみたら、再起動ループが発生

$ 2013-12-04 12:09:30 +0900 [info]: fluent/supervisor.rb:171:supervise: starting fluentd-0.10.42
2013-12-04 12:09:30 +0900 [info]: fluent/supervisor.rb:273:read_config: reading config file path="./fluentd/fluent/fluent.conf"
2013-12-04 12:09:30 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered buffer plugin 'file'
2013-12-04 12:09:30 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered buffer plugin 'memory'
2013-12-04 12:09:30 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'debug_agent'
2013-12-04 12:09:30 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'exec'
2013-12-04 12:09:30 +0900 [trace]: fluent/plugin.rb:87:register_impl: registered input plugin 'forward'
・・・省略・・・
2013-12-04 12:09:31 +0900 [info]: plugin/in_forward.rb:64:listen: listening fluent socket on 0.0.0.0:24224
2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:218:rescue in run: unexpected error error_class=Errno::EADDRINUSE error=#<Errno::EADDRINUSE: Address already in use - bind(2)>
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/cool.io-1.1.1/lib/cool.io/server.rb:57:in `initialize'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/cool.io-1.1.1/lib/cool.io/server.rb:57:in `new'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/cool.io-1.1.1/lib/cool.io/server.rb:57:in `initialize'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/plugin/in_forward.rb:65:in `new'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/plugin/in_forward.rb:65:in `listen'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/plugin/in_forward.rb:40:in `start'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:250:in `block in start'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:249:in `each'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:249:in `start'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:204:in `run'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:386:in `run_engine'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:94:in `block in start'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:198:in `call'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:198:in `main_process'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:173:in `block in supervise'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:172:in `fork'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:172:in `supervise'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:87:in `start'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/command/fluentd.rb:146:in `<top (required)>'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:55:in `require'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:55:in `require'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/bin/fluentd:6:in `<top (required)>'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/bin/fluentd:23:in `load'
  2013-12-04 12:09:31 +0900 [error]: fluent/engine.rb:219:rescue in run: /usr/bin/fluentd:23:in `<main>'
2013-12-04 12:09:31 +0900 [info]: fluent/engine.rb:221:run: shutting down fluentd
2013-12-04 12:09:33 +0900 [info]: fluent/supervisor.rb:188:supervise: process finished code=0
2013-12-04 12:09:33 +0900 [error]: fluent/supervisor.rb:97:start: fluentd main process died unexpectedly. restarting.
以降最初に戻って以下ループ・・・

とりあえずrubyのプロセスを片っ端からkillしたら止まった。

$kill -9 xxxx xxxx xxxx xxxx
$ 2013-12-04 12:27:36 +0900 [debug]: fluent/supervisor.rb:354:block in install_main_process_signal_handlers: fluentd main process get SIGTERM
2013-12-04 12:27:36 +0900 [debug]: fluent/supervisor.rb:357:block in install_main_process_signal_handlers: getting start to shutdown main process
2013-12-04 12:27:36 +0900 [info]: fluent/engine.rb:221:run: shutting down fluentd

起動するとrubyのプロセスが2つ起ち上がるらしいので、
これを両方ともkillすれば再起動ループは発生しない(この止め方が正しいかは不明)

$ ps -a
      PID    PPID    PGID     WINPID   TTY     UID    STIME COMMAND
     7148    6768    7148       1608  pty1    21751 14:05:57 /usr/bin/ruby
     2256    7148    7148       2256  pty1    21751 14:05:59 /usr/bin/ruby

VirtualBox側の作業

apacheの設定

apacheのaccessログをcygwinから参照できるところにも吐くように
CustomLog設定を追加してapache再起動

httpd.conf

CustomLog /media/sf_share/logs/access_log combined

以上。

apacheのアクセスログファイル→fluentで別ファイルに出力を試してみる

設定ファイルを調整(※コメントアウトはずしてパスを変えただけ)

fluent.conf

## File input
## read apache logs with tag=apache.access
<source>
  type tail
  format apache
  path /cygdrive/c/share/logs/access_log
  tag apache.access
</source>

## match tag=apache.access and write to file
<match apache.access>
  type file
  path /cygdrive/c/share/logs/fluent/access
</match>

例えばこれが、

10.0.2.2 - [email protected] [04/Dec/2013:11:41:12 +0900] "GET /testapp/controller_b/action_1 HTTP/1.1" 500 3280 "http://127.0.0.1/testapp/index.php/controller_a/action_1/param" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36"

こう出力される。

2013-12-04T11:41:12+09:00 apache.access {"host":"10.0.2.2","user":"[email protected]","method":"GET","path":"/testapp/controller_b/action_1","code":"500","size":"3280","referer":"http://127.0.0.1/testapp/index.php/controller_a/action_1/param","agent":"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36"}

で、何に使うんだろうね。

Author And Source

この問題について(Cygwinにfluentdをインストール～動作確認までのメモ), 我々は、より多くの情報をここで見つけました https://qiita.com/maryco/items/a8e154c09703e27f0a76

著者帰属：元の著者の情報は、元のURLに含まれています。著作権は原作者に属する。

Content is automatically searched and collected through network algorithms . If there is a violation . Please contact us . We will adjust (correct author information ,or delete content ) as soon as possible .

ZOJ 3686線分樹

leetcode奇技淫巧-int最大値と最小値オーバーフロー