Rancherで作ったworkloadのログをまとめて見る。

6734 ワード
Docker rancher tail kubernetes tail テキストリンク

モチベーション

ローカルPCにRancherでk8sクラスタを立てて、
キャッキャッしながら自分で作ったアプリを動かしてみたら、
エラーが出てしょっぱいことになった。

エラーログを追跡しようと思ったが、面白全部でコンテナの数を５とかにした結果、
どのPodでログが吐かれているかわからなくなった。

この悩み、解消したい。

この記事のオチ

tail -f /var/log/containers/${ワークロード名}-${なんかのID}-*

アスタリスクのところは、PodのIDが入るっぽい。

エラー

通常時

↓戯れにPodをスケールして連打する。

エラー

たくさんのPod


うーん。どうしよう。

よくみると、Podの名前に共通点がある。
先頭のワークロード名＋なにかのID

rancherのログの項目を見る。

/var/log/containersにありそうだ。

ゴリ押しでtailする

tail -f /var/log/containers/sharemd-api-576f57c8fd-*

↓こんな感じで出力される。

[root@ssmzenbook ~]# tail -f /var/log/containers/sharemd-api-576f57c8fd-* 
==> /var/log/containers/sharemd-api-576f57c8fd-5466q_default_sharemd-api-6f6eb7dbeacf4d2c6343671b24a84d93b2bcd15a6ab29444076d6365fa07c6b6.log <==
{"log":" * Serving Flask app \"main\" (lazy loading)\r\n","stream":"stdout","time":"2018-08-12T12:46:39.120671317Z"}
{"log":" * Environment: production\r\n","stream":"stdout","time":"2018-08-12T12:46:39.120720386Z"}
{"log":"\u001b[31m   WARNING: Do not use the development server in a production environment.\u001b[0m\r\n","stream":"stdout","time":"2018-08-12T12:46:39.120728501Z"}
{"log":"\u001b[2m   Use a production WSGI server instead.\u001b[0m\r\n","stream":"stdout","time":"2018-08-12T12:46:39.120801044Z"}
{"log":" * Debug mode: off\r\n","stream":"stdout","time":"2018-08-12T12:46:39.120815698Z"}
{"log":" * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)\r\n","stream":"stdout","time":"2018-08-12T12:46:39.132532425Z"}

==> /var/log/containers/sharemd-api-576f57c8fd-65s42_default_sharemd-api-a64cc910ceb9680dbedcda27fdfae1a7d4dc172db31185cbfd4ebefbab05b897.log <==
{"log":" * Serving Flask app \"main\" (lazy loading)\r\n","stream":"stdout","time":"2018-08-12T12:46:34.070577741Z"}
{"log":" * Environment: production\r\n","stream":"stdout","time":"2018-08-12T12:46:34.070627264Z"}
{"log":"\u001b[31m   WARNING: Do not use the development server in a production environment.\u001b[0m\r\n","stream":"stdout","time":"2018-08-12T12:46:34.07069189Z"}
{"log":"\u001b[2m   Use a production WSGI server instead.\u001b[0m\r\n","stream":"stdout","time":"2018-08-12T12:46:34.070704736Z"}
{"log":" * Debug mode: off\r\n","stream":"stdout","time":"2018-08-12T12:46:34.070710825Z"}
{"log":" * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)\r\n","stream":"stdout","time":"2018-08-12T12:46:34.082531548Z"}

==> /var/log/containers/sharemd-api-576f57c8fd-g95qw_default_sharemd-api-94c6acb805fd4c6d908caa8638ec6853d4e516033ace296e3f83b6d543a629c3.log <==
{"log":" * Serving Flask app \"main\" (lazy loading)\r\n","stream":"stdout","time":"2018-08-12T12:46:37.15602442Z"}
{"log":" * Environment: production\r\n","stream":"stdout","time":"2018-08-12T12:46:37.156067984Z"}
{"log":"\u001b[31m   WARNING: Do not use the development server in a production environment.\u001b[0m\r\n","stream":"stdout","time":"2018-08-12T12:46:37.156073009Z"}
{"log":"\u001b[2m   Use a production WSGI server instead.\u001b[0m\r\n","stream":"stdout","time":"2018-08-12T12:46:37.156154701Z"}
{"log":" * Debug mode: off\r\n","stream":"stdout","time":"2018-08-12T12:46:37.156167877Z"}
{"log":" * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)\r\n","stream":"stdout","time":"2018-08-12T12:46:37.16624707Z"}

==> /var/log/containers/sharemd-api-576f57c8fd-g9hws_default_sharemd-api-3a05ac4762256a86b42c7f3661d3663e71f067cf5b8314bff2da2bcbd4804c9e.log <==
{"log":" * Serving Flask app \"main\" (lazy loading)\r\n","stream":"stdout","time":"2018-08-12T12:46:29.003961061Z"}
{"log":" * Environment: production\r\n","stream":"stdout","time":"2018-08-12T12:46:29.003993907Z"}
{"log":"\u001b[31m   WARNING: Do not use the development server in a production environment.\u001b[0m\r\n","stream":"stdout","time":"2018-08-12T12:46:29.004071121Z"}
{"log":"\u001b[2m   Use a production WSGI server instead.\u001b[0m\r\n","stream":"stdout","time":"2018-08-12T12:46:29.004085122Z"}
{"log":" * Debug mode: off\r\n","stream":"stdout","time":"2018-08-12T12:46:29.00408865Z"}
{"log":" * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)\r\n","stream":"stdout","time":"2018-08-12T12:46:29.015052717Z"}

==> /var/log/containers/sharemd-api-576f57c8fd-lkzn8_default_sharemd-api-d33a7859c37afc16e08b6afc0613fa9918690d2ae3844111cebe36ec01cb0651.log <==
{"log":" * Serving Flask app \"main\" (lazy loading)\r\n","stream":"stdout","time":"2018-08-12T12:46:31.899536133Z"}
{"log":" * Environment: production\r\n","stream":"stdout","time":"2018-08-12T12:46:31.899571802Z"}
{"log":"\u001b[31m   WARNING: Do not use the development server in a production environment.\u001b[0m\r\n","stream":"stdout","time":"2018-08-12T12:46:31.899576595Z"}
{"log":"\u001b[2m   Use a production WSGI server instead.\u001b[0m\r\n","stream":"stdout","time":"2018-08-12T12:46:31.899581022Z"}
{"log":" * Debug mode: off\r\n","stream":"stdout","time":"2018-08-12T12:46:31.899605233Z"}
{"log":" * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)\r\n","stream":"stdout","time":"2018-08-12T12:46:31.909541576Z"}

※記事を作るためにtailしたので、エラーは出てない。

色々やってて、問題は何も解消していないけど、
何も残らないのは辛いので、ここに記録を残します。
誰かの助けになれば幸いです。