ローカル環境のLinuxでスマートスピーカーにチャレンジ(1)

16921 ワード
conversation SpeechToText smartspeaker ubuntu16.04 node-red node-red テキストリンク

スマートスピーカーを自作してみたい

Raspberry Piではなく、ローカル環境のLinuxを使ってスマートスピーカーにチャンレジしてみたいと思います。

  • とりあえず、あとで拡張し易いようにNode-REDを使います。
  • 超小型PCにLinuxを入れてその上に構築を想定。最近の超小型PCは、指でつまめるサイズ(これ)や手のひらサイズ(あれ)があります。
  • Raspberry Pi と比べてパワーがありますので、業務システム連携するビジネス向けスマートスピーカーとして使える可能性があります。
  • ローカル環境のLinuxをベースにしておけば、あとでデスクトップPCとして再利用できるので、会社や家庭の財務責任者から許可を得やすいです。

作るものは下図です。先ずは基本の音声で対話できるチャットボットです。

  1. inject node を用いて、特定の時間間隔で実行
  2. exec node を用いて、ローカル環境のLinuxマシンに接続されているマイクで、音声をWAV形式のファイルに録音
  3. 録音したファイルを、Watson Speech to Text → Conversation → Text to Speech の順に、音声をテキスト、テキストでチャットボットと会話、会話の出力を音声に変換
  4. play Audio で音声出力

図のフローは、下記になります。実行環境に合わせて、inject node の実行タイミングの調整を行います。インポートする前に、ローカルLinux上で構築しますので、先ずはNode-REDとNode-REDに付属していない追加ノードをインストールします。

サンプルフロー


[{"id":"18d5b90c.2336d7","type":"tab","label":"local Linux speaking bot","disabled":false,"info":""},{"id":"a8986293.6b22d","type":"inject","z":"18d5b90c.2336d7","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"x":100,"y":40,"wires":[["d603c7c3.9e18b8"]]},{"id":"d603c7c3.9e18b8","type":"exec","z":"18d5b90c.2336d7","command":"arecord -t wav -f dat -d 5 /home/kolinz/out.wav","addpay":true,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"Record wav","x":270,"y":40,"wires":[["e648bef.b34274"],[],[]]},{"id":"e648bef.b34274","type":"file in","z":"18d5b90c.2336d7","name":"","filename":"/home/kolinz/out.wav","format":"","chunk":false,"sendError":false,"x":500,"y":40,"wires":[["68cff7ea.646628"]]},{"id":"68cff7ea.646628","type":"watson-speech-to-text","z":"18d5b90c.2336d7","name":"","alternatives":1,"speakerlabels":true,"smartformatting":false,"lang":"ja-JP","langhidden":"ja-JP","langcustom":"NoCustomisationSetting","langcustomhidden":"","band":"NarrowbandModel","bandhidden":"","password":"uSV0HfZxUGyX","payload-response":false,"default-endpoint":true,"service-endpoint":"https://stream.watsonplatform.net/speech-to-text/api","x":140,"y":180,"wires":[["543bf384.d4912c","2644a054.77383"]]},{"id":"543bf384.d4912c","type":"debug","z":"18d5b90c.2336d7","name":"","active":true,"console":"false","complete":"transcription","x":370,"y":140,"wires":[]},{"id":"2644a054.77383","type":"function","z":"18d5b90c.2336d7","name":"Convert to text","func":"msg.payload = msg.transcription\nreturn msg;","outputs":1,"noerr":0,"x":360,"y":220,"wires":[["ce415522.ef4d08"]]},{"id":"ce415522.ef4d08","type":"watson-conversation-v1","z":"18d5b90c.2336d7","name":"","workspaceid":"xxxxxxxxxx","multiuser":false,"context":true,"empty-payload":false,"default-endpoint":true,"service-endpoint":"https://gateway.watsonplatform.net/conversation/api","x":550,"y":220,"wires":[["b8c07837.77f4a8"]]},{"id":"f67b59d5.c9f128","type":"debug","z":"18d5b90c.2336d7","name":"conversation output","active":true,"console":"false","complete":"payload","x":670,"y":340,"wires":[]},{"id":"b8c07837.77f4a8","type":"function","z":"18d5b90c.2336d7","name":"output text","func":"msg.payload = msg.payload.output.text[0] + msg.payload.output.text[1]\nreturn msg;","outputs":1,"noerr":0,"x":430,"y":340,"wires":[["f67b59d5.c9f128","987aef3a.b55d5"]]},{"id":"987aef3a.b55d5","type":"watson-text-to-speech","z":"18d5b90c.2336d7","name":"","lang":"ja-JP","langhidden":"ja-JP","langcustom":"NoCustomisationSetting","langcustomhidden":"","voice":"ja-JP_EmiVoice","voicehidden":"","format":"audio/wav","password":"BuGdZWsCXXC4","payload-response":false,"default-endpoint":true,"service-endpoint":"https://stream.watsonplatform.net/text-to-speech/api","x":440,"y":420,"wires":[["48e6320f.5577cc"]]},{"id":"48e6320f.5577cc","type":"play audio","z":"18d5b90c.2336d7","name":"","voice":"","x":620,"y":420,"wires":[]}]

Node-REDのインストール

ローカルPCに、Linuxを入れます。今回は、Ubuntu 16.04を使用しました。見た目がWindowsライクな環境で作業したい人は、UbuntuベースのZorin OS Lite を使うと良いでしょう。とてもそっくりです。


$ curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
$ sudo apt-get install -y nodejs
$ sudo npm install -g --unsafe-perm node-red

PCを再起動

$ node-red & 
node-red をバックグラウンド起動。その後Enterキーを押す。

http://localhost:1880 にアクセスし、Node-RED初期設定後、画面右上の「三」→「パレットの管理」→「ノードを追加」の順にクリックし、不足しているノードを追加します。Node-RED初期設定については、こちらの記事の「Node-RED初期セットアップ」をご覧ください。

追加するノードは下記です。

  • node-red-contrib-play-audio
  • node-red-node-watson

IBM Cloud ライトプランでWatsonサービスを有効化

IBM Cloud ライトプランで使用可能なサービスのうちWatsonの「Conversation」「Speech to Text」「Text to Speech」をそれぞれ有効化し、接続のための資格情報を取得、メモしておきます。

上記jの他に、「Discovery」や「Language Translator」も組み込みやすいです。
また、「Conversation」を使用するためにはWorkspaceに会話ロジックを仕込んでおく必要がありますので、ConversationにおけるWorkspaceインポート用のサンプルデータを下記に記載しておきます。本来はWebサイト向けチャットボット用のデータなので、動作確認しましたら手を加えることをおすすめします。


{"name":"Tutorial Helpdesk chatbot","created":"2017-07-26T00:15:45.276Z","intents":[{"intent":"No","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","examples":[{"text":"違う","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"No","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"ではない","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"まさか","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"いいえ","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"いや","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"いえ","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"}],"description":null},{"intent":"set_hungry","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","examples":[{"text":"腹へった","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"何か食べたい","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"何かない？","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"hungry","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"おなかへった","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"}],"description":null},{"intent":"set_communication","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","examples":[{"text":"ネットが見れない","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"繋がらない","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"接続できない","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"Android","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"さんかくマークがでる","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"つながらない","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"ファイルサーバに","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"}],"description":null},{"intent":"Yes","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","examples":[{"text":"Yes","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"イエス","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"しかり","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"さよう","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"はい","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"そう","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"},{"text":"ええやん","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z"}],"description":null}],"updated":"2017-07-26T00:15:45.276Z","entities":[{"entity":"network","values":[{"type":"synonyms","value":"ネット","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["インターネット","LAN","wifi","有線","無線","無線LAN"]},{"type":"synonyms","value":"ファイルサーバー","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["サイト","ファイル","WebDAV","ファイル共有"]},{"type":"synonyms","value":"日報","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["コールメモ","議事録"]},{"type":"synonyms","value":"ログイン画面","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["ログイン","ポータルログイン"]},{"type":"synonyms","value":"メール画面","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["メール","Gmail","Outlook"]}],"created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"description":null},{"entity":"food","values":[{"type":"synonyms","value":"そば","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["かけ","ざる","つけ","もり","おかめ","おろし","きつね","たぬき","しっぽく","南蛮","天ざる","山かけ"]},{"type":"synonyms","value":"ケバブ","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["ピタ","シシカバブー","ドネルケバブ","イスケンデルケバブ","串焼き","羊肉"]},{"type":"synonyms","value":"ステーキ","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["ハム","ラム","チキン","ポーク","サイコロ","ビフテキ","ハンバーグ","アワビステーキ","マグロステーキ","タルタルステーキ","Tボーン"]},{"type":"synonyms","value":"あんかけスパゲティ","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["ピカタ","ミラカン","カントリー","ミラネーズ","ミラネーゼ","あんかけスパ","インディアン"]},{"type":"synonyms","value":"魚料理","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["ポキ","フライ","バカリャウ","アクアパッツア","刺身","塩焼き","照り焼き","煮付","煮魚","お造り"]}],"created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"description":null},{"entity":"Devices","values":[{"type":"synonyms","value":"iOS端末","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["iPad","iPad mini","iPad Pro","iPhone"]},{"type":"synonyms","value":"PC","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["Windows","Windows 10","Windows 7"]},{"type":"synonyms","value":"Mac","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["Sierra","Marvericks","El Captain","Yosemite"]},{"type":"synonyms","value":"Ubuntu","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["Trusty Tahr","Vivid","Xenial Xerus"]},{"type":"synonyms","value":"Android","created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"synonyms":["Nexus","Nougat","KitKat","Honeycomb","Lollipop","Marshmallow"]}],"created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"description":null}],"language":"ja","metadata":{"api_version":{"major_version":"v1","minor_version":"2017-05-26"},"runtime_version":"2017-02-03"},"description":null,"dialog_nodes":[{"type":"standard","title":"network","output":{"text":{"values":["<? @network ?>への接続についてですね。近くにサポートスタッフはいますか？"],"selection_policy":"sequential"}},"parent":"set_communication","context":{"network":"@network"},"created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"next_step":null,"conditions":"@network","description":null,"dialog_node":"network","previous_sibling":null},{"type":"response_condition","title":null,"output":{"text":{"values":["すみません。よくわかりません。"],"selection_policy":"sequential"}},"parent":"その他","context":null,"created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"next_step":null,"conditions":null,"description":null,"dialog_node":"node_7_1491443079313","previous_sibling":null},{"type":"standard","title":"Intent_No","output":{"text":{"values":["わかりました。お使いの端末名を教えてください。"],"selection_policy":"sequential"}},"parent":"network","context":null,"created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"next_step":{"behavior":"jump_to","selector":"user_input","dialog_node":"Devices"},"conditions":"#No","description":null,"dialog_node":"Intent_No","previous_sibling":"Intent_Yes"},{"type":"standard","title":"true","output":{"text":{"values":["端末名を入力してください。"],"selection_policy":"sequential"}},"parent":"network","context":null,"created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"next_step":null,"conditions":"true","description":null,"dialog_node":"true","previous_sibling":"Intent_No"},{"type":"standard","title":"Intent_Yes","output":{"text":{"values":["お近くのサポートスタッフにご確認ください。"],"selection_policy":"sequential"}},"parent":"network","context":null,"created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"next_step":{"behavior":"jump_to","selector":"body","dialog_node":"ようこそ"},"conditions":"#Yes","description":null,"dialog_node":"Intent_Yes","previous_sibling":null},{"type":"standard","title":"Devices","output":{"text":{"values":["<?  @Devices ?>から<? $network ?>接続ですね。如何でしょうか？\nhttps://www.google.co.jp/search?q=<? $network ?>+<? @Devices ?>"]}},"parent":null,"context":{"Devices":"@Devices"},"created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"next_step":{"behavior":"jump_to","selector":"body","dialog_node":"ようこそ"},"conditions":"@Devices","description":null,"dialog_node":"Devices","previous_sibling":"set_communication"},{"type":"standard","title":"その他","output":{},"parent":null,"context":null,"created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"next_step":null,"conditions":"anything_else","description":null,"dialog_node":"その他","previous_sibling":"Devices"},{"type":"standard","title":"set_communication","output":{"text":{"values":["どこにつなげますか？"],"selection_policy":"random"}},"parent":null,"context":null,"created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"next_step":{"behavior":"jump_to","selector":"condition","dialog_node":"network"},"conditions":"#set_communication","description":null,"dialog_node":"set_communication","previous_sibling":"ようこそ"},{"type":"standard","title":"ようこそ","output":{"text":{"values":["","何かお探しですか？","チャットボットにようこそ","何をしていますか？","どうかしましたか？"],"selection_policy":"sequential"}},"parent":null,"context":null,"created":"2017-07-26T00:15:45.276Z","updated":"2017-07-26T00:15:45.276Z","metadata":null,"next_step":null,"conditions":"welcome","description":null,"dialog_node":"ようこそ","previous_sibling":null}],"workspace_id":"e35f36ad-87b7-4e3c-a10d-406bf6c703ac","counterexamples":[],"learning_opt_out":false}

フローの読み込みと設定ポイント

あとは実際に、先ほどのサンプルフローを、ローカルのLinux環境で稼働するNode-REDでインポートします。
インポート後各ノードを設定していきますが、注意すべき場所を記載します。

  • Record wav

    • execノードを使用します。
    • コマンド欄に、arecord -t wav -f dat -d 3 /home/Linuxユーザー名/out.wav と入力。3秒間録音し、out.wavを出力という意味です。また、録音コマンドのarecordで、dat とすると音質が向上します。

  • フローをインポートした後に、上記の「Record wav」で、wav形式の録音ファイルの保存場所や、「Speech to Text」ノード、「Text to Speech」ノードでの、UsernameとPasswordの入力忘れ、「Conversation」ノードのWorkspace ID忘れに注意しましょう。

まとめ

先ずは基本の音声対話型のチャットボットにしました。無事動けば、次は天気を聞いたり、商品番号を伝えて、在庫情報を喋らせたりするように拡張すると業務向けスマートスピーカーになります。

Raspberry Piで稼働するOSは、基本的にLinuxですから、余っているPCや超小型PCにLinuxを入れて、スマートスピーカーにしてしまえば面白いだろうという考えて作り始めました。2日くらいで考えましたので、Raspberry Pi に限らず、Node-REDはとても便利なツールです。