[一番下からの深走り2]06言語モデル


(x0x1…x9x500x501…x509)→(x10x11…x19x510x511…x519)→…\begin{pmatrix} x_0&x_1&\dots&x_9\\x_{500}&x_{501}&\dots&x_{509}\end{pmatrix}\rarr\begin{pmatrix} x_{10}&x_{11}&\dots&x_{19}\\x_{510}&x_{511}&\dots&x_{519}\end{pmatrix}\rarr\dots(x0​x500​​x1​x501​​……​x9​x509​​)→(x10​x510​​x11​x511​​……​x19​x519​​)→… (xtWx(i)+Ht−1Wh(i)+b(i))\;\;\;\;
i =\sigma(x_tW_x^{(i)}+H_{t-1}W_h^{(i)}+b^{(i)})i=