PyTorch中nn.Linearの浅い分析

ソースの表示
Linearの初期化セクション:

class Linear(Module):
	...
	__constants__ = ['bias']
	
	def __init__(self, in_features, out_features, bias=True):
	    super(Linear, self).__init__()
	    self.in_features = in_features
	    self.out_features = out_features
	    self.weight = Parameter(torch.Tensor(out_features, in_features))
	    if bias:
	        self.bias = Parameter(torch.Tensor(out_features))
	    else:
	        self.register_parameter('bias', None)
	    self.reset_parameters()
	...

実装する必要がある内容:y=xA T+b y=xA^T+b y=xAT+b計算ステップ:

@weak_script_method
    def forward(self, input):
        return F.linear(input, self.weight, self.bias)

戻ってくるのは、input*weight+biasweightの場合

weight: the learnable weights of the module of shape
    :math:`(\text{out\_features}, \text{in\_features})`. The values are
    initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})`, where
    :math:`k = \frac{1}{\text{in\_features}}`

biasの場合

bias:   the learnable bias of the module of shape :math:`(\text{out\_features})`.
        If :attr:`bias` is ``True``, the values are initialized from
        :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where
        :math:`k = \frac{1}{\text{in\_features}}`

インスタンスの表示
例を挙げます.

>>> import torch
>>> nn1 = torch.nn.Linear(100, 50)
>>> input1 = torch.randn(140, 100)
>>> output1 = nn1(input1)
>>> output1.size()
torch.Size([140, 50])

テンソルの大きさは140 x 100から140 x 50に変わりました
実行されるアクションは、[140,100]です.× [ 100 , 50 ] = [ 140 , 50 ] [140,100]×[100,50]=[140,50] [140,100]×[100,50]=[140,50]