[PyTorch] Lab05 - Logistic Regression

27776 ワード
PyTorch テキストリンク

📌 学習目標

  • Logistic Regression理論
  • Logistic Regressionの実施
  • Logistic Regression with nn.Module

    • Logistic Regression理論


  • Logistic RegressionのHypothesis
    H(x)=11+e−WTXH(x) =\frac{1}{1 + e^{-W^{T}X}}H(x)=1+e−WTX1​

  • Logistic Regressionのコスト
    cost(W)=−1m∑ylog(H(x))+(1−y)(log(1−H(x)))cost(W) = -\frac{1}{m}\sum{ylog(H(x)) + (1 - y)(log(1-H(x)))}cost(W)=−m1​∑ylog(H(x))+(1−y)(log(1−H(x)))

    • Logistic Regressionの実装


  • Logistic Regressionを実装するHypothesis
    H(x)=11+e−WTXH(x) =\frac{1}{1 + e^{-W^{T}X}}H(x)=1+e−WTX1​
    exp使用torch.exp()(使用torch.sigmoid()容易.)
    重み付け計算はmatmul()メソッドを使用します.
    • import torch

W = torch.zeros((2, 1), requires_grad=True)
b = torch.zeros(1, requires_grad=True)

x_data = [[1, 2], [2, 3], [3, 1], [4, 3], [5, 3], [6, 2]] 
y_data = [[0], [0], [0], [1], [1], [1]]

x_train = torch.FloatTensor(x_data) # (6, 2)
y_train = torch.FloatTensor(y_data) # (6, 1)

hypothesis = 1 / (1 + torch.exp(-(x_train.matmul(W) + b)))

print(hypothesis)
print(hypothesis.shape)
'''
tensor([[0.5000],
        [0.5000],
        [0.5000],
        [0.5000],
        [0.5000],
        [0.5000]], grad_fn=<MulBackward0>)
torch.Size([6, 1])
'''

  • Logistic RegressionのCostを実施
    cost(W)=−1m∑ylog(H(x))+(1−y)(log(1−H(x)))cost(W) = -\frac{1}{m}\sum{ylog(H(x)) + (1 - y)(log(1-H(x)))}cost(W)=−m1​∑ylog(H(x))+(1−y)(log(1−H(x)))
    使用torch.log()和torch.mean()使用可能torch.nn.functional.binary_cross_entropy()
    • import torch
import torch.nn.functional as F

losses = -(y_train*torch.log(hypothesis) + (1-y_train)*torch.log(1-hypothesis))
print(losses)

cost = losses.mean()
print(cost)

print(F.binary_cross_entropy(hypothesis, y_train))
'''
tensor([[0.6931],
        [0.6931],
        [0.6931],
        [0.6931],
        [0.6931],
        [0.6931]], grad_fn=<NegBackward>)
tensor(0.6931, grad_fn=<MeanBackward0>)
tensor(0.6931, grad_fn=<BinaryCrossEntropyBackward>)
'''
  • Logistic Regression実装フルコード
    • import torch
import torch.nn.functional as F
import torch.optim as optim

x_data = [[1, 2], [2, 3], [3, 1], [4, 3], [5, 3], [6, 2]] # (6, 2)
y_data = [[0], [0], [0], [1], [1], [1]]

x_train = torch.FloatTensor(x_data)
y_train = torch.FloatTensor(y_data)

# 모델 초기화
W = torch.zeros((2, 1), requires_grad=True)
b = torch.zeros(1, requires_grad=True)

# optimizer 설정
optimizer = optim.SGD([W, b], lr=1)

nb_epochs = 1000

for epoch in range(nb_epochs+1):
    
    # cost 계산
    hypothesis = torch.sigmoid(x_train.matmul(W)+b)
    cost = F.binary_cross_entropy(hypothesis, y_train)
    
    # weight 갱신
    optimizer.zero_grad()
    cost.backward()
    optimizer.step()
    
    # cost 출력
    if epoch % 100 == 0:
        print('Epoch {:4d}/{} Cost : {:.6f}'.format(
            epoch, nb_epochs, cost.item()
        ))
'''
Epoch    0/1000 Cost : 0.693147
Epoch  100/1000 Cost : 0.134722
Epoch  200/1000 Cost : 0.080643
Epoch  300/1000 Cost : 0.057900
Epoch  400/1000 Cost : 0.045300
Epoch  500/1000 Cost : 0.037261
Epoch  600/1000 Cost : 0.031673
Epoch  700/1000 Cost : 0.027556
Epoch  800/1000 Cost : 0.024394
Epoch  900/1000 Cost : 0.021888
Epoch 1000/1000 Cost : 0.019852
'''

  • Logistic RegressionのEvaluation
    仮定値が0.5以上であればTrueとする.y_trainと比較してcorrect_predictionに格納される.
    • hypothesis = torch.sigmoid(x_train.matmul(W)+b)
print(hypothesis[:5])

prediction = hypothesis >= torch.FloatTensor([0.5])
print(prediction[:5])

correct_prediction = prediction.float() == y_train
print(correct_prediction[:5])

'''
tensor([[2.7648e-04],
        [3.1608e-02],
        [3.8977e-02],
        [9.5622e-01],
        [9.9823e-01]], grad_fn=<SliceBackward>)
tensor([[False],
        [False],
        [False],
        [ True],
        [ True]])
tensor([[True],
        [True],
        [True],
        [True],
        [True]])
'''

    Logistic Regression with nn.Module

  • nn.Module継承使用.
    • class BinaryClassifier(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(2, 1)
        self.sigmoid = nn.Sigmoid()
        
    def forward(self, x):
        return self.sigmoid(self.linear(x))

model = BinaryClassifier()
  • nn.Module with full code 
    • import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

class BinaryClassifier(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(2, 1)
        self.sigmoid = nn.Sigmoid()
        
    def forward(self, x):
        return self.sigmoid(self.linear(x))

model = BinaryClassifier()

optimizer = optim.SGD(model.parameters(), lr=1)

nb_epochs = 100
for epoch in range(nb_epochs+1):
    
    # h(x)
    hypothesis = model(x_train)
    
    # cost
    cost = F.binary_cross_entropy(hypothesis, y_train)
    
    # 갱신
    optimizer.zero_grad()
    cost.backward()
    optimizer.step()
    
    if epoch % 10 == 0:
        prediction = hypothesis >= torch.FloatTensor([0.5])
        correct_prediction = prediction.float() == y_train
        accuracy = correct_prediction.sum().item() / len(correct_prediction)
        print('Epoch : {:4d}/{} Cost : {:.6f} Accuracy {:2.2f}%'.format(
            epoch, nb_epochs, cost.item(), accuracy * 100,
        ))

'''
Epoch :    0/100 Cost : 1.958243 Accuracy 50.00%
Epoch :   10/100 Cost : 0.673601 Accuracy 50.00%
Epoch :   20/100 Cost : 0.501478 Accuracy 66.67%
Epoch :   30/100 Cost : 0.423174 Accuracy 66.67%
Epoch :   40/100 Cost : 0.359201 Accuracy 83.33%
Epoch :   50/100 Cost : 0.304263 Accuracy 83.33%
Epoch :   60/100 Cost : 0.255134 Accuracy 83.33%
Epoch :   70/100 Cost : 0.210761 Accuracy 100.00%
Epoch :   80/100 Cost : 0.174970 Accuracy 100.00%
Epoch :   90/100 Cost : 0.153584 Accuracy 100.00%
Epoch :  100/100 Cost : 0.141670 Accuracy 100.00%
'''
    [セットトップ][Asp.Net]QRCode生成QRコード(続き)-QRコードピクチャの中心にLogoまたは画像を追加
    Asp.Net aspxページをPDFファイルに置き換える