Deep Learning day 7 CIFAR 10のパフォーマンス向上
107197 ワード
CIFAR 10データセットニューラルネットワークを使用したトレーニング
imports
import numpy as np
import tensorflow as tf
from tensorflow.keras.datasets import cifar10
import matplotlib.pyplot as plt
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, Flatten, Dense
from tensorflow.keras.optimizers import SGD, Adam
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Dropout, BatchNormalization, Activation, GlobalAveragePooling2D
from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau, EarlyStoppingデータ・ロード
(train_images, train_labels), (test_images, test_labels) = cifar10.load_data()Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz
170500096/170498071 [==============================] - 6s 0us/step
170508288/170498071 [==============================] - 6s 0us/stepprint(train_images.shape, train_labels.shape)
print(test_images.shape, test_labels.shape)
print(train_images[0].shape)(50000, 32, 32, 3) (50000, 1)
(10000, 32, 32, 3) (10000, 1)
(32, 32, 3)class_names = ["airplane", "automobile", "bird", "cat", "deer", "dog", "frog", "horse", "ship", "truck"]train_labels[:10]array([[6],
[9],
[9],
[4],
[1],
[1],
[2],
[7],
[8],
[3]], dtype=uint8)データの表示
train_labels = np.int32(train_labels.flatten())figure, axes = plt.subplots(figsize=(22, 10), nrows=3, ncols=8)
for i in range(3):
for j in range(8):
axes[i][j].imshow(train_images[(i*8)+j])
axes[i][j].set_title(class_names[train_labels[(i*8)+j]])
データプリプロセッシング
train_images = np.array(train_images/255.0, dtype=np.float32)
train_labels = np.array(train_labels, dtype=np.float32)
test_images = np.array(test_images/255.0, dtype=np.float32)
test_labels = np.array(test_labels, dtype=np.float32)
train_oh_labels = to_categorical(train_labels)
test_oh_labels = to_categorical(test_labels)test_oh_labelsarray([[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 1., 0.],
[0., 0., 0., ..., 0., 1., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 1., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 1., 0., 0.]], dtype=float32)モデルの作成
def build_model():
tf.keras.backend.clear_session()
input_tensor = Input(shape=(32, 32, 3))
x = Conv2D(filters=32, kernel_size=(3, 3), strides=1, padding="same")(input_tensor)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = Conv2D(filters=32, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Conv2D(filters=64, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = Conv2D(filters=64, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Conv2D(filters=128, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = Conv2D(filters=128, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Flatten()(x)
x = Dropout(rate=0.5)(x)
x = Dense(300, activation="relu")(x)
x = Dropout(rate=0.3)(x)
x = Dense(100, activation="relu")(x)
x = Dropout(rate=0.3)(x)
output = Dense(10, activation="softmax")(x)
model = Model(inputs=input_tensor, outputs=output)
return model
model = build_model()モデルのコンパイル
model.compile(optimizer=Adam(0.001), loss="categorical_crossentropy", metrics=["accuracy"])学習モデル
batch_size = 32
history = model.fit(x=train_images, y=train_oh_labels, batch_size = batch_size, epochs=30, validation_split=0.2)Epoch 1/30
1250/1250 [==============================] - 58s 24ms/step - loss: 1.7155 - accuracy: 0.3638 - val_loss: 1.5016 - val_accuracy: 0.4403
Epoch 2/30
1250/1250 [==============================] - 26s 21ms/step - loss: 1.2472 - accuracy: 0.5586 - val_loss: 1.0512 - val_accuracy: 0.6155
Epoch 3/30
1250/1250 [==============================] - 26s 21ms/step - loss: 1.0346 - accuracy: 0.6438 - val_loss: 0.9817 - val_accuracy: 0.6552
Epoch 4/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.9000 - accuracy: 0.6937 - val_loss: 0.9400 - val_accuracy: 0.6809
Epoch 5/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.8184 - accuracy: 0.7207 - val_loss: 0.7343 - val_accuracy: 0.7447
Epoch 6/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.7414 - accuracy: 0.7515 - val_loss: 0.7432 - val_accuracy: 0.7463
Epoch 7/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.6703 - accuracy: 0.7750 - val_loss: 0.8668 - val_accuracy: 0.7133
Epoch 8/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.6305 - accuracy: 0.7915 - val_loss: 0.7891 - val_accuracy: 0.7423
Epoch 9/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.5728 - accuracy: 0.8088 - val_loss: 0.8163 - val_accuracy: 0.7435
Epoch 10/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.5403 - accuracy: 0.8192 - val_loss: 0.6920 - val_accuracy: 0.7665
Epoch 11/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.4981 - accuracy: 0.8346 - val_loss: 0.7544 - val_accuracy: 0.7571
Epoch 12/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.4645 - accuracy: 0.8453 - val_loss: 0.5526 - val_accuracy: 0.8186
Epoch 13/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.4379 - accuracy: 0.8540 - val_loss: 0.7270 - val_accuracy: 0.7592
Epoch 14/30
1250/1250 [==============================] - 26s 20ms/step - loss: 0.4110 - accuracy: 0.8624 - val_loss: 0.6688 - val_accuracy: 0.7872
Epoch 15/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.3858 - accuracy: 0.8714 - val_loss: 0.6721 - val_accuracy: 0.7901
Epoch 16/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.3684 - accuracy: 0.8782 - val_loss: 0.6161 - val_accuracy: 0.8136
Epoch 17/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.3441 - accuracy: 0.8842 - val_loss: 0.6608 - val_accuracy: 0.7985
Epoch 18/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.3275 - accuracy: 0.8897 - val_loss: 0.6437 - val_accuracy: 0.8024
Epoch 19/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.3172 - accuracy: 0.8950 - val_loss: 0.6624 - val_accuracy: 0.8087
Epoch 20/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.2923 - accuracy: 0.9019 - val_loss: 0.5486 - val_accuracy: 0.8332
Epoch 21/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.2845 - accuracy: 0.9068 - val_loss: 0.6702 - val_accuracy: 0.8041
Epoch 22/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.2702 - accuracy: 0.9087 - val_loss: 0.7007 - val_accuracy: 0.8002
Epoch 23/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.2590 - accuracy: 0.9139 - val_loss: 0.7227 - val_accuracy: 0.8109
Epoch 24/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.2468 - accuracy: 0.9169 - val_loss: 0.6479 - val_accuracy: 0.8093
Epoch 25/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.2339 - accuracy: 0.9213 - val_loss: 0.7285 - val_accuracy: 0.8013
Epoch 26/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.2267 - accuracy: 0.9271 - val_loss: 0.6021 - val_accuracy: 0.8256
Epoch 27/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.2175 - accuracy: 0.9273 - val_loss: 0.5780 - val_accuracy: 0.8340
Epoch 28/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.2020 - accuracy: 0.9330 - val_loss: 0.6194 - val_accuracy: 0.8256
Epoch 29/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.1978 - accuracy: 0.9340 - val_loss: 0.6877 - val_accuracy: 0.8298
Epoch 30/30
1250/1250 [==============================] - 26s 21ms/step - loss: 0.1994 - accuracy: 0.9324 - val_loss: 0.6958 - val_accuracy: 0.8185plt.plot(history.history["accuracy"], label="train" )
plt.plot(history.history["val_accuracy"], label="valid" )
plt.legend()<matplotlib.legend.Legend at 0x7fec37cc9250>
テストデータの検証
evaluate_accuracy = model.evaluate(test_images, test_oh_labels, batch_size=batch_size, verbose=1)313/313 [==============================] - 3s 9ms/step - loss: 0.7298 - accuracy: 0.8141Calbackの適用
モデルの作成とコンパイル
model = build_model()
model.summary()Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 32, 32, 3)] 0
_________________________________________________________________
conv2d (Conv2D) (None, 32, 32, 32) 896
_________________________________________________________________
batch_normalization (BatchNo (None, 32, 32, 32) 128
_________________________________________________________________
activation (Activation) (None, 32, 32, 32) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 32, 32, 32) 9248
_________________________________________________________________
batch_normalization_1 (Batch (None, 32, 32, 32) 128
_________________________________________________________________
activation_1 (Activation) (None, 32, 32, 32) 0
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 16, 16, 32) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 16, 16, 64) 18496
_________________________________________________________________
batch_normalization_2 (Batch (None, 16, 16, 64) 256
_________________________________________________________________
activation_2 (Activation) (None, 16, 16, 64) 0
_________________________________________________________________
conv2d_3 (Conv2D) (None, 16, 16, 64) 36928
_________________________________________________________________
batch_normalization_3 (Batch (None, 16, 16, 64) 256
_________________________________________________________________
activation_3 (Activation) (None, 16, 16, 64) 0
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 8, 8, 64) 0
_________________________________________________________________
conv2d_4 (Conv2D) (None, 8, 8, 128) 73856
_________________________________________________________________
batch_normalization_4 (Batch (None, 8, 8, 128) 512
_________________________________________________________________
activation_4 (Activation) (None, 8, 8, 128) 0
_________________________________________________________________
conv2d_5 (Conv2D) (None, 8, 8, 128) 147584
_________________________________________________________________
batch_normalization_5 (Batch (None, 8, 8, 128) 512
_________________________________________________________________
activation_5 (Activation) (None, 8, 8, 128) 0
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 4, 4, 128) 0
_________________________________________________________________
flatten (Flatten) (None, 2048) 0
_________________________________________________________________
dropout (Dropout) (None, 2048) 0
_________________________________________________________________
dense (Dense) (None, 300) 614700
_________________________________________________________________
dropout_1 (Dropout) (None, 300) 0
_________________________________________________________________
dense_1 (Dense) (None, 100) 30100
_________________________________________________________________
dropout_2 (Dropout) (None, 100) 0
_________________________________________________________________
dense_2 (Dense) (None, 10) 1010
=================================================================
Total params: 934,610
Trainable params: 933,714
Non-trainable params: 896
_________________________________________________________________model.compile(optimizer=Adam(0.001), loss="categorical_crossentropy", metrics=["accuracy"])Calback定義
checkpoint_cb = ModelCheckpoint("my_keras_model.h5", save_best_only=True, verbose=1)
early_stopping_cb = EarlyStopping(patience=10, restore_best_weights=True)
reducelr_cb = ReduceLROnPlateau(monitor="val_loss", factor=0.2, patience=5, mode="min", verbose=1)学習モデル
batch_size = 32
history = model.fit(x=train_images, y=train_oh_labels, batch_size = batch_size, epochs=40, validation_split=0.2,
callbacks=[checkpoint_cb, early_stopping_cb, reducelr_cb])Epoch 1/40
1250/1250 [==============================] - 32s 25ms/step - loss: 1.6663 - accuracy: 0.3844 - val_loss: 1.4542 - val_accuracy: 0.4628
Epoch 00001: val_loss improved from inf to 1.45418, saving model to my_keras_model.h5
Epoch 2/40
1250/1250 [==============================] - 28s 22ms/step - loss: 1.2310 - accuracy: 0.5645 - val_loss: 1.1085 - val_accuracy: 0.6126
Epoch 00002: val_loss improved from 1.45418 to 1.10846, saving model to my_keras_model.h5
Epoch 3/40
1250/1250 [==============================] - 28s 22ms/step - loss: 1.0336 - accuracy: 0.6461 - val_loss: 0.9541 - val_accuracy: 0.6673
Epoch 00003: val_loss improved from 1.10846 to 0.95408, saving model to my_keras_model.h5
Epoch 4/40
1250/1250 [==============================] - 27s 21ms/step - loss: 0.8996 - accuracy: 0.6940 - val_loss: 0.7979 - val_accuracy: 0.7329
Epoch 00004: val_loss improved from 0.95408 to 0.79795, saving model to my_keras_model.h5
Epoch 5/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.8033 - accuracy: 0.7307 - val_loss: 1.0418 - val_accuracy: 0.6532
Epoch 00005: val_loss did not improve from 0.79795
Epoch 6/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.7354 - accuracy: 0.7497 - val_loss: 0.7277 - val_accuracy: 0.7503
Epoch 00006: val_loss improved from 0.79795 to 0.72774, saving model to my_keras_model.h5
Epoch 7/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.6741 - accuracy: 0.7749 - val_loss: 0.7393 - val_accuracy: 0.7479
Epoch 00007: val_loss did not improve from 0.72774
Epoch 8/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.6260 - accuracy: 0.7897 - val_loss: 0.6670 - val_accuracy: 0.7748
Epoch 00008: val_loss improved from 0.72774 to 0.66701, saving model to my_keras_model.h5
Epoch 9/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.5721 - accuracy: 0.8102 - val_loss: 0.7466 - val_accuracy: 0.7450
Epoch 00009: val_loss did not improve from 0.66701
Epoch 10/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.5339 - accuracy: 0.8224 - val_loss: 0.7760 - val_accuracy: 0.7438
Epoch 00010: val_loss did not improve from 0.66701
Epoch 11/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.5010 - accuracy: 0.8364 - val_loss: 0.6011 - val_accuracy: 0.8052
Epoch 00011: val_loss improved from 0.66701 to 0.60115, saving model to my_keras_model.h5
Epoch 12/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.4711 - accuracy: 0.8432 - val_loss: 0.5533 - val_accuracy: 0.8161
Epoch 00012: val_loss improved from 0.60115 to 0.55327, saving model to my_keras_model.h5
Epoch 13/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.4342 - accuracy: 0.8553 - val_loss: 0.6798 - val_accuracy: 0.7764
Epoch 00013: val_loss did not improve from 0.55327
Epoch 14/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.4109 - accuracy: 0.8668 - val_loss: 0.5865 - val_accuracy: 0.8066
Epoch 00014: val_loss did not improve from 0.55327
Epoch 15/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.3894 - accuracy: 0.8715 - val_loss: 0.5771 - val_accuracy: 0.8114
Epoch 00015: val_loss did not improve from 0.55327
Epoch 16/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.3592 - accuracy: 0.8819 - val_loss: 0.5703 - val_accuracy: 0.8173
Epoch 00016: val_loss did not improve from 0.55327
Epoch 17/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.3452 - accuracy: 0.8866 - val_loss: 0.5462 - val_accuracy: 0.8268
Epoch 00017: val_loss improved from 0.55327 to 0.54615, saving model to my_keras_model.h5
Epoch 18/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.3275 - accuracy: 0.8942 - val_loss: 0.5313 - val_accuracy: 0.8296
Epoch 00018: val_loss improved from 0.54615 to 0.53128, saving model to my_keras_model.h5
Epoch 19/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.3109 - accuracy: 0.8960 - val_loss: 0.5931 - val_accuracy: 0.8161
Epoch 00019: val_loss did not improve from 0.53128
Epoch 20/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.2976 - accuracy: 0.9035 - val_loss: 0.5827 - val_accuracy: 0.8216
Epoch 00020: val_loss did not improve from 0.53128
Epoch 21/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.2803 - accuracy: 0.9077 - val_loss: 0.6104 - val_accuracy: 0.8157
Epoch 00021: val_loss did not improve from 0.53128
Epoch 22/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.2666 - accuracy: 0.9125 - val_loss: 0.5648 - val_accuracy: 0.8285
Epoch 00022: val_loss did not improve from 0.53128
Epoch 23/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.2601 - accuracy: 0.9144 - val_loss: 0.5800 - val_accuracy: 0.8252
Epoch 00023: val_loss did not improve from 0.53128
Epoch 00023: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026.
Epoch 24/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.1734 - accuracy: 0.9431 - val_loss: 0.5098 - val_accuracy: 0.8551
Epoch 00024: val_loss improved from 0.53128 to 0.50984, saving model to my_keras_model.h5
Epoch 25/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.1389 - accuracy: 0.9549 - val_loss: 0.5479 - val_accuracy: 0.8561
Epoch 00025: val_loss did not improve from 0.50984
Epoch 26/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.1275 - accuracy: 0.9584 - val_loss: 0.5546 - val_accuracy: 0.8568
Epoch 00026: val_loss did not improve from 0.50984
Epoch 27/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.1142 - accuracy: 0.9617 - val_loss: 0.5843 - val_accuracy: 0.8573
Epoch 00027: val_loss did not improve from 0.50984
Epoch 28/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.1070 - accuracy: 0.9639 - val_loss: 0.5649 - val_accuracy: 0.8594
Epoch 00028: val_loss did not improve from 0.50984
Epoch 29/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.0996 - accuracy: 0.9672 - val_loss: 0.5825 - val_accuracy: 0.8590
Epoch 00029: val_loss did not improve from 0.50984
Epoch 00029: ReduceLROnPlateau reducing learning rate to 4.0000001899898055e-05.
Epoch 30/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.0870 - accuracy: 0.9714 - val_loss: 0.5821 - val_accuracy: 0.8635
Epoch 00030: val_loss did not improve from 0.50984
Epoch 31/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.0846 - accuracy: 0.9727 - val_loss: 0.5829 - val_accuracy: 0.8642
Epoch 00031: val_loss did not improve from 0.50984
Epoch 32/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.0776 - accuracy: 0.9741 - val_loss: 0.5964 - val_accuracy: 0.8622
Epoch 00032: val_loss did not improve from 0.50984
Epoch 33/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.0775 - accuracy: 0.9740 - val_loss: 0.6022 - val_accuracy: 0.8632
Epoch 00033: val_loss did not improve from 0.50984
Epoch 34/40
1250/1250 [==============================] - 26s 21ms/step - loss: 0.0756 - accuracy: 0.9747 - val_loss: 0.6016 - val_accuracy: 0.8636
Epoch 00034: val_loss did not improve from 0.50984
Epoch 00034: ReduceLROnPlateau reducing learning rate to 8.000000525498762e-06.plt.plot(history.history["accuracy"], label="train" )
plt.plot(history.history["val_accuracy"], label="valid" )
plt.legend()<matplotlib.legend.Legend at 0x7fec39d2b610>
evaluate_accuracy = model.evaluate(test_images, test_oh_labels, batch_size=batch_size, verbose=1)313/313 [==============================] - 3s 9ms/step - loss: 0.5252 - accuracy: 0.8521フィルタの数、非表示レイヤの数を増やす
モデルの作成
def build_extended_model():
tf.keras.backend.clear_session()
input_tensor = Input(shape=(32, 32, 3))
x = Conv2D(filters=32, kernel_size=(3, 3), strides=1, padding="same")(input_tensor)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = Conv2D(filters=64, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Conv2D(filters=128, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = Conv2D(filters=128, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Conv2D(filters=256, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = Conv2D(filters=256, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
#x = MaxPooling2D(pool_size=(2, 2))(x)
x = Conv2D(filters=512, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Flatten()(x)
x = Dropout(rate=0.5)(x)
x = Dense(300, activation="relu")(x)
x = Dropout(rate=0.3)(x)
x = Dense(100, activation="relu")(x)
x = Dropout(rate=0.3)(x)
output = Dense(10, activation="softmax")(x)
model = Model(inputs=input_tensor, outputs=output)
return model
モデルの作成とコンパイル
model = build_extended_model()
model.summary()Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 32, 32, 3)] 0
_________________________________________________________________
conv2d (Conv2D) (None, 32, 32, 32) 896
_________________________________________________________________
batch_normalization (BatchNo (None, 32, 32, 32) 128
_________________________________________________________________
activation (Activation) (None, 32, 32, 32) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 32, 32, 64) 18496
_________________________________________________________________
batch_normalization_1 (Batch (None, 32, 32, 64) 256
_________________________________________________________________
activation_1 (Activation) (None, 32, 32, 64) 0
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 16, 16, 64) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 16, 16, 128) 73856
_________________________________________________________________
batch_normalization_2 (Batch (None, 16, 16, 128) 512
_________________________________________________________________
activation_2 (Activation) (None, 16, 16, 128) 0
_________________________________________________________________
conv2d_3 (Conv2D) (None, 16, 16, 128) 147584
_________________________________________________________________
batch_normalization_3 (Batch (None, 16, 16, 128) 512
_________________________________________________________________
activation_3 (Activation) (None, 16, 16, 128) 0
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 8, 8, 128) 0
_________________________________________________________________
conv2d_4 (Conv2D) (None, 8, 8, 256) 295168
_________________________________________________________________
batch_normalization_4 (Batch (None, 8, 8, 256) 1024
_________________________________________________________________
activation_4 (Activation) (None, 8, 8, 256) 0
_________________________________________________________________
conv2d_5 (Conv2D) (None, 8, 8, 256) 590080
_________________________________________________________________
batch_normalization_5 (Batch (None, 8, 8, 256) 1024
_________________________________________________________________
activation_5 (Activation) (None, 8, 8, 256) 0
_________________________________________________________________
conv2d_6 (Conv2D) (None, 8, 8, 512) 1180160
_________________________________________________________________
batch_normalization_6 (Batch (None, 8, 8, 512) 2048
_________________________________________________________________
activation_6 (Activation) (None, 8, 8, 512) 0
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 4, 4, 512) 0
_________________________________________________________________
flatten (Flatten) (None, 8192) 0
_________________________________________________________________
dropout (Dropout) (None, 8192) 0
_________________________________________________________________
dense (Dense) (None, 300) 2457900
_________________________________________________________________
dropout_1 (Dropout) (None, 300) 0
_________________________________________________________________
dense_1 (Dense) (None, 100) 30100
_________________________________________________________________
dropout_2 (Dropout) (None, 100) 0
_________________________________________________________________
dense_2 (Dense) (None, 10) 1010
=================================================================
Total params: 4,800,754
Trainable params: 4,798,002
Non-trainable params: 2,752
_________________________________________________________________model.compile(optimizer=Adam(0.001), loss="categorical_crossentropy", metrics=["accuracy"])Calback定義
checkpoint_cb = ModelCheckpoint("my_keras_model.h5", save_best_only=True, verbose=1)
early_stopping_cb = EarlyStopping(patience=10, restore_best_weights=True)
reducelr_cb = ReduceLROnPlateau(monitor="val_loss", factor=0.2, patience=5, mode="min", verbose=1)
学習モデル
batch_size = 32
history = model.fit(x=train_images, y=train_oh_labels, batch_size = batch_size, epochs=40, validation_split=0.2,
callbacks=[checkpoint_cb, early_stopping_cb, reducelr_cb])Epoch 1/40
1250/1250 [==============================] - 45s 34ms/step - loss: 1.9010 - accuracy: 0.2892 - val_loss: 1.8232 - val_accuracy: 0.3345
Epoch 00001: val_loss improved from inf to 1.82324, saving model to my_keras_model.h5
Epoch 2/40
1250/1250 [==============================] - 44s 36ms/step - loss: 1.4290 - accuracy: 0.4776 - val_loss: 1.3944 - val_accuracy: 0.4879
Epoch 00002: val_loss improved from 1.82324 to 1.39438, saving model to my_keras_model.h5
Epoch 3/40
1250/1250 [==============================] - 42s 34ms/step - loss: 1.1695 - accuracy: 0.5913 - val_loss: 1.1177 - val_accuracy: 0.6297
Epoch 00003: val_loss improved from 1.39438 to 1.11772, saving model to my_keras_model.h5
Epoch 4/40
1250/1250 [==============================] - 44s 36ms/step - loss: 0.9969 - accuracy: 0.6601 - val_loss: 1.0691 - val_accuracy: 0.6441
Epoch 00004: val_loss improved from 1.11772 to 1.06911, saving model to my_keras_model.h5
Epoch 5/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.8816 - accuracy: 0.7061 - val_loss: 0.8764 - val_accuracy: 0.7024
Epoch 00005: val_loss improved from 1.06911 to 0.87643, saving model to my_keras_model.h5
Epoch 6/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.7859 - accuracy: 0.7386 - val_loss: 0.8396 - val_accuracy: 0.7186
Epoch 00006: val_loss improved from 0.87643 to 0.83959, saving model to my_keras_model.h5
Epoch 7/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.7091 - accuracy: 0.7648 - val_loss: 0.7364 - val_accuracy: 0.7564
Epoch 00007: val_loss improved from 0.83959 to 0.73639, saving model to my_keras_model.h5
Epoch 8/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.6374 - accuracy: 0.7911 - val_loss: 0.7749 - val_accuracy: 0.7428
Epoch 00008: val_loss did not improve from 0.73639
Epoch 9/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.5746 - accuracy: 0.8111 - val_loss: 0.6017 - val_accuracy: 0.7990
Epoch 00009: val_loss improved from 0.73639 to 0.60167, saving model to my_keras_model.h5
Epoch 10/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.5181 - accuracy: 0.8325 - val_loss: 0.7209 - val_accuracy: 0.7607
Epoch 00010: val_loss did not improve from 0.60167
Epoch 11/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.4633 - accuracy: 0.8485 - val_loss: 0.5826 - val_accuracy: 0.8070
Epoch 00011: val_loss improved from 0.60167 to 0.58264, saving model to my_keras_model.h5
Epoch 12/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.4183 - accuracy: 0.8652 - val_loss: 0.6030 - val_accuracy: 0.8050
Epoch 00012: val_loss did not improve from 0.58264
Epoch 13/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.3746 - accuracy: 0.8813 - val_loss: 0.5998 - val_accuracy: 0.8124
Epoch 00013: val_loss did not improve from 0.58264
Epoch 14/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.3308 - accuracy: 0.8943 - val_loss: 0.6030 - val_accuracy: 0.8033
Epoch 00014: val_loss did not improve from 0.58264
Epoch 15/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.3115 - accuracy: 0.9032 - val_loss: 0.6408 - val_accuracy: 0.8196
Epoch 00015: val_loss did not improve from 0.58264
Epoch 16/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.2732 - accuracy: 0.9134 - val_loss: 0.5918 - val_accuracy: 0.8176
Epoch 00016: val_loss did not improve from 0.58264
Epoch 00016: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026.
Epoch 17/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.1587 - accuracy: 0.9490 - val_loss: 0.5035 - val_accuracy: 0.8632
Epoch 00017: val_loss improved from 0.58264 to 0.50347, saving model to my_keras_model.h5
Epoch 18/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.1165 - accuracy: 0.9622 - val_loss: 0.5649 - val_accuracy: 0.8637
Epoch 00018: val_loss did not improve from 0.50347
Epoch 19/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.0993 - accuracy: 0.9679 - val_loss: 0.5565 - val_accuracy: 0.8660
Epoch 00019: val_loss did not improve from 0.50347
Epoch 20/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.0882 - accuracy: 0.9722 - val_loss: 0.6048 - val_accuracy: 0.8638
Epoch 00020: val_loss did not improve from 0.50347
Epoch 21/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.0767 - accuracy: 0.9751 - val_loss: 0.6195 - val_accuracy: 0.8672
Epoch 00021: val_loss did not improve from 0.50347
Epoch 22/40
1250/1250 [==============================] - 43s 34ms/step - loss: 0.0661 - accuracy: 0.9787 - val_loss: 0.6542 - val_accuracy: 0.8675
Epoch 00022: val_loss did not improve from 0.50347
Epoch 00022: ReduceLROnPlateau reducing learning rate to 4.0000001899898055e-05.
Epoch 23/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.0511 - accuracy: 0.9844 - val_loss: 0.6510 - val_accuracy: 0.8714
Epoch 00023: val_loss did not improve from 0.50347
Epoch 24/40
1250/1250 [==============================] - 45s 36ms/step - loss: 0.0468 - accuracy: 0.9853 - val_loss: 0.6816 - val_accuracy: 0.8694
Epoch 00024: val_loss did not improve from 0.50347
Epoch 25/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.0391 - accuracy: 0.9881 - val_loss: 0.7098 - val_accuracy: 0.8710
Epoch 00025: val_loss did not improve from 0.50347
Epoch 26/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.0374 - accuracy: 0.9881 - val_loss: 0.7221 - val_accuracy: 0.8720
Epoch 00026: val_loss did not improve from 0.50347
Epoch 27/40
1250/1250 [==============================] - 42s 34ms/step - loss: 0.0356 - accuracy: 0.9891 - val_loss: 0.7484 - val_accuracy: 0.8700
Epoch 00027: val_loss did not improve from 0.50347
Epoch 00027: ReduceLROnPlateau reducing learning rate to 8.000000525498762e-06.plt.plot(history.history["accuracy"], label="train" )
plt.plot(history.history["val_accuracy"], label="valid" )
plt.legend()<matplotlib.legend.Legend at 0x7fec973bb390>
#### 테스트 데이터에서 평가evaluate_accuracy = model.evaluate(test_images, test_oh_labels, batch_size=batch_size, verbose=1)313/313 [==============================] - 4s 11ms/step - loss: 0.5149 - accuracy: 0.8612グローバル使用可能な電源冷却の適用
def build_extended_gap_model():
tf.keras.backend.clear_session()
input_tensor = Input(shape=(32, 32, 3))
x = Conv2D(filters=32, kernel_size=(3, 3), strides=1, padding="same")(input_tensor)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = Conv2D(filters=64, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Conv2D(filters=128, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = Conv2D(filters=128, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Conv2D(filters=256, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = Conv2D(filters=256, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
#x = MaxPooling2D(pool_size=(2, 2))(x)
x = Conv2D(filters=512, kernel_size=(3, 3), strides=1, padding="same")(x)
x = BatchNormalization()(x)
x = Activation("relu")(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
# x = Flatten()(x)
x = GlobalAveragePooling2D()(x)
x = Dropout(rate=0.5)(x)
x = Dense(300, activation="relu")(x)
x = Dropout(rate=0.3)(x)
x = Dense(100, activation="relu")(x)
x = Dropout(rate=0.3)(x)
output = Dense(10, activation="softmax")(x)
model = Model(inputs=input_tensor, outputs=output)
return model
モデルの作成とコンパイル
model = build_extended_gap_model()
model.summary()Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 32, 32, 3)] 0
_________________________________________________________________
conv2d (Conv2D) (None, 32, 32, 32) 896
_________________________________________________________________
batch_normalization (BatchNo (None, 32, 32, 32) 128
_________________________________________________________________
activation (Activation) (None, 32, 32, 32) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 32, 32, 64) 18496
_________________________________________________________________
batch_normalization_1 (Batch (None, 32, 32, 64) 256
_________________________________________________________________
activation_1 (Activation) (None, 32, 32, 64) 0
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 16, 16, 64) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 16, 16, 128) 73856
_________________________________________________________________
batch_normalization_2 (Batch (None, 16, 16, 128) 512
_________________________________________________________________
activation_2 (Activation) (None, 16, 16, 128) 0
_________________________________________________________________
conv2d_3 (Conv2D) (None, 16, 16, 128) 147584
_________________________________________________________________
batch_normalization_3 (Batch (None, 16, 16, 128) 512
_________________________________________________________________
activation_3 (Activation) (None, 16, 16, 128) 0
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 8, 8, 128) 0
_________________________________________________________________
conv2d_4 (Conv2D) (None, 8, 8, 256) 295168
_________________________________________________________________
batch_normalization_4 (Batch (None, 8, 8, 256) 1024
_________________________________________________________________
activation_4 (Activation) (None, 8, 8, 256) 0
_________________________________________________________________
conv2d_5 (Conv2D) (None, 8, 8, 256) 590080
_________________________________________________________________
batch_normalization_5 (Batch (None, 8, 8, 256) 1024
_________________________________________________________________
activation_5 (Activation) (None, 8, 8, 256) 0
_________________________________________________________________
conv2d_6 (Conv2D) (None, 8, 8, 512) 1180160
_________________________________________________________________
batch_normalization_6 (Batch (None, 8, 8, 512) 2048
_________________________________________________________________
activation_6 (Activation) (None, 8, 8, 512) 0
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 4, 4, 512) 0
_________________________________________________________________
global_average_pooling2d (Gl (None, 512) 0
_________________________________________________________________
dropout (Dropout) (None, 512) 0
_________________________________________________________________
dense (Dense) (None, 300) 153900
_________________________________________________________________
dropout_1 (Dropout) (None, 300) 0
_________________________________________________________________
dense_1 (Dense) (None, 100) 30100
_________________________________________________________________
dropout_2 (Dropout) (None, 100) 0
_________________________________________________________________
dense_2 (Dense) (None, 10) 1010
=================================================================
Total params: 2,496,754
Trainable params: 2,494,002
Non-trainable params: 2,752
_________________________________________________________________model.compile(optimizer=Adam(0.001), loss="categorical_crossentropy", metrics=["accuracy"])Calback定義
checkpoint_cb = ModelCheckpoint("my_keras_model.h5", save_best_only=True, verbose=1)
early_stopping_cb = EarlyStopping(patience=10, restore_best_weights=True)
reducelr_cb = ReduceLROnPlateau(monitor="val_loss", factor=0.2, patience=5, mode="min", verbose=1)
学習モデル
batch_size = 32
history = model.fit(x=train_images, y=train_oh_labels, batch_size = batch_size, epochs=40, validation_split=0.2,
callbacks=[checkpoint_cb, early_stopping_cb, reducelr_cb])Epoch 1/40
1250/1250 [==============================] - 43s 33ms/step - loss: 1.6606 - accuracy: 0.3770 - val_loss: 2.4914 - val_accuracy: 0.2898
Epoch 00001: val_loss improved from inf to 2.49137, saving model to my_keras_model.h5
Epoch 2/40
1250/1250 [==============================] - 41s 33ms/step - loss: 1.2225 - accuracy: 0.5664 - val_loss: 1.2138 - val_accuracy: 0.5749
Epoch 00002: val_loss improved from 2.49137 to 1.21383, saving model to my_keras_model.h5
Epoch 3/40
1250/1250 [==============================] - 41s 33ms/step - loss: 0.9928 - accuracy: 0.6583 - val_loss: 0.9320 - val_accuracy: 0.6799
Epoch 00003: val_loss improved from 1.21383 to 0.93197, saving model to my_keras_model.h5
Epoch 4/40
1250/1250 [==============================] - 41s 33ms/step - loss: 0.8404 - accuracy: 0.7151 - val_loss: 1.1202 - val_accuracy: 0.6427
Epoch 00004: val_loss did not improve from 0.93197
Epoch 5/40
1250/1250 [==============================] - 41s 33ms/step - loss: 0.7286 - accuracy: 0.7586 - val_loss: 0.9172 - val_accuracy: 0.7016
Epoch 00005: val_loss improved from 0.93197 to 0.91718, saving model to my_keras_model.h5
Epoch 6/40
1250/1250 [==============================] - 41s 33ms/step - loss: 0.6403 - accuracy: 0.7894 - val_loss: 0.8578 - val_accuracy: 0.7237
Epoch 00006: val_loss improved from 0.91718 to 0.85783, saving model to my_keras_model.h5
Epoch 7/40
805/1250 [==================>...........] - ETA: 13s - loss: 0.5560 - accuracy: 0.8163plt.plot(history.history["accuracy"], label="train" )
plt.plot(history.history["val_accuracy"], label="valid" )
plt.legend()<matplotlib.legend.Legend at 0x7fec973bb390>#### 테스트 데이터에서 평가evaluate_accuracy = model.evaluate(test_images, test_oh_labels, batch_size=batch_size, verbose=1)313/313 [==============================] - 4s 11ms/step - loss: 0.5149 - accuracy: 0.8612Reference
この問題について(Deep Learning day 7 CIFAR 10のパフォーマンス向上), 我々は、より多くの情報をここで見つけました https://velog.io/@bbkyoo/Deep-Learning-day-7-CIFAR10-성능올리기テキストは自由に共有またはコピーできます。ただし、このドキュメントのURLは参考URLとして残しておいてください。
Collection and Share based on the CC Protocol