anacondaは使わない。AWSGPU+Ubuntu14.04+jupyter+theano+chainer+OpenCV3.1.0+cuDNNな環境構築
環境
AWSのGPUインスタンスを用いる
・インスタンスタイプ: g2.2xlarge
・AMI ID: ubuntu/images/hvm-ssd/ubuntu-trusty-14.04-amd64-server-20160627 (ami-2d39803a)
ベースインストール
$ sudo apt-get update
$ sudo apt-get install build-essential cmake pkg-config
$ sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
$ sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
$ sudo apt-get install libxvidcore-dev libx264-dev
$ sudo apt-get install libgtk2.0-dev
$ sudo apt-get install libatlas-base-dev gfortran
$ sudo apt-get -y install cmake git libgtk2.0-dev ocl-icd-opencl-dev
$ python --version
Python 2.7.6
$ sudo apt-get install python-dev python-pip
$ sudo apt-get install unzip
$ pip install --upgrade pip --user
$ pip install ipython pyzmq tornado --user
$ pip install jsonschema --user
$ pip install numpy matplotlib scipy --user
cudaインストール
$ wget http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
$ sudo dpkg -i cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
$ sudo apt-get update
$ sudo apt-get install cuda
$ echo 'export PATH=/usr/local/cuda-7.5/bin:$PATH' >> ~/.bashrc
$ echo 'export LD_LIBRARY_PATH=/usr/local/cuda-7.5/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
$ source .bashrc
$ sudo init 6
$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 367.57 Mon Oct 3 20:37:01 PDT 2016
GCC version: gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
$ dpkg -l | grep nvidia
ii nvidia-352 367.57-0ubuntu0.14.04.1 amd64 Transitional package for nvidia-367
ii nvidia-352-dev 367.57-0ubuntu0.14.04.1 amd64 Transitional package for nvidia-367-dev
ii nvidia-352-uvm 352.39-0ubuntu1 amd64 Transitional package for nvidia-352
ii nvidia-367 367.57-0ubuntu0.14.04.1 amd64 NVIDIA binary driver - version 367.57
ii nvidia-367-dev 367.57-0ubuntu0.14.04.1 amd64 NVIDIA binary Xorg driver development files
ii nvidia-modprobe 352.39-0ubuntu1 amd64 Load the NVIDIA kernel driver and create device files
ii nvidia-opencl-icd-352 367.57-0ubuntu0.14.04.1 amd64 Transitional package for nvidia-opencl-icd-367
ii nvidia-opencl-icd-367 367.57-0ubuntu0.14.04.1 amd64 NVIDIA OpenCL ICD
ii nvidia-prime 0.6.2 amd64 Tools to enable NVIDIA's Prime
ii nvidia-settings 352.39-0ubuntu1 amd64 Tool for configuring the NVIDIA graphics driver
cudnnインストール
-
cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
の入手方法
https://developer.nvidia.com/cudnn
から登録して、Downloadボタンを押す。
Download cuDNN v4 (Feb 10, 2016), for CUDA 7.0 and later.
を選択、さらに cuDNN v4 Library for Linux
を選択してダウンロードしてscpで /home/ubuntu
にファイル転送しておく。
$ cd ~
$ tar -xvzf cudnn-7.0-linux-x64-v4.0-prod.tgz
libcudnn.so libcudnn.so.4 libcudnn.so.4.0.7 libcudnn_static.aが入ってるディレクトリの絶対パスをLD_LIBRARY_PATHに追加
$ cd cuda/lib64
$ ls
libcudnn.so libcudnn.so.4 libcudnn.so.4.0.7 libcudnn_static.a
$ pwd
/home/ubuntu/cuda/lib64
$ LD_LIBRARY_PATH=/home/ubuntu/cuda/lib64; export LD_LIBRARY_PATH
$ cd ~
$ sudo cp cuda/include/cudnn.h /usr/local/cuda/include
$ sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
$ sudo chmod a+r /usr/local/cuda/lib64/libcudnn*
$ export PATH=/usr/local/cuda/bin:$PATH
$ export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
$ source .bashrc
Jupyterインストール
$ sudo apt-get remove --purge ipython
$ sudo apt-get install libjpeg tk-dev
$ pip install jupyter --user
$ pip install wcwidth --user
$ ~/.local/bin/jupyter notebook --generate-config
$ vim ~/.jupyter/jupyter_notebook_config.py
$ python
>>> from notebook.auth import passwd
>>> passwd()
Enter password: 【なんらかのパスワードを入力】
Verify password: 【なんらかのパスワードを入力】
'sha1:84acf9ab4e38:5498f3245ea50214fd5bdd3aaa2050b33d5c5f91'
>>> exit()
$ vi ~/.jupyter/jupyter_notebook_config.py
+c.NotebookApp.ip ='*'
+c.NotebookApp.port = 8888
+c.NotebookApp.password = u'sha1:84acf9ab4e38:5498f3245ea50214fd5bdd3aaa2050b33d5c5f91'
$ echo 'export PATH=$PATH:~/.local/bin' >> ~/.bashrc
$ source ~/.bashrc
cudaパス関連付け
$ echo 'export PATH=/usr/local/cuda-7.5/bin:$PATH' >> ~/.bashrc
$ echo 'export LD_LIBRARY_PATH=/usr/local/cuda-7.5/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
$ source .bashrc
theano & chainerインストール
$ sudo apt-get update
$ sudo apt-get install build-essential cmake pkg-config
$ sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
$ sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
$ sudo apt-get install libxvidcore-dev libx264-dev
$ sudo apt-get install libgtk2.0-dev
$ sudo apt-get install libatlas-base-dev gfortran
$ sudo apt-get -y install cmake git libgtk2.0-dev ocl-icd-opencl-dev
$ python --version
Python 2.7.6
$ sudo apt-get install python-dev python-pip
$ sudo apt-get install unzip
$ pip install --upgrade pip --user
$ pip install ipython pyzmq tornado --user
$ pip install jsonschema --user
$ pip install numpy matplotlib scipy --user
$ wget http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
$ sudo dpkg -i cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
$ sudo apt-get update
$ sudo apt-get install cuda
$ echo 'export PATH=/usr/local/cuda-7.5/bin:$PATH' >> ~/.bashrc
$ echo 'export LD_LIBRARY_PATH=/usr/local/cuda-7.5/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
$ source .bashrc
$ sudo init 6
$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 367.57 Mon Oct 3 20:37:01 PDT 2016
GCC version: gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
$ dpkg -l | grep nvidia
ii nvidia-352 367.57-0ubuntu0.14.04.1 amd64 Transitional package for nvidia-367
ii nvidia-352-dev 367.57-0ubuntu0.14.04.1 amd64 Transitional package for nvidia-367-dev
ii nvidia-352-uvm 352.39-0ubuntu1 amd64 Transitional package for nvidia-352
ii nvidia-367 367.57-0ubuntu0.14.04.1 amd64 NVIDIA binary driver - version 367.57
ii nvidia-367-dev 367.57-0ubuntu0.14.04.1 amd64 NVIDIA binary Xorg driver development files
ii nvidia-modprobe 352.39-0ubuntu1 amd64 Load the NVIDIA kernel driver and create device files
ii nvidia-opencl-icd-352 367.57-0ubuntu0.14.04.1 amd64 Transitional package for nvidia-opencl-icd-367
ii nvidia-opencl-icd-367 367.57-0ubuntu0.14.04.1 amd64 NVIDIA OpenCL ICD
ii nvidia-prime 0.6.2 amd64 Tools to enable NVIDIA's Prime
ii nvidia-settings 352.39-0ubuntu1 amd64 Tool for configuring the NVIDIA graphics driver
cudnnインストール
-
cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
の入手方法
https://developer.nvidia.com/cudnn
から登録して、Downloadボタンを押す。
Download cuDNN v4 (Feb 10, 2016), for CUDA 7.0 and later.
を選択、さらに cuDNN v4 Library for Linux
を選択してダウンロードしてscpで /home/ubuntu
にファイル転送しておく。
$ cd ~
$ tar -xvzf cudnn-7.0-linux-x64-v4.0-prod.tgz
libcudnn.so libcudnn.so.4 libcudnn.so.4.0.7 libcudnn_static.aが入ってるディレクトリの絶対パスをLD_LIBRARY_PATHに追加
$ cd cuda/lib64
$ ls
libcudnn.so libcudnn.so.4 libcudnn.so.4.0.7 libcudnn_static.a
$ pwd
/home/ubuntu/cuda/lib64
$ LD_LIBRARY_PATH=/home/ubuntu/cuda/lib64; export LD_LIBRARY_PATH
$ cd ~
$ sudo cp cuda/include/cudnn.h /usr/local/cuda/include
$ sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
$ sudo chmod a+r /usr/local/cuda/lib64/libcudnn*
$ export PATH=/usr/local/cuda/bin:$PATH
$ export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
$ source .bashrc
Jupyterインストール
$ sudo apt-get remove --purge ipython
$ sudo apt-get install libjpeg tk-dev
$ pip install jupyter --user
$ pip install wcwidth --user
$ ~/.local/bin/jupyter notebook --generate-config
$ vim ~/.jupyter/jupyter_notebook_config.py
$ python
>>> from notebook.auth import passwd
>>> passwd()
Enter password: 【なんらかのパスワードを入力】
Verify password: 【なんらかのパスワードを入力】
'sha1:84acf9ab4e38:5498f3245ea50214fd5bdd3aaa2050b33d5c5f91'
>>> exit()
$ vi ~/.jupyter/jupyter_notebook_config.py
+c.NotebookApp.ip ='*'
+c.NotebookApp.port = 8888
+c.NotebookApp.password = u'sha1:84acf9ab4e38:5498f3245ea50214fd5bdd3aaa2050b33d5c5f91'
$ echo 'export PATH=$PATH:~/.local/bin' >> ~/.bashrc
$ source ~/.bashrc
cudaパス関連付け
$ echo 'export PATH=/usr/local/cuda-7.5/bin:$PATH' >> ~/.bashrc
$ echo 'export LD_LIBRARY_PATH=/usr/local/cuda-7.5/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
$ source .bashrc
theano & chainerインストール
cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
の入手方法https://developer.nvidia.com/cudnn から登録して、Downloadボタンを押す。
Download cuDNN v4 (Feb 10, 2016), for CUDA 7.0 and later.
を選択、さらに cuDNN v4 Library for Linux
を選択してダウンロードしてscpで /home/ubuntu
にファイル転送しておく。$ cd ~
$ tar -xvzf cudnn-7.0-linux-x64-v4.0-prod.tgz
libcudnn.so libcudnn.so.4 libcudnn.so.4.0.7 libcudnn_static.aが入ってるディレクトリの絶対パスをLD_LIBRARY_PATHに追加
$ cd cuda/lib64
$ ls
libcudnn.so libcudnn.so.4 libcudnn.so.4.0.7 libcudnn_static.a
$ pwd
/home/ubuntu/cuda/lib64
$ LD_LIBRARY_PATH=/home/ubuntu/cuda/lib64; export LD_LIBRARY_PATH
$ cd ~
$ sudo cp cuda/include/cudnn.h /usr/local/cuda/include
$ sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
$ sudo chmod a+r /usr/local/cuda/lib64/libcudnn*
$ export PATH=/usr/local/cuda/bin:$PATH
$ export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
$ source .bashrc
$ sudo apt-get remove --purge ipython
$ sudo apt-get install libjpeg tk-dev
$ pip install jupyter --user
$ pip install wcwidth --user
$ ~/.local/bin/jupyter notebook --generate-config
$ vim ~/.jupyter/jupyter_notebook_config.py
$ python
>>> from notebook.auth import passwd
>>> passwd()
Enter password: 【なんらかのパスワードを入力】
Verify password: 【なんらかのパスワードを入力】
'sha1:84acf9ab4e38:5498f3245ea50214fd5bdd3aaa2050b33d5c5f91'
>>> exit()
$ vi ~/.jupyter/jupyter_notebook_config.py
+c.NotebookApp.ip ='*'
+c.NotebookApp.port = 8888
+c.NotebookApp.password = u'sha1:84acf9ab4e38:5498f3245ea50214fd5bdd3aaa2050b33d5c5f91'
$ echo 'export PATH=$PATH:~/.local/bin' >> ~/.bashrc
$ source ~/.bashrc
cudaパス関連付け
$ echo 'export PATH=/usr/local/cuda-7.5/bin:$PATH' >> ~/.bashrc
$ echo 'export LD_LIBRARY_PATH=/usr/local/cuda-7.5/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
$ source .bashrc
theano & chainerインストール
$ echo 'export PATH=/usr/local/cuda-7.5/bin:$PATH' >> ~/.bashrc
$ echo 'export LD_LIBRARY_PATH=/usr/local/cuda-7.5/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
$ source .bashrc
※ chainerのインストールをcudnnのインストールより先にやるとGPU処理が正常に通らなくなるので注意
$ pip install theano --user
$ pip install chainer --user
opencv3.1.0インストール
$ wget --no-check-certificate https://github.com/Itseez/opencv/archive/3.1.0.zip -O opencv-3.1.0.zip
$ unzip opencv-3.1.0.zip
$ cd opencv-3.1.0
$ git clone --depth 1 https://github.com/Itseez/opencv_contrib.git opencv_contrib
$ cd opencv_contrib
$ git fetch origin --tags --depth 1
$ git checkout 3.1.0
$ sudo apt-get -y -qq install cmake git libgtk2.0-dev ocl-icd-opencl-dev qt5-default
$ sudo apt-get install libhdf5-dev
$ pip install h5py --user
$ cd ..
$ mkdir build
$ cd build
$ cmake -D CMAKE_BUILD_TYPE=RelWithDebugInfo -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D OPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D WITH_CUDA=ON -D ENABLE_PRECOMPILED_HEADERS=OFF -D CUDA_ARCH_BIN="3.0" -D CUDA_ARCH_PTX="" -D WITH_OPENGL=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_OPENGL=ON -D ENABLE_FAST_MATH=1 -D CUDA_FAST_MATH=1 ..
$ make -j 4
$ sudo make install -j 4
$ sudo ldconfig
$ echo 'export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
$ source ~/.bashrc
$ pkg-config --modversion opencv
3.1.0
OpenCVテスト
$ sudo ln /dev/null /dev/raw1394
$ cd ~
$ mkdir test
$ cd test
$ wget http://www.sonicjapan.co.jp/sample/download/AVI-352x240-2sec.AVI
$ python
>>> import numpy as np
>>> import cv2
>>> cap = cv2.VideoCapture('./AVI-352x240-2sec.AVI')
>>> cap.read()
jupyter notebookでの動画読込確認
$ cd ~
$ jupyter notebook
先ほどのtestディレクトリへ移動してnotebookを作り、下記実行
import numpy as np
import cv2
cap = cv2.VideoCapture('./AVI-352x240-2sec.AVI')
cap.read()
$ wget --no-check-certificate https://github.com/Itseez/opencv/archive/3.1.0.zip -O opencv-3.1.0.zip
$ unzip opencv-3.1.0.zip
$ cd opencv-3.1.0
$ git clone --depth 1 https://github.com/Itseez/opencv_contrib.git opencv_contrib
$ cd opencv_contrib
$ git fetch origin --tags --depth 1
$ git checkout 3.1.0
$ sudo apt-get -y -qq install cmake git libgtk2.0-dev ocl-icd-opencl-dev qt5-default
$ sudo apt-get install libhdf5-dev
$ pip install h5py --user
$ cd ..
$ mkdir build
$ cd build
$ cmake -D CMAKE_BUILD_TYPE=RelWithDebugInfo -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D OPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D WITH_CUDA=ON -D ENABLE_PRECOMPILED_HEADERS=OFF -D CUDA_ARCH_BIN="3.0" -D CUDA_ARCH_PTX="" -D WITH_OPENGL=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_OPENGL=ON -D ENABLE_FAST_MATH=1 -D CUDA_FAST_MATH=1 ..
$ make -j 4
$ sudo make install -j 4
$ sudo ldconfig
$ echo 'export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
$ source ~/.bashrc
$ pkg-config --modversion opencv
3.1.0
$ sudo ln /dev/null /dev/raw1394
$ cd ~
$ mkdir test
$ cd test
$ wget http://www.sonicjapan.co.jp/sample/download/AVI-352x240-2sec.AVI
$ python
>>> import numpy as np
>>> import cv2
>>> cap = cv2.VideoCapture('./AVI-352x240-2sec.AVI')
>>> cap.read()
jupyter notebookでの動画読込確認
$ cd ~
$ jupyter notebook
先ほどのtestディレクトリへ移動してnotebookを作り、下記実行
import numpy as np
import cv2
cap = cv2.VideoCapture('./AVI-352x240-2sec.AVI')
cap.read()
$ cd ~
$ jupyter notebook
先ほどのtestディレクトリへ移動してnotebookを作り、下記実行
import numpy as np
import cv2
cap = cv2.VideoCapture('./AVI-352x240-2sec.AVI')
cap.read()
theanoのGPU処理テスト
$ cd ~
$ git clone https://github.com/aidiary/deep-learning-theano
$ cd ~/deep-learning-theano/scripts
$ THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python convnet.py
Using gpu device 0: GRID K520
....
Optimization complete.
Best validation score of 0.910000 % obtained at iteration 16400, with test performance 0.930000 %
The code for file convnet.py ran for 39.85m
chainerのGPU処理テスト
$ git clone https://github.com/pfnet/chainer.git
$ python ~/chainer/examples/mnist/train_mnist.py -g 0
$ cd ~
$ git clone https://github.com/aidiary/deep-learning-theano
$ cd ~/deep-learning-theano/scripts
$ THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python convnet.py
Using gpu device 0: GRID K520
....
Optimization complete.
Best validation score of 0.910000 % obtained at iteration 16400, with test performance 0.930000 %
The code for file convnet.py ran for 39.85m
$ git clone https://github.com/pfnet/chainer.git
$ python ~/chainer/examples/mnist/train_mnist.py -g 0
Author And Source
この問題について(anacondaは使わない。AWSGPU+Ubuntu14.04+jupyter+theano+chainer+OpenCV3.1.0+cuDNNな環境構築), 我々は、より多くの情報をここで見つけました https://qiita.com/algopia/items/c1c740c98d8810f7167d著者帰属:元の著者の情報は、元のURLに含まれています。著作権は原作者に属する。
Content is automatically searched and collected through network algorithms . If there is a violation . Please contact us . We will adjust (correct author information ,or delete content ) as soon as possible .