Data Science / Machine Learning / Technology

NUC with eGPU – a Big Little ML Rig

Introduction

eGPU obsessives might like this. We are going to set up an Intel® NUC Kit NUC6i7KYK (Skull Canyon) connected via Thunderbolt 3 (USB Type-C) cable to an eGPU as the AORUS GTX 1080 Gaming Box running Linux (Ubuntu 17.04).  But wait there’s more. Then we are going to install the CUDA and cuDNN libraries to turn our little NUC/eGPU into a home office machine learning Rig. We will show it off with some tensorflow-gpu examples using python and a Jupyter notebook.

IMG_20171011_073909

Reproducing this setup is not for the fainthearted, but the rewards are there later when we demonstrate a subsequent blog using Tensorflow, Keras and RStudio (gpuR).  You probably found this blog because you’ve hit issue trying this yourself, so you know what I mean.  Be prepared to rebuild your machine a few times along the way.

Setup

1. Ubuntu 17.04 Installation

Make an Ubuntu 17.04 bootable USB drive using the iso image at http://releases.ubuntu.com/zesty/ .  Windows user may find the Rufus tool useful for creating this boot drive, https://rufus.akeo.ie/ .

Insert the USB boot drive into your NUC, press F7 to enter into the boot drive settings then select your USB. Follow the installation protocols until you have successfully installed the linux O/S. Once completed, System Preferences > Details will then report something like this.

Screenshot from 2017-10-11 04-43-26

2. NUC BIOS and Driver Update

Update your BIOS using your preferred technique. For the NUC model used here, instructions and media for this can be found at  https://downloadcenter.intel.com/download/27119/NUCs-BIOS-Update-KYSKLi70-86A-?product=89187.  Find the driver update bundle for your NUC and apply that too, e.g. https://downloadcenter.intel.com/product/89187/Intel-NUC-Kit-NUC6i7KYK

3. Change BIOS Settings

Enter into your BIOS configuration tools. For the NUC you would reboot and Press F2. Change the Thunderbolt security setting to LEGACY.

3. Update Kernel

Update your O/S kernel. For v4.13 you would visit http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.13/ and then download the various am64 debs.

$ cd ~/Downloads
$ sudo dpkg -i *.deb

Reboot and then check that the new kernel is active.

$ uname -r
4.13.0-041300-generic

4. Connect eGPU

At this stage, recommend you fully power down your NUC and eGPU, including removing the power cable and then reinserting after a pause. Now first power up your eGPU and insert the Thunderbolt cable from the eGPU to the NUC. Now power up the NUC and verify the NVIDIA device is detected.

Screenshot from 2017-10-11 04-58-56

If the eGPU device has not been detected, try a different Thunderbolt 3 device, e.g. a Google PIXEL, to eliminate the eGPU as being at fault. If it has been detected update the Drivers to use the proprietary NVIDIA, by launching System Settings > Software & Update.

Screenshot from 2017-10-11 05-11-54

Install hardinfo, then launch it to inspect your system device information. Check out your NVIDIA x Server settings too.

$ sudo apt install hardinfo

Screenshot from 2017-10-11 05-24-40

Screenshot from 2017-10-11 05-46-51

5. CUDA Installation

The CUDA installation instructions can be reviewed at http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html .  I suggest using the “3. Package Manager Installation” technique.  Read and follow these instructions closely!  The download chosen here was the local deb file as per https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1704&target_type=deblocal .

Screenshot from 2017-10-11 05-43-52

The apt-key instruction issued was as per below. Install the nvidia-cuda-toolkit while your there:

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1704/x86_64/7fa2af80.pub
$ sudo apt-key add 7fa2af80.pub
$ sudo apt install nvidia-cuda-toolkit
$ nvcc -V
$ gcc --version

Ensure to complete the CUDA “7. Post Installation Actions”. Build (make) and run some of the samples below the NVIDIA-9.0_Sample directory. The deviceQuery tool should return output similar to below. The are a bunch of other samples worth exploring too, e.g. look at 2_Graphics/Mandelbrot, 5_Simulations/smokeParticles and many more.

Screenshot from 2017-10-11 05-50-49

Previous releases can be found at https://developer.nvidia.com/cuda-toolkit-archive , for example a V8 local ubuntu/debian is at https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64-deb .

6. NVIDIA Docker

To set up NVIDIA’s Docker engine, follow the instructions at  https://github.com/NVIDIA/nvidia-docker . You will need to have Docker installed (not shown). Instructions for this can be located at https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/ . For Ubuntu 17.04 I downloaded a few extra packages as per below (using wget not shown) and the installed and verified as follows:

$ docker version
$ sudo dpkg -i insserv_1.14.0-2.1ubuntu2_amd64.deb
$ sudo dpkg -i sysv-rc_2.88dsf-59.3ubuntu2_all.deb
$ sudo dpkg -i nvidia-docker_1.0.1-1_amd64.deb
$ nvidia-docker run --rm nvidia/cuda nvidia-smi

Screenshot from 2017-10-11 06-01-23

7. cuDNN Installation

The instructions for installing the cuDNN neural network deep learning libraries are at http://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html . First, create yourself a Developer account at https://developer.nvidia.com/ and then download the necessary media at https://developer.nvidia.com/rdp/cudnn-download . For cuDNN v7.03 for Ubuntu, the relevant packages would be:

$ sudo dpkg -i libcudnn7_7.0.3.11-1+cuda9.0_amd64.deb
$ sudo dpkg -i libcudnn7-doc_7.0.3.11-1+cuda9.0_amd64.deb
$ sudo dpkg -i libcudnn7-dev_7.0.3.11-1+cuda9.0_amd64.deb

Note that for the cudnn_samples_v7/mnistCUDNN sample to work in the “2.4 Verifying” section of the cuDNN installation docs, you need to edit “$ sudo vi /usr/include/cudnn.h” and change line 63 to  “#include ” . Once you make the sample, run it as follows:

Screenshot from 2017-10-11 07-02-09

While you’re here, also download and install cuDNN V6 as we will need it later for our Tensorflow examples.

$ sudo dpkg -i libcudnn6_6.0.21-1+cuda8.0_amd64.deb
$ sudo dpkg -i libcudnn6-doc_6.0.21-1+cuda8.0_amd64.deb
$ sudo dpkg -i libcudnn6-dev_6.0.21-1+cuda8.0_amd64.deb

Verify

Let’s try a few things out with our new Rig. Clone down some tensorflow tutorial assets and try and few python examples. Before you do free up as much RAM as you can. Inspect “System Monitor” for hints as to which processes to terminate.

1. Python and Tensorflow

First thing to do is set up python, tensorflow and check all is healthy.

$ python --version 
$ python3 --version 
$ pip --version 
$ pip3 --version 
$ pip install tensorflow-gpu 
$ pip3 install tensorflow-gpu 
$ pip show tensorflow-gpu 
$ pip3 show tensorflow-gpu 

$ python3 
>>> import tensorflow as tf 
>>> hello=tf.constant('Hello, TensorFlow!') 
>>> sess = tf.Session() 
>>> print(sess.run(hello)) 
b'Hello TensforFlow!'

2. Tensorflow Samples

Now clone done some tensorflow samples, and try a few, e.g. an image classifier, imagenet.

$ git clone https://github.com/tensorflow/tensorflow.git 
$ git clone https://github.com/tensorflow/models.git
$ cd models\tutorials\image\imagenet
$ python classify_image.py
...
giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca (score = 0.89632)
indri, indris, Indri indri, Indri brevicaudatus (score = 0.00766)
lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens (score = 0.00266)
custard apple (score = 0.00138)
earthstar (score = 0.00104)

As an homage to “Sillicon Valley” epsiode, save this hotdog.jpg to the imagenet directory and try again.

hotdog

$ python classify_image.py --image_file='hotdpg.jpg'
...
hotdog, hot dog, red hot (score = 0.96892)
chiton, coat-of-mail shell, sea cradle, polyplacophore (score = 0.00389)
flatworm, platyhelminth (score = 0.00112)
nematode, nematode worm, roundworm (score = 0.00089)
sea slug, nudibranch (score = 0.00046)

3. Jupyer Notebook

Try using a Jupyter notebook.  Install the software and then launch a Python2 script to verify usage of the tensorflow GPU libraries.

$ pip install jupyter 
$ pip install notebook 
$ pip install --upgrade ipykernel 
$ sudo apt-get install libcupti-dev
$ jupyter notebook

jupyter

Now try something more interesting.  Copy in the Mandelbrot example lifted from https://www.tensorflow.org/tutorials/ and try it using using a Python2 script in your Jupyter notebook.

download

Trivia

NVIDIA Driver Not Loaded

If executing e.g. nvidia-smi FAILS, then launch the NVIDIA X Server Settings application. Check that the NVIDIA driver is selected, rather than the default in-board GPU and then logout/in for the changes to apply.

WiFi Problems

Well all that worked, but now my WiFi service on Ubuntu 17.04 has stopped. If the first command below suggests no firmware has been assigned, do the following.

$ sudo lshw -C network
$ dmesg | grep iwl
$ wget http://mirrors.kernel.org/ubuntu/pool/main/l/linux-firmware/linux-firmware_1.162_all.deb
$ sudo dpkg -i linux-firmware/linux-firmware_1.162_all.deb
$ reboot

GPU Resource Problems

If the samples report errors allocating resources, check available memory and stop unnecessary process.

Installing Older Version of CUDA

Remove the latest current version, download the version required, e.g. 8.0 would appear as follows:

$ sudo apt-get purge cuda-9.0
$ sudo dpkg -i cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb
$ sudo apt-key add /var/cuda-repo-8-0-local-ga2/7fa2af80.pub
$ sudo dpkg -i cuda-repo-ubuntu1604-8-0-local-cublas-performance-update_8.0.61-1_amd64.deb
$ sudo apt-get cuda-8-0

 

 

Advertisements

One thought on “NUC with eGPU – a Big Little ML Rig

  1. Pingback: R with eGPU | emergile

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s