Intel neural compute stick 2 and open source openvino toolkit

ncappzoo Repository Branches

There are main three branches in the repository; their descriptions are below. The master branch is the one most developers will want. The others are provided for legacy compatibility.

  • master branch: This is the most current branch, and the content relies on the DLDT from the OpenVINO Toolkit. This is the only branch that is compatible with the Intel NCS2 however, it is also compatible with the original Intel Movidius NCS device.
  • ncsdk2 branch: This branch is a legacy branch and the content relies on the NCSDK 2.x tools and APIs rather than the OpenVINO toolkit. This branch is only compatible with the original Intel Movidius NCS device and is NOT compatible with the Intel NCS2 device.
  • ncsdk1 branch: This branch is a legacy branch and the content relies on the NCSDK 1.x tools and APIs rather than OpenVINO toolkit. This branch is only compatible with the original Intel Movidius Neural Compute Stick and is NOT compatible with the Intel NCS2 device.

You can use the following git command to use the master branch of the repo:

git clone

Installation and Build Notes


The article linked above asks you to download OpenCV from source directly from OpenCV’s public GitHub repository.  However, as of 7/18/2019, the current repository fails to build on Raspbian 10. Instead, download the stable OpenCV-4.1.0 and build from that source package. Commands are below to build and install OpenCV from that package in a folder called OpenVINO on the current user’s home directory. This will also build the Python* 3 libraries for OpenCV:

This will build a compatible version of OpenCV. You can continue on to build the Deep Learning Deployment Toolkit at this point. Don’t forget to set the OpenCV_DIR environment variable to the install location of OpenCV:

To verify the correct version of OpenCV, use the commands below. These commands require the OpenCV Python 3 wrapper, which is built with the commands above:

The command should return ‘4.1.0’.


OpenVINO should build correctly on your Raspberry Pi 4 if you’ve built the correct version of OpenCV. However, an issue with the GCC* compiler’s default settings may cause runtime issues. To solve this, build the toolkit with the following CMake flag:

Your full CMake command should look like the following:

Powering Your Raspberry Pi

The more modern Raspberry Pi boards, and especially the latest model the Raspberry Pi 3, Model B+, needs a USB power supply that will provide +5V consistently at 2 to 2.5A. Depending on what peripherals the board needs to support that can be a problem.

Typically the Raspberry Pi uses between 500 and 1,000mA depending on the current state of the board. However attaching a monitor to the HDMI port uses 50mA, adding a camera module requires 250mA, and keyboards and mice can take as little as 100mA or well over 1,000mA depending on the model. With the Neural Compute Stick itself requiring at least 500mA.

However I’ve found that most USB chargers will tend to under supply the Raspberry Pi, and as a result the board will register a low power condition and start to throttle the CPU speed. If things get worse the board may suffer brown outs and start to randomly, or repeatedly, reboot.

If you have a monitor attached to your Raspberry Pi board this is the point where you’ll see a yellow lightning bolt in the top right hand corner of your screen. However if you’re running your Raspberry Pi headless, you can still check from the command line using .

$ vcgencmd get_throttled

However the output is in the form of binary flags, and therefore more than somewhat impenetrable. Fortunately it’s not that hard to put together a script to parse the output of the command and get something that’s a bit more human readable.

$ sh ./throttled.shStatus: 0x50005Undervolted:   Now: YES   Run: YESThrottled:   Now: YES   Run: YESFrequency Capped:   Now: NO   Run: NO$

If you the script reports that the board is under-volted it’s likely you should replace your power supply with a more suitable one before proceeding. Unless you’re really sure about your own power supply, I’d recommend you pick on the official Raspberry Pi USB power supply.

The official supply has been designed to consistently provide +5.1V despite rapid fluctuations in current draw. It also has an attached micro USB cable, which means that you don’t accidentally use a poor quality cable — something that can really be an issue.

Those fluctuations in demand is something that happens a lot with when you’re using peripherals with the Raspberry Pi, and something that other supplies — designed to provide consistent current for charging cellphones — usually don’t cope with all that well.

Аппаратное обеспечение Intel для машинного обучения

У Intel есть продукты общего назначения – центральные процессоры, которые могут использоваться как для обучения, так и для использования нейронной сети. Для обучения модели предпочтительны серверные микропроцессоры Intel Xeon ( (в первую очередь это связано со скоростью работы), довольно частая практика – это использование облачных вычислений для тренировки модели, в этом случае, сервера, на которых это облако работает, могут функционировать под управлением Intel Xeon. Для того, чтобы использовать полученную модель подойдут процессоры Intel Xeon, Core, Atom, требования к вычислительным мощностям при работе с готовой моделью ниже, чем к процессу ее обучения, поэтому тут можно обойтись менее производительными устройствами.

Также есть специализированные решения, которые изначально проектировались для работы в рамках направления Deep Learning.  В первую очередь хотелось бы сказать про процессор (если его вообще так можно назвать) Intel Nervana, он в первую очередь ориентирован на тренировку нейронных сетей.

Для запуска натренированной сети предоставляется целый набор различных устройств. Это Intel FPGA  – программируемые логические схемы. После покупки Intel фирмы Altera чипы Stratix, Arria, MAX теперь выпускаются под брендом Intel.

Для этих целей также может использоваться Intel Iris Graphics – интегрированное в микропроцессор графические ядро. Есть специализированные устройства для распознавания речи (Intel GNA (IP)).  Более подробно про аппаратные и программные средства Intel для работы в области искусственного интеллекта можете прочитать здесь. Для нас наибольший интерес представляют чипы Movidius Myriad, в частности Myriad 2, на котором построен Movidius NCS, обзору которого будет посвящен данный цикл статей.

CMake/make Options

The -DENABLE_MKL_DNN=OFF and -DENABLE_CLDNN=OFF options for cmake in the above command prevents attempting to build the CPU and GPU plugins respectively. These plugins only apply to x86 platforms. If building for a platform that can support these plugins the OFF can be changed to ON in the cmake command.

The -DENABLE_GNA=OFF option prevents attempting to build the GNA plugin which applies to the Intel Speech Enabling Developer Kit, the Amazon Alexa* Premium Far-Field Developer Kit, Pentium Silver J5005 processor, Intel Celeron processor J4005, Intel Core i3 processor 8121U, and others.

The -DENABLE_SSE42=OFF option prevents attempting to build for Streaming SIMD Extensions 4.2 which likely aren’t available for most SBC platforms.  

The -DTHREADING=SEQ option builds without threading. Other possible values are OMP which will build with OpenMP* threading and the default value of TBB which is to build with Intel Threading Building Blocks (IntelTBB).

The make command above can also take a -j<number of jobs> parameter to speed up the build via parallelization. Typically the number of cores on the build machine is a good value to use with the -j option. For example with a build system that has 4 CPU cores, a good place to start would be -j4. When building on a lower performance system such as Raspberry P,i omitting the -j option seems to be the most reliable.

Verifying the Build

After successfully completing the inference engine build, the result should be verified to make sure everything is working. When building on the same platform that will be the deployment platform, this is relatively straight forward. If there is a separate system on which the software will be deployed then the Deploying section below should be followed to get the required components on the system prior to verifying.

To verify that the Intel NCS 2 device and the OpenVINO toolkit is working correctly, the following items need to be considered. Each item is discussed below in more detail.

  • A program that uses the OpenVINO toolkit inference engine API.
  • A trained Neural Network in the OpenVINO toolkit Intermediate Representation (IR) format.
  • Input for the Neural Network.
  • Linux* USB driver configuration for the Intel NCS 2 device.

Области применения

USB-сопроцессор Neural Compute Stick может пригодиться разработчикам систем искусственного интеллекта, которые могут его использовать в качестве акселератора уже имеющихся ПК для локального ускорения процесса машинного обучения или создания новых нейронных сетей. По данным Movidius, несколько USB-сопроцессоров Neural Compute Stick, подключенных к системе, увеличивают ее производительность практически линейно. 

Интеллектуальная система распознавания на базе Neural Compute Stick

Compute Stick также может заинтересовать компании, планирующие выпускать собственные продукты с возможностью оперативного локального формирования нейронных сетей с помощью простого подключения USB-совместимого устройства.

Running Machine Learning in Python

The Python wrappers for the Intel OpenVINO toolkit require NumPy and OpenCV support so before we do anything else, lets go ahead and install both those packages for Python 3.5.

$ sudo apt-get install python3-numpy$ pip3 install opencv-python

and install some other additional libraries needed by the code.

$ sudo apt-get install libgtk-3-dev$ sudo apt-get install libavcodec-dev$ sudo apt-get install libavformat-dev$ sudo apt-get install libswscale-dev

We’ll be using a modified version of the demo script. While I managed to get it to mostly run, it had problems opening the camera stream, so I had to kludge things a little bit to make them work.

Open the script in your favourite editor

$ cd ~/inference_engine_vpu_arm/deployment_tools/inference_engine/samples/python_sample/object_detection_demo_ssd_async$ vi

then modify lines 83 through to 85,

if args.input == 'cam':        input_stream = 0    else:

to have the full path to our V4L device.

if args.input == 'cam':        input_stream = '/dev/video0'    else:

First of all this will stop the script hanging with a pipeline error, and secondly avoid the infamous error you get when trying to force quit the script after it hangs.

We can reuse the same model we downloaded for our earlier example. So going ahead and dropping the copy of our new script into our home directory we can set it going,

$ cd ~/inference_engine_vpu_arm/deployment_tools/inference_engine/samples/python_sample/object_detection_demo_ssd_async$ python3 ./ -m ../../build/face-detection-adas-0001.xml  -i cam -d MYRIAD -pt 0.5

If all goes well you should see a window open up on your desktop with a video feed from the Pi Camera Module, with real time inferencing overlaid on top.

Video with real time inferencing overlaid streamed directly to the Raspberry Pi monitor.

This will work just fine on the primary display for your Raspberry Pi, so if you have a monitor attached the window should just pop open.

If you’re running headless the easiest thing is to enable VNC, and connect to your Raspberry Pi that way. Although bear in mind that if you don’t have a monitor connected you‘ll need to set a default resolution using , as the default display is just 720×480 pixels in size. Go to and then and select a display size that’ll fit on your local machine’s display

Real time video with inferencing results on my macOS desktop.

If you’re are connected to the Raspberry Pi from your local machine it is possible to get the window to display on your local desktop, so long as you have an X Server running and have enabled X11 forwarding to your local machine.

$ ssh -XY pi@neural.localpassword:

Otherwise the script will close with a error.

Object detection on custom model

Important note: To run custom model one must optimize trained tflite model in order to use it together with Coral USB. For this reason one must install Edge TPU Compiler first, however it is not possible to deploy this application on Raspberry Pi since The Edge TPU Compiler is , herefore it shall be installed on a host machine:

1. Install Edge TPU Compiler:

sudo apt-get install edgetpu-compiler

2. Compile tflite-model:

edgetpu_compiler /path/to/detect.tflite --out_dir /path/to/output/directory/

Note #1: «/path/to/output/directory/» is any folder where all trainingresults could be saved to.Note #2: In «Object detection with TensorFlow on Raspberry Pi ▸ Training» is described how to create «detect.tflite» (follow steps #4–10).

3. Compiler saves optimized model as «ORIGINAL_NAME_edgetpu.tflite»together with log-file.

4. Create label-file and add all objects which the model was trained on enumerating them starting from zero and separating numbers from object names by whitespace:

5. Run the command:

python3 /path/to/detect_edgetpu.tflite-l /path/to/labelmap.txt -c 0.5


  1. Install OpenCV 4 on Raspberry Pi 4 and Raspbian Buster
  2. OpenVINO, OpenCV, and Movidius NCS on the Raspberry Pi
  3. Install OpenVINO toolkit for Raspbian* OS
  4. Raspberry Pi and Movidius NCS Face Recognition
  5. Intel Distribution of OpenVINO Toolkit. Q&A
  6. Install OpenVINO toolkit for Raspbian OS
  7. Object detection and image classification with Google Coral USB Accelerator
  8. Getting started with Google Coral’s TPU USB Accelerator
  9. Edge TPU Compiler
  10. TensorFlow models on the Edge TPU
  11. Get started with the USB Accelerator

ncappzoo Repository Layout

The ncappzoo contains the following top-level directories. See the README file in each of these directories or just click on the links below to explore the contents of the ncappzoo.

  • apps : Applications built to use the Intel Movidius Intel NCS and Intel Neural Compute Stick 2. This is a great place to start in the ncappzoo!
  • networks : Scripts to download models and optimize neural networks based on any framework for use with the Intel NCS and Intel NCS2.
  • caffe : Scripts to download caffe models and optimize neural networks for use with the Intel NCS and Intel NCS2. Note: this is a legacy directory and new networks will be in the networks directory.
  • tensorflow : Scripts to download TensorFlow models and optimize neural networks for use with the Intel NCS and Intel NCS2. Note: this is a legacy directory and new networks will be in the networks directory.
  • data : Data and scripts to download data for use with models and applications that use the Intel NCS and Intel NCS2

The top-level directories above have subdirectories that hold project content. Each of these project subdirectories has one or more owners that assumes responsibility for it. The OWNERS file contains the mapping of subdirectory to owner.

Intel Neural Compute Stick. Искусственный разум на флешке — 2 +4

  • 22.11.18 07:28





Машинное обучение, Обработка изображений, Блог компании Intel, Интернет вещей

Не дожидаясь, когда другие вендоры воспользуются ее референсным дизайном AI-модулей на все случаи жизни, Intel сама вывела на рынок новое поколение специализированного компьютера в форм-факторе USB-донгла — Intel Neural Compute Stick 2. Предыдущий Compute Stick еще носил имя Movidius и был выпущен в 2017 году непосредственно после покупки Intel одноименной компании. Теперь у него новое название и новый процессор, существенно превосходящий по производительности предыдущий. А вот цена осталась та же.
Основой устройства служит Intel Movidius Myriad X Vision Processing Unit (VPU) — специализированный SoC, содержащий в себе 16 вычислительных ядер общего назначения, а также аппаратные компоненты для ускорения инференса нейронных сетей и компьютерного зрения. По сравнению со своим предшественником Intel Neural Compute Stick 2 обеспечивает 8-кратный прирост производительности в таких задачах, как классификация изображений и определение объектов.
Основные характеристики устройства таковы:

  • Поддерживаемые фреймворки: TensorFlow, Caffe
  • Подключение: USB 3.0 Type-A
  • Размеры: 72.5 x 27 x 14 мм
  • Совместимые ОС: Ubuntu 16.04.3 LTS (64 bit), CentOS 7.4 (64 bit), Windows 10 (64 bit)

Open Visual Inference & Neural Network Optimization (OpenVINO) toolkitна специальном разделе сайта IDZдоступен для предзаказа

Комментарии (16):

  1. Sdima1357

    22.11.18 10:59/#19401666
    / -3

    Никому не нужная фигня версии 2. Цена не соответствует функциональности. Если бы стоила в десять раз меньше, то можно было бы подумать к чему приспособить, а так нет…

    • saul

      23.11.18 09:26/#19406168

      Если в 10 раз меньше — это уже обычная флешка получится.


      23.11.18 11:06/#19406602

      Производительность около одного ТОРs. Orange Pi stick около 2.8 ТОРs за те же деньги .($67 ) Топовый Nvidia более 100 int8 tera op per second.

  2. Vespertilio

    22.11.18 11:05/#19401696
    / +2

    > обеспечивает 8-кратный прирост производительности в таких задачах, как классификация изображений и определение объектов.
    Относительно чего? Есть какие-нибудь сравнительные тесты относительно современных карт Нвидиа например? И почему нет поддержки МакОС?


    22.11.18 11:19/#19401758

    Тоже интересно


    22.11.18 03:11/#19403300
    / +2

    Относительно первой версии.


    23.11.18 09:22/#19406156

    Они же не соревнуются в производительности с видеокартами.
    У них же фокус на портативные девайсы с нейронкой.

  3. Plesser

    22.11.18 12:02/#19401970

    На предыдущей версии нельзя было обучать модели, можно ли на этой это делать?


    22.11.18 07:16/#19404608
    / +1

    Думаю, что нет, т.к. в этом нет смысла. Эти штуки предназначены чтобы впихнуть предобученную нейросеть в какой-нибудь квадрик. Не для того, чтобы из них собирать кластер. (Я про эту не скажу, но прошлые и rapsberry pi прикидывал, топовая видеокарта уделывает такие кластеры по соотношению цена/производительность, цена/энергопотребление как детей.)

  4. leshabirukov

    22.11.18 04:08/#19403606

    Основные характеристики устройства таковы: Производительность? Напишите, сколько опс и каких (типы данных).

  5. OnelaW

    22.11.18 05:35/#19404134

    Мозг сломал пока сообразил что это за зверек. Как отдельный донгл идея интересная, а если в виде платы расширения под pci? Или гибридное решение.

    • saul

      23.11.18 09:25/#19406162

      Про платы расширения — это предыдущий пост. Сейчас есть ускорители на FPGA, но они сильно дорогие.

      • Mogwaika

        23.11.18 10:25/#19406408

        А какие опреации нужны для нейронок для обучения и для работы?


        23.11.18 11:04/#19409448

        Для работы — смотря что за функция в нейроне. Обычно нужна сложение-умножение. Для обучения — смотря что за метод.

  6. r_ii

    22.11.18 07:03/#19404572

    Вполне себе полезный «ускоритель» для IoT устройств. Практический пример — позволяет распознавать объекты в видео потоке в режиме реального времени на Raspberry Pi 3.
    Но я бы не стал его рассматривать как устройство для PC.

  7. n0th1ng

    23.11.18 06:39/#19405876

    Интересно, можно ли с помощью этого стика и Raspberry Pi 3 сделать автономного робота?

Intel Movidius Neural Compute Stick (NCS)

Если “в двух словах”, NCS – это устройство, которое подключается по USB к вашему компьютеру или Raspberry PI, оно предназначено для запуска на нем уже готовых (обученных) нейросетей, обладающее очень низким энергопотреблением, порядка 1 Вт.

Данное устройство содержит 4Gb памяти LPDDR3 DRAM и VPU Intel Movidius. В  состав Myriad 2 входят 12 VLIW 128-битных векторных процессора SHAVE, конфигурируемые ускорители для обработки изображений, микропроцессор Leon и интерфейсы для организации связи с различными устройствами.

Блок схема устройства показана на рисунке ниже.

Векторные процессоры SHAVE используются для запуска на них нейросетей. Основное назначение микропроцессора Leon – это координация работы через USB интерфейс (загрузка графа нейросети и изображений для анализа и т.п.), также он выполняет задачу мониторинга температуры и управление устройством, в случае ее повышения.

Ваше программное обеспечение может взаимодействовать с NCS через специальный NCAPI, который входит в состав NCSDK. Разрабатывать ПО можно на языке Python либо C/C++. В процессе ознакомления с данным продуктом мы будем использовать Python. Схема организации взаимодействия с NCS выглядит так как представлено на следующем рисунке.

Verifying Installation

After successfully completing the inference engine build, you should verify that everything is set up correctly. To verify that the toolkit and Intel NCS 2 work on your device, complete the following steps:

  • Run the sample program benchmark_app to confirm that all libraries load correctly
  • Download a trained model
  • Select an input for the neural network
  • Configure the Intel NCS 2 Linux* USB driver
  • Run benchmark_app with selected model and input.

Sample Programs: benchmark_app

The Intel OpenVINO toolkit includes some sample programs that utilize the inference engine and IIntel NCS 2. One of the programs is benchmark_app, a tool for estimating deep learning inference performance. It can be found in ~/dldt/inference-engine/bin/intel64/Release.
Run the following command in the folder to test benchmark_app:

It should print a help dialog, describing the available options for the program.

Downloading a Model

The program needs a model to pass the input through. Models for Intel OpenVINO toolkit in IR format can be obtained by:

  • Using the Model Optimizer to convert an existing model from one of the supported frameworks into IR format for the Inference Engine
  • Using the Model Downloader tool to download from the Open Model Zoo
  • Download the IR files directly from

For our purposes, downloading directly is easiest. Use the following commands to grab an age and gender recognition model:

Note: The Intel NCS 2 requires models that are optimized for the 16-bit floating point format known as FP16. Your model, if it differs from the example, may require conversion using the Model Optimizer to FP16.

Input for the Neural Network

The last item needed is input for the neural network. For the model we’ve downloaded, you need a 62×62 image with 3 channels of color. This article includes an archive that contains an image that you can use, and is used in the example below. Copy the
archive to a USB Storage Device, connect the device to your board, and use the following commands to mount the drive and copy its contents to a folder called OpenVINO on your home directory:

Use the lsblk command to list the available block devices, and make a note of your connected USB drive. Use its name in place of sdX in the next command:

The OpenVINO folder should contain two images, a text file, and a folder named squeezenet. Note that the name of the archive may differ – it should match what you have downloaded from this article.

Configure the Intel NCS 2 Linux* USB Driver

Some udev rules need to be added to allow the system to recognize Intel NCS 2 USB devices. Inside the tar.gz file attached there is a file called 97-myriad-usbboot.rules_.txt. It should be downloaded to the user’s home directory. Follow the commands below to add the rules to your device:

Note: If the current user is not a member of the users group then run the following command and reboot your device:  

While logged in as a user in the users group:

The USB driver should be installed correctly now. If the Intel NCS 2 is not detected when running demos, restart your device and try again.

Running benchmark_app

When the model is downloaded, an input image is available, and the Intel NCS 2 is plugged into a USB port, use the following command to run the benchmark_app:

This will run the application with the selected options. The –d flag tells the program which device to use for inferencing – MYRIAD activates the MYRAID plugin, utilizing the Intel NCS 2. After the command successfully executes the terminal will display statistics for inferencing.
If the application ran successfully on your Intel NCS 2, then Intel OpenVINO toolkit and Intel NCS 2 are set up correctly for use on your device.

Inferencing at the Edge

Now that you’ve confirmed your ARMv7 is setup and working with Intel NCS 2, you can now start building and deploying your AI applications or use one of the prebuilt sample applications to test your use-case. Next, we will try to do a simple image classification using SqueezeNetv1.1 and an image downloaded to the board. To simplify things the attached archive contains both the image and the network. The SqueezeNetv1.1 network has already been converted to IR format for use by the Inference Engine.

The following command will take the cat.jpg image that was included in the archive, use the squeezenet1.1 network model, use the MYRIAD plugin to load the model to the connected Intel NCS 2, and infer the output. As before, the location of the sample application is <OpenVINO folder>/inference-engine/bin/armv7/Release/

The program will output a list of the top 10 results of the inferencing and an average of the image throughput.


Despite getting there first, or more likely because of it, I haven’t been that impressed with the standard of documentation around the Intel Neural Compute Stick 2. There are multiple versions of the documentation on the developer site, and they mostly all contradict each other in subtle ways. There are also a number of, albeit minor, errors in the Intel demo code, and at least for the Raspberry Pi, the setup scripts that install pre-compiled models seem to be missing from the distribution entirely.

Compared with the documentation around Google’s new Coral hardware, or NVIDIA’s new Jetson Nano, the getting started instructions for the Compute Stick are not developer friendly. The Python wrappers also seem to be at a lower abstraction level than those shipping with the Coral hardware.

Finally unlike Google’s new Coral hardware, which has an online compiler to convert your own models to the correct format, you’ll need to install OpenVINO on another x86-based Linux box if you want to convert your models to use with the Compute Stick on the Raspberry Pi.

Overall the developer experience on the Raspberry Pi just feels like a cut down version of the full OpenVINO toolkit thrown together. Which is somewhat disappointing.

Компактный нейронный сопроцессор

Компания Movidius, принадлежащая Intel и занимающаяся разработкой визуальных процессоров для интернета вещей, представила миниатюрное компактное устройство Neural Compute Stick. Новинка позиционируется как вычислительный сопроцессор с функциональностью искусственного интеллекта, позволяющий добавить возможность машинного обучения подключенному ПК простым подключением к порту USB.

Ключевая особенность устройства заключается в том, что для процесса машинного обучения или развертывания новой нейронной сети с применением Neural Compute Stick не требуется подключение к интернету: USB-сопроцессор функционирует совершенно автономно.

В сентябре 2016 г. компания Movidius была приобретена Intel за неназванную сумму. По итогам сделки в Intel объявили о планах использовать технологии Movidius при разработке устройств интернета вещей, дополненной, виртуальной и совмещенной реальности, таких как роботах, дронах, автоматических цифровых камерах безопасности и т.д.

Впервые USB-сопроцессор Neural Compute Stick был представлен в качестве прототипа под рабочим названием Fathom в апреле 2016 г. – тогда еще независимой компанией Movidius. Некоторое время после приобретения компании новости о разработках Movidius исчезли со страниц новостных сайтов. Теперь Neural Compute Stick коммерциализирован и официально поступает в продажу, однако технологическая идея устройства претерпела минимальные изменения по сравнению с Fathom.

USB-сопроцессор Movidius Neural Compute Stick

Устройство Neural Compute Stick выполнено на базе точно такого же процессора, который используется во множестве устройств с машинным зрением — например, автономном дроне DJI. Потребителю или производителю техники, желающему усилить возможности искусственного интеллекта своей системы, достаточно подключить один или несколько сопроцессоров Neural Compute Stick к порту (портам) USB.

OpenVINO™ Toolkit Program: benchmark_app

When the inference engine was built via cmake/make some sample applications were built as well. All the build samples can be found in the <directory where cloned>/dldt/inference-engine/bin/<arch>/Release directory. For the purpose of verifying the build, the benchmark_app in this directory can be used. To see the possible arguments to this application the following command can be used from within the Release directory:

Running the command above will show that the following options, among others, which are relevant to verifying the build with an Intel NCS 2 device. 

Arguments for the benchmark_app
Argument Description
-i «<path>» Is the path to an image file or directory.
-m «<path>» Is the path to the trained model’s .xml file. This is one of the two files created when a trained model is converted to the IR format. There is an assumption that the other file, the .bin file exists in the same directory as this xml file.
-pp «<path>»  Is the path to the directory that contains the inference engine plugins. This is where the Myriad plugin is located on the file system. The Myriad plugin is the shared object library. After the build completes this is in the …/Release/lib directory.
-d «<device>» Specifies the target device on which to run the inference. To verify the Intel NCS 2 device pass -d MYRIAD 
-api «<sync|async>» Specifies if the Inference Engine API to call should be synchronous or asynchronous. The best performance will be seen when passing «async».

Trained Neural Network

There are a few ways a trained neural network in IR format can be obtained:

  • Run the model optimizer to convert a trained network in one of the supported frameworks to IR.
  • Use the model downloader tool from the open_model_zoo repository
  • Download the IR files from directly

Downloading directly from is the simplest. The following commands will create a directory in the current user’s home directory and then download an age and gender recognition network that is already in IR format.

Note: After the above commands are successfully executed the following files will be on the system: ~/models/age-gender-recognition-retail-0013.xml, and ~/models/age-gender-recognition-retail-0013.bin. These files will be passed to the benchmark_app when executed below.

Note: The Intel NCS 2 device requires IR models that are optimized for the 16 bit floating point format known as FP16. The open model zoo also contains models in FP32 and Int8 depending on the specific model. 

What’s Relevant in the Repository

The inference-engine provides the API used to initiate neural network inferences and retrieve the results of those inferences. In addition to the API, the inference engine directory also includes plugins for different hardware targets such as CPU, GPU, and the MYRIAD VPU. When the API is called it will rely on the plug-ins to execute inferences on these hardware targets.For execution on the Intel NCS 2 (or the original Intel Movidius NCS) the inference engine needs to call the MYRIAD VPU plug-in. Therefore, the bare minimum which will need to be built to use an Intel NCS 2 is the inference engine API and the MYRIAD plug-in. The other plug-ins in the repository will not be needed and depending on your platform may not apply.

Setting Up to Build

To prepare the system for building, the build tools, the git submodules, and the dependencies need to be on the build system. Each of these are explained in this section.

Object detection on pretrained model

1. Install EdgeTPU examples:

sudo apt-get install edgetpu-examplessudo chmod a+w /usr/share/edgetpu/examplestree — dirsfirst

In folder «/usr/share/edgetpu/examples» one can find already pretrained models in tflite-formats optimized for EdgeTPU.

2. Unfortunately Coral USB vendor does not provide ready script to rundetection using webcam, so one should use custom script:


3. Run the script using SSD Mobilnet V1 COCO:

python3 /path/to/ssd_mobilenet_v1_coco_quant_postprocess_edgetpu.tflite -l /path/to/coco_labels.txt -c 0.5

Note: If white LED on Coral USB does not blink it means that either the modelis broken or not optimized for using together with Raspberry Pi.

Setting Up Your Raspberry Pi

Unlike Google’s new Coral Dev Board, which needs a lot of setup work done before you can get started, there isn’t really a lot to do here. Grab a Raspberry Pi, a power supply, a USB cable, and a micro SD card, and you’re ready.

Everything you need to get started setting up the Intel Neural Compute Stick 2.

If you’re used the Raspberry Pi for anything before it’s probably a good idea to install a fresh version of the operating systems and work from a clean slate. Go ahead and download the latest release of Raspbian Stretch (with desktop) and then set up your Raspberry Pi.

Unless you’re using wired networking, or have a display and keyboard attached to the Raspberry Pi, at a minimum you’ll need to put the Raspberry Pi on to your wireless network, and enable SSH. Although you might want to go ahead and enable VNC as well, as it might prove useful.

Once you’ve set up your Raspberry Pi go ahead and power it on, and then open up a Terminal window on your laptop and SSH into the Raspberry Pi.

% ssh pi@raspberrypi.local

Once you’ve logged in you might want to change the hostname to something less generic using the tool, to let you tell it apart from all the other Raspberry Pi boards on your network, I chose .