Coral USB Accelerator: A USB Accessory For Machine Learning Inferencing in Existing Systems

Artificial intelligence is no longer the future of technology; it is here. It comes in various types, such as voice assistants and machine learning, and is an exciting field to try out different projects when developing intelligent systems. However, running AI systems can be resource-intensive, eating up all your memory and processing power. But with the Google Coral USB Accelerator, you can get extra processing power via USB for ML inferencing. Here's all you need to know about the single board computer and how to start using it.  

What is the Coral USB Accelerator?

The Google Coral USB Accelerator is a USB device that brings powerful machine learning inference capabilities to existing Windows, Mac, or Linux host computers. It features an Edge TPU to function as a coprocessor for your host PC, bringing low power cost over USB 3.0. This Coral Edge TPU is a small Google-designed and built ASIC computer. Its primary benefits include the following:

• Low power consumption
• High-speed TensorFlow Lite inferencing (can do real-time inferencing)
• Small size

With the TPU USB accelerator, you can run mobile vision models like the MobileNet v2 at over 100 fps in a power-efficient and privacy-preserving manner.  

Coral USB Accelerator Specifications and Features

The Coral USB accelerator has the following features.

• A Google Edge TPU machine learning accelerator coprocessor
• USB 3.0 socket (Type-C)
• Google cloud compatibility
• Linux Debian support on host CPU
• Built using TensorFlow (supports Inception, MobileNet, and custom architectures)

 

A TPU 3.0 device

Source: Wikimedia Commons   Going further into the details, these are the accelerator's specifications.  

Get Started With the USB Accelerator

Before setting up the Google Coral USB accelerator to run an object detection model, you need the following items/ tools.  

Requirements

• A PC running either of the following operating systems:
  • Windows 10
  • macOS Catalina of Big Sur with HomeBrew or MacPorts installed
  • Linux Debian 10 with ARMV7 (32-bit), ARMV8 (64-bit), or Intel with x86 or 64 architecture
• Python 3.6-3.9
• A USB port (USB 3.0 is best)

 

Install the Edge TPU Runtime

Once you have assembled the above requirements, the next step is to install the Edge TPU runtime. It contains the core programming required for interfacing the Edge TPU with your host PC.  

For Windows

1: Install the latest version of Microsoft Visual C++ 2019 redistributable. 2: Download the Edge TPU runtime zip file. 3: Unzip this file and run the file "install.bat." It should open the install script, and you'll get a prompt asking whether to run at the maximum operating frequency. Type either Y or N. Keep in mind that running at max frequency leads to more power consumption and makes the USB accelerator extremely hot. However, it raises the inferencing speed, delivering better performance. Step 4: Connect the accelerator to your computer using the USB 3.0 cable.  

For Mac

Step 1: Download and unzip the Edge TPU runtime zip file. Use the following commands.

Step 2: Install the runtime using the commands below.

You will get a prompt to enable or disable the maximum operating frequency. Respond with Y or N to alter the inference speeds depending on your performance requirements. Step 3: Connect the device (USB accessory) to your Mac via the USB 3.0 cable.  

For Linux

Step 1: Add the Debian package repository to your system using the following commands.

Step 2: Install the runtime using this command.

Use this command instead to enable maximum operating frequency for high inference speeds.

Step 3: Connect the Google Coral USB accelerator to your computer using the USB 3.0 cable. You have to unplug and plug it back in again for the "udev rule" to take effect if you initiated the process with the device connected.  

Install the PyCoral Library

The PyCoral python library is an extension to the TensorFlow Lite library, and its purpose is to hasten code while providing extra functionality to the TPU. Use the following commands to install the library plus its dependencies.  

For Windows

You have three options for installing the library on a Windows PC.

• Use the py launcher (with python 3.5 or newer)
• Run one of the following commands

Or

• Download the specific PyCoral wheel file, then use the pip install command to install it

 

For Mac

The only option for Mac computers is to run one of the following commands

Or

 

For Linux

The Linux command is short and simple.

 

Run a Model on the Edge TPU

Lastly, run a model in the TPU. To set up the image classification model, Step 1: Download the code from GitHub using the following command.

Step 2: Next, download the bird photo, model, and label file using this command. bash examples/install_requirements.sh classify_image.py Step 3: Complete the process by running this bird classifier model.

 

Results

You should get a result similar to the one below.

It is vital to note that the inference speeds can vary depending on the transfer speed of your USB (3.0 is faster), host computer, and the model architecture. You can try out another image or run different models for semantic segmentation, real-time object detection, etc. Alternatively, you can build your projects using TensorFlow Lite API in C++ or Python. However, it requires more lines of code to get the pre-trained TensorFlow lite models working. If you don't like the precompiled models, you can build custom ones from the ground up.  

Coral USB Accelerator Alternative

If you don't get the Coral USB accelerator, the computer has the following alternatives.  

UDOO Bolt

UDOO's Bolt arguably had the most powerful GPU in a single board computer in 2019, the AMD Vega. It also features the AMD Ryzen Embedded V1XXX CPU and SO-DIMM RAM. However, it is relatively huge (120 x 120 mm) and expensive.  

Intel NUC

The NUC has three advantages: a low 10-15W TDP, supports most operating systems, and 6th+ Gen models offer CEC support. However, the unit has modest features, so you need to spend more to buy the iX-based models.  

The Intel NUC

Source: Wikimedia Commons  

Coral USB Accelerator: Raspberry Pi 4 Model B

As one of the most popular Coral USB accelerator alternatives, the Raspberry Pi 4 Model B is affordable, compact (85 x 56 mm), and has a sizable RAM (1, 2, or 4 GB). However, it is laggy and can get pretty hot. The Pi 4 Model B is a newer version of the Pi 3 Model B+, which costs the same price, but is inferior performance-wise.  

The Raspberry Pi 4 Model B

Source: Wikimedia Commons  

Coral USB Accelerator: ASUS Tinker Board S

The Tinker Board S has one of the most powerful processors in a single board computer, the Rockchip Quad-Core RK3288. Combined with its GPU (ARM Mali-T764), the device can also stream 4K media at 30Hz via its HDMI port. However, it comes without a power supply unit and consumes a lot of power (requires a 3A power supply).  

Coral USB Accelerator: ODROID-XU4

One of the best features of the XU4 is its fast gigabit ethernet port. On top of that, it includes a cooling system, USB 3.0 ports, and supports eMMC 5.0 storage. However, it corrupts SD cards, has no audio CODEC & SATA port, and does not support most sensors and accessories natively in the market.  

The ODROID-XU4

Source: Wikimedia Commons  

Summary

As you can see, the Google Coral USB Accelerator provides high-performance ML inferencing, including real-time classification using custom or pre-trained deep learning models. If you have any queries, reach out for further clarification.