Pixy2: A Compact Object Detection, Barcode Reading, and Line Tracking Camera

Artificial intelligence is advancing every day, and it needs sensors and cameras to get input automatically from its surroundings. Robotics and autonomous cars are just a few examples where such equipment is essential. Considering cameras, the Pixy2 is an ideal device for such tasks. It is faster, smaller, and more capable than its predecessor (the Pixy cam). We have covered the gadget in detail below, so take a look!  

What is Pixy2?

The Pixy2 is a tiny camera designed primarily for the following three functions:

• Object detection (road signs, intersections, etc.)
• Line detection and tracking
• Barcode detection capabilities/Barcode reading capabilities

Pixy2 Camera Features and Specifications

Features

• Compact, quick, affordable, easy-to-use, and readily available vision system
• Detects and tracks intersections, lines, and tiny barcodes for line-following robots
• 60 FPS
• Color-based object detection using tracking algorithms
• Built-in light source
• Improved, simplified libraries for Raspberry Pi, Arduino, and other single-board computer controllers
• Supports python and C/C++ libraries of code
• A multitude of interface options (I2C, SPI, USB, UART, and analog/digital output)
• Windows, Linux, and macOS compatible configuration utility
• Connects to the controllers via the included cable

Specifications

• Dual-core, 204 MHz, NXP LPC4330 processor
• 1296×976 resolution Aptina MT9M114 dual-core image sensor
• 60° horizontal, 40° vertical field of view (lens)
• 140mA power consumption
• Unregulated 6-10V or 5V USB power input
• 264KB RAM & 2MB flash memory
• USB, UART serial, I2C, SPI, digital, and analog output ports
• Built-in 20-lumen light source

Getting Started Pixy2

Before using Pixy2, install it on your computer using the following steps.  

Install PixyMon

The installation requires a high-quality micro USB cable and the latest PixyMon version.  

Mac Installation

Double-click the downloaded DMG file to mount it. A PixyMon application window will pop up. Drag the app to your applications directory, and that's it for software installation.

Next, connect the Pixy2 camera to your computer using the USB cable. Double-click to launch the installed app, which will connect to the camera.  

Windows Vista, 7, 8, and 10 Installation

After downloading the latest Windows PixyMon version, double-click to install. Proceed with the installation until you get to the "driver pre-installation" part. After a few moments, you should get a dialog box with the text "Would you like to install this device software?"

Click "install." The same dialog box will pop up again after a while because the camera has two device drivers that need installation. Click "Install" again, connect the Pixy2 to your computer using the USB cable and run PixyMon.  

Linux Installation

This process assumes you are running Linux Mint (or any Debian package manager) and connected to the internet.  

Dependency Installation

Next, install the PixyMon dependencies by first opening the terminal. Type the following command to install git.

After that, install libusb-1.0-0-dev using this command.

Next, install QT by typing these three commands.

The g++ compiler installation is next. Type:

At this point, the dependency installation is complete. The next step is building PixyMon.  

PixyMon Building

First, download the source code using the following command.

Second, run the build script by typing:

This script might take some time.

Lastly, add permissions for the USB interface by running the following script.

Running PixyMon

Connect the camera to your PC using the micro USB cable, then change the directory to the PixyMon build folder by typing the following command.

Complete the process by running PixyMon using this command.

The output should be a live video feed from the camera.  

Teach Pixy2 An Object

The Pixy2 sensor uses a hue/color-based filtering algorithm for image detection. Therefore, the colored objects for training should have a distinct hue and include items like green tennis balls, pink dolls, etc.

Green tennis ball

Next, power the camera via USB or batteries. Once all the lights have stopped flashing, press and hold the button at the top of the Pixy2. The LEDs will turn on again in the following sequence: white, followed by red, then other colors. Release the button when the LED turns red.

The camera will enter a light pipe mode and lock onto objects right in the middle of its video frame. Therefore, you should put the item 6-20 inches ahead of the lens.

Pixy2 will determine which pixels cover the object and which ones are part of the background. The LED color should match the object's color, and the brighter the LED, the better the lock. The LED will flash several times to show it has learned your thing. Once satisfied with the results, press and release the Pixy 2 button.

The camera can learn up to seven distinct objects based on the color signatures. These include orange, red, yellow, cyan, violet, blue, and green.  

Lighting Conditions and White Balance

It is crucial to note that the lighting condition (such as fluorescent and incandescent lighting) can affect the white balance. This issue will affect the camera's ability to detect objects accurately. You can solve this problem by retraining the Pixy2 signatures or adjusting the white balance.  

Hook Up to your Microcontroller (Arduino and Raspberry Pi)

Arduino

First, download the latest Arduino library and import it into the IDE to set up the connection.

After that, load the example "ccc_hello_world" (File > Examples > Pixy2), upload it to the board and open the serial monitor. Also, set the baud rate to 115200.

Some messages will get printed on the monitor to show what the color-connected components program has identified. However, it will only print messages if an object matching one of the CCC's color signatures is in front of the camera.  

Raspberry Pi

Connecting a Raspberry Pi to a Pixy2 requires the libpixyusb2 library and a PC running on Linux Mint or any Debian package manager.

First, install the dependencies for building the library. Start the terminal, then type the following command.

Next, download the Pixy2 source code using this command.

After that, build the library using:

Connect the Pixy2 to your computer, then try out one of the example projects. For instance, build the color identification example using this command.

After that, start the example by running the following commands.

Pixy2 Modes / Programs

Color Connected Components

Pixy2's color-based filtering algorithm for detecting object signatures is the Color Connected Components (CCC). Although this algorithm remembers and detects up to seven different color signatures, it can find objects in the hundreds at a time. Pixy2 compiles the location and size of each object, then outputs this data via one of the interfaces.

This multiple object detection is possible via color coding. Color codes are two or more tags placed near each other. The camera can detect and decode color codes, then output them as particular objects.  

Line Tracking

Pixy2 uses an image array sensor to follow a line, which takes in data about the line direction and intersections formed by it. The camera can also read simple barcodes to inform the robot to slow down, turn left or right, etc.

However, the advanced line-tracking algorithm can only detect light lines on a dark background or vice versa. The built-in light source kicks in to make the camera work optimally if it is too dark. Also, you can program Pixy2 to detect specific line widths only.  

Pixy2 Application Projects

Introducing Pixy2 for Lego Mindstorms EV3

Pixy2 communicates with the Lego sensor protocol seamlessly. You only have to plug the included cable into your lego brick, then observe the detected objects on the EV3. Alternatively, you can take the data to your Lego program using Pixy's software blocks. This software simplifies Lego program writing to drag and drop programming, with new blocks including RGB color readout, line-following, and CCC.

Lego Mindstorms EV3 robot

PhantomX Pincher Color Sorting Robot

Since Pixy2 uses a color-based filtering algorithm, it would come in handy in building a color sorting robot using the PhantomX Pincher. The robotic arm can pick an object coming in from a conveyor belt, then place it before the camera for identification. Pixy2 will tell the robot where to place the item based on its color.  

Summary

In conclusion, Pixy2 is a vital sensor for modern applications, especially in the robotics field. Setting up is the tricky part. But once done, you will have fun using the camera in your project. Contact us for further information if you encounter any problems or have any questions.