“AI on Edge” – With PIXY

AI on edge is getting more popular with many embedded chip vendors eyeing on AI market – 
Intel AI
STMicro
NXP
Qualcom
Understanding this trend, AI service providers like Google , Amazon and Microsoft are launching lite version which can run on micro-controllers- for example Tensor-lite

What is AI on Edge?
In the traditional IoT world, data is collected by sensors and transmitted to cloud for further processing. With the advent of ML and DL, lot of inferences can be made from the collected data. However there is big catch, what if communication fails and secondly action need to happen on the ground, hence data from cloud has to flow back to the shop floor to make the final actions/response. Thus taking the data and inference had to be taken at the edge (micro-controller) itself is getting popular. Thus power consumption and communication cost can be reduced.

However there is a challenge , normally the micro-controllers are bare metal. Micro-controllers are often optimized for low energy consumption and small size, at the cost of reduced processing power, memory, and storage. Hence there is a need for lighter AI frameworks. Google’s tensorflow is made trim to suit this need. More details here. 

Typical steps of AI has to be modified to suit Edge
— > In stead of heavy models, lightweight models to be made
–> convert model and c-byte array
–>Integrate this c byte into micro-controller C++ library and make inference on edge
–>deploy and test

Google supports STM32F , Apolo V3 blueboard by Ambiq , PIXY from CMU &charmedlabs

Machine vision which requires heavy power and rich dataset, is far fetched dream for micro controllers for some years .. But last two-three years, there are microcontrollers that can do machine vision objectives like object detection, line following, barcode reading..etc..

You can see both “yellow ball ” and “bluesmiley” is detected and their co-ordinates are shown in serial monitor.

In this post,we shall talk about pixy camera & understand how to achieve the above objective.

What is PIXY 2

The Pixy2 has these additional functions:
Detects objects

• Detects lines, intersections and simple barcodes.
• Has a framerate of 60 frames per second.
• Has an integrated light source consisting of two white LEDs.
• New tracking algorithms that allow you to track objects.

Connections:

• USB connection.
• Serial UART connection.
• I2C bus connection.
• SPI Bus connection
• Analog output.
• Digital output.

A lot of information is made available in the Pixy2 Wiki. 

Package contains, one pixy2 board, usb cable and one ribbon cable to connect to Arduino (SPI) interface

pixymon is the software to work with PIXY 2 camera.
download pixymon and install as defined in the page

Next thing is teaching pixy2 to read an object (in ML terms, training the system). Pixy2 follows color based object detection. SO it works well when the object has higher hue (Like full blue object or yellow)

There are two ways :
1) using guidance of led lights of pixy2
2) using pixymon

Bring the object to be trained in the middle of camera view and then press button and hold it for few seconds, the object will get locked, then release and click the button again to complete the training. THats it, Afterwards, where ever object is in the frame, that will get identified and bounding boxes drawn around it

Next way is using pixymon, click on setsignature and draw the area of the object border , then object get locked

Using configure options, fine tuning of the object locking can be done

Now the objects which are trained and set signature can be tracked.

Next step is to integrate this output to microcontroller so that actions can be performed.

Here I connected pixy2 with arduino uno board. Connection is straight forward – just connect the cable provided along with pixy2, it sits rightly in pixy2, in arduino, be watchful to connect (red line to be outside)

Then install the example library of the pixy2 into arduino

What are the uses-cases you can think off, that can make user of pixy