Smart video surveillance system for a bike parking based on the SONY Spresense AI camera

Client

This international project was initiated by one of the local offices of SONY. SONY cooperated with a French startup Nielsen concept, focused on innovative bike parking solutions. Promwad engineers worked on the project with Aduk, an electronics development company in Germany.

Challenge

SONY has released a new AI platform, Spresense, and wanted to develop a proof of concept to demonstrate possible applications of the platform. 

The project's goal was to make a Spresense-based tracking system for bike parks that automatically detects available parking spaces, reacts to motion, switches on the power and turns on the light for new users. 

There were several other requirements:

• Creating a software solution to communicate data to a cloud service.
• Creating a hardware solution: power source and GSM modem peripherals for the Spresence.
• Creating a case and mounting for the prototype.

Solution

As a solution, we proposed a system that consists of a camera, an image processor, and a GSM modem, and all are assembled in a compact enclosure.

Due to the limited memory (1.5 MB) and limited computational capability of the board, it was not possible to use standard object localization neural networks (NN), such as YOLO or Mobilenet SSD. It was only possible to run restricted object detection NN-based on Mobilenet on the platform.

However, the advantage was that the computation time could be very large: once a bicycle is parked, nothing is changed in the picture, and we can process it for many seconds.

^{Figure 1. Sliding window for bicycle counting.}

As a result, we developed a solution that used the Mobilenet neural network in the object detection mode. We used the sliding window technique to trade recognition speed for the low memory footprint: the resulting Mobilenet was less than 1 Mb in size, but the scanning time was about one minute.

To address the limitations of our dataset, we created a synthetic dataset using Blender, and an open-source 3D computer graphics software toolset used for visual effects and 3D-printed models.

^{Figure 2. Synthetic image generated with Blender}

We also developed connectivity software, designed the enclosure and developed the peripherals for the Spresence board.

To work with Mobilenet, we used the Tensorflow framework — the tools provided with this framework to train a model on bicycle shots (the real and synthetic images). 

To enhance the dataset and fight misclassification, we involved the LIME analysis. LIME is the acronym for local interpretable model-agnostic explanations.

Also, we used Tensorflow Lite Micro to port the model into the Spresence microcontroller environment.

Business Value

Our engineers have designed the first smart bike parking system based on the SONY Spresense computer vision system. A test sample device was installed in Nantes, France. The client received a unique customised solution that is unparalleled in the market.

As part of this project, our team also trained a neural network for bicycle counting. However, this tracking system can be used not only for bicycle parks: our team can adapt this solution to any project where remote tracking is needed. We will be happy to adapt our AI system to your requirements, please feel free to contact us.