Embedded Access Control System with Facial Recognition

Client

A European car-sharing company.

Challenge

The project goal was to build and manufacture a security monitoring hardware and software product with facial recognition designed to track access to carsharing and other restricted areas. 

Solution

We designed a hardware and software system that consists of the following components:

• A series of client devices (prototypes / smart cameras) for the face recognition and detection of various activities or events around shared vehicles.
• Cloud-based service for administering multiple car fleets, users and client devices, data reading from devices and user access logging.
• A mobile app for user registration and status control.

The system architecture:

^{Fig. 1. The service software architecture}

1. Hardware Design

The features of the designed hardware platform:

• NXP i.MX 8M Plus Quad applications processor with
  • 4x Cortex-A53 up to 1.8 GHz
  • Machine Learning Neural Processing Unit (NPU) with 2.3 TOPS
• 1 MIPI CSI сamera

2. Firmware Development

The tasks we completed:

• built Yocto Zeus OS;
• implemented a working prototype for facial recognition;
• integrated the prototype with a GCP application;
• implemented tampering detection; 
• implemented OTA firmware update.

The software packages we used:

• OpenCV
• Python 3.9, TensorFlow Lite v2.6.0
• OS Yocto Linux

As a result, the system will provide the following features:

1. Movement detection and facial recognition.
2. Access verification.
3. Anti-spoofing alert.
4. Visitors’ identity, logging and analytics.
5. Strange behaviour alert.
6. Video footage recording and storage.
7. Data security.
8. Privacy safety. Facial photographs are encrypted as a mathematical representation of the image in storage and transit, not the image itself.
9. Cloud-based scalable solution.

^{Fig. 2. Results of faces recognition in real-time}

3. Cloud Software Service

We have developed a prototype of the web service hosted on GCP, which enables the managing and logging of multiple car fleet companies, its client devices and users. The services provide the following features:

• Administering company profile, company clients’ and hardware.
• New client registration via the mobile app or browser. 
• Database of clients’ face signatures (no raw photo of a client is stored, just its mathematical representation, to preserve privacy).
• Real-time data exchanging with an installed fleet of devices.
• Access control based on face recognition, anti-spoofing, driver drowsiness detection, register time logs/regular access control.
• Retrieving video footage of critical events.
• Collecting device and client metrics.
• OTA firmware updating.
• ADAS — advanced driver-assistance system. 
• Alerting the drivers before they fall asleep by monitoring their pupils and blink rate. 

Business Value

Our engineering team successfully completed the proof of concept phase and built and tested the MVP. We achieved a recognition accuracy of over 99% with a frame rate of up to 16 frames per second. The device would be manufactured and installed in the customer's cars. 

One of the deliverables from this project is an API that can be used in other areas. The technology can be used for driver identification in the automotive industry, including fleet management and car insurance. It could also be used in surveillance and security systems to enhance identification and tracking capabilities. The technology would also be helpful as an access system for storage facilities and other similar environments.