Exploring Lattice Semiconductor FPGA for Digital Rear-View Mirrors

The automotive industry is increasing driving safety by adding advanced sensors to driver assistance systems with real-time reactions, high-speed embedded processing, and 5G high-bandwidth connectivity. Using SoCs seems to be the logical solution for such systems; however, they lose significantly to FPGAs, which have the necessary processing power and compliance with ASPICE CL2 and ASIL-B safety standards.

In automobiles, microcontroller units (MCUs) and application processors (APs) solve computational tasks, and their number can vary from 25 to 70 units, depending on the vehicle class.

As digital mirrors and automotive driver state monitoring systems become more popular, there is a need to process data from various sensors and cameras within ADAS. The most advanced solutions in this area are built on AI and machine vision algorithms, which require relatively high computing power and adaptability.

The current approach in developing such solutions is to use system-on-a-chip (SoC), but these are not always able to meet the needs for a number of reasons:

• there is no support for multi-sensor systems;
   
• chips are not compliant with ASPICE CL2 and ASIL-B requirements;
   
• there is a lack of standards for the use of chips in automotive designs;
   
• a general shortage of chips, which can jeopardise the release of solutions.
   

However, there is a solution on the market without these disadvantages, and it can be used to design digital mirrors and other multi-sensor systems. And that solution is the FPGA.

In accordance with the UN Regulation No 158 concerning the approval of devices for motor vehicles, including digital reverse mirrors, there are three critical requests in the development of hardware and software components for ADAS camera-based platforms:

1. High speed of data processing, which affects safety and the speed of feedback to control systems.
    
2. Reconfiguration or the ability to change the system to meet new requirements without having to rebuild the entire model. This process increases system uptime without changing it from model to model.
    
3. Connectivity between several functional zones in the car.
    

Whatever the advantages of FPGAs, not all of them are suitable for automotive solutions. OEMs require that the electronic systems supplied meet other industry specifications in addition to the ISO 26262 functional safety standard. For example:

• IATF 16949 defines quality management system requirements for companies that build, manufacture and service automotive solutions;
   
• AEC-Q100 is a set of automotive industry standards that define the qualification requirements for integrated circuits (ICs) used in automotive applications.
   

FPGAs from Lattice fully meet these requirements; they are suitable for automotive systems where data processing speed and high performance are critical. FPGAs do not replace the MCU and AP but link them with auxiliary systems:

_{FPGA-based automotive applications. Source: latticesemi.com}

FPGAs act as a link between MCUs, APs and peripherals that enables seamless communication due to the following characteristics:

• Functional safety compliance: ASPICE CL2 and ASIL-B.
   
• Real-time processing of sensor and camera data.
   
• Parallel processing of multiple data streams.
   
• Flexibility and programmability: FPGAs allow rapid upgrades and enhancements, enabling new features and algorithms as technology evolves.
   
• Low power consumption, which is critical for electric vehicles.

FPGAs are suitable for the design of camera-based driver assistance systems due to their flexibility and other features. Such systems can perform the following functions:

• driver identification for access to vehicles;
   
• driver impairment detection;
   
• gaze capturing to prevent the driver from sleeping while driving;
   
• capturing distractions such as smoking and cell phone use.
   

In addition, cameras are the basis for digital car mirrors:

• smart rearview mirrors are an outgoing technology that represents the traditional rearview mirror with an integrated display;
   
• exterior digital rearview mirror: a camera located on the exterior rear of the vehicle that transmits the image to a monitor on the dashboard;
   
• bird-eye camera view: a four-camera system that provides an all-round view of the vehicle's position on the road.
   

To develop digital car mirrors, we use machine vision and other AI algorithms. With their help, the system can signal approaching objects, regulate speed, and help the driver stay in lane.

Major car manufacturers offer digital mirrors in some models. Here are examples from Lexus, Audi, and Honda:

_{Digital side-view mirror on the Lexus ES. Source: lexus.co.uk}

_{A virtual exterior mirror in the Audi Q8 e-tron model. Source: audi-mediacenter.com}

_{Honda’s side camera mirror system. Source: honda.co.uk}

Digital car mirrors are a system of cameras mounted in place of the traditional side mirrors and a rear-view camera located on the outer rear of modern vehicles. The resulting images are displayed in real time on digital displays, giving drivers an accurate view of the situation around their vehicle and in blind spots.

The digital display provides a clear view that is unobstructed by rear seat passengers, insufficient light or glare. And thanks to the FPGA at the heart of the system, the data is processed without delay.

One of the popular interfaces for video signal transmission in automotive applications is the flat panel display link (FPD-Link III) and Gigabit Multimedia Serial Link (GMSL). These interfaces do their job, but as they are based on SERDES chips produced by only a few manufacturers in the world, it limits the choice of electronic components for system design and leads to vendor lock-in with a single chip manufacturer.

Vendor lock-in goes against the trend towards disaggregation, which gives freedom of choice and compatibility of vehicle parts from different suppliers. And we at Promwad help our customers avoid this dependency and risks by developing FPGA-based solutions.

Our customer, a supplier of all-around vision systems for automobiles, was in a vendor lock-in situation. The components of their 360 vision system had been discontinued. The Promwad engineering team was tasked with finding a replacement component and updating the 360 vision system to meet ASPICE CL2 and ASIL-B safety standards.

We used a low-cost Crosslink-NX FPGA from Lattice and an S32K228 microcontroller from NXP as an alternative to the legacy components. The new solution supports MIPI CSI, MIPI DSI, CAN and USB interfaces.

_{Image by Promwad: video streams from cameras are combined into a single image}

The Lattice CrossLink-NX FPGA was selected due to three essential factors:

1. the ability to be used in vision and data processing applications;
    
2. support for the D-PHY (PHY) physical layer, which enables the MIPI CSI and DSI interfaces;
    
3. available IP cores for connecting and configuring cameras and displays.

_{CrossLink-NX VIP sensor input board. Source: latticesemi.com}

In this design project, it was also important to consider that FPGAs for ADAS systems must meet international quality and safety standards. Crosslink-NX is one of such FPGA platforms; it complies with:

• the AEC-Q100 standard for integrated systems in the automotive industry;
   
• the ISO/TS 16949 specification, which defines the requirements for a quality management system for the automotive supply chain.
   

As a result, our client received an all-around camera system in compliance with ASPICE CL2 and the ASIL-B safety level. And thanks to our partnership with Lattice, we provided our client’s engineers with full tech support and access to the necessary documentation.

***

FPGAs provide fast data processing, save main processor resources, and are programmable to meet system needs. The advantages of FPGAs make them a suitable solution for ADAS systems, including smart car mirrors that can improve visibility behind a vehicle.

Lattice FPGA-based development services are one of our key areas of expertise, and our portfolio includes many projects involving FPGAs for various industries. Feel free to contact us if your project needs any engineering support!