AI-Powered Assistance System for Construction Cranes Sensor fusion of LiDAR and camera to improve operation efficiency, accuracy and safety
Sensor fusion: LiDAR + Video Camera - Laser beams measure distances to create detailed 3D maps - A camera captures visual information and provides additional details
The system detects nearby objects with colors ranging from pink to purple and distant objects with colors ranging from green to yellow. Colour differntiation The system detects nearby objects with colors ranging from pink to purple and distant objects with colors ranging from green to yellow
AI-Powered Assistance System for Construction Cranes Sensor fusion of LiDAR and camera to improve operation efficiency, accuracy and safety
Sensor fusion: LiDAR + Video Camera - Laser beams measure distances to create detailed 3D maps - A camera captures visual information and provides additional details
The system detects nearby objects with colors ranging from pink to purple and distant objects with colors ranging from green to yellow. Colour differntiation The system detects nearby objects with colors ranging from pink to purple and distant objects with colors ranging from green to yellow

LiDAR-Camera Fusion Assistance Systems for Crane Operators

Client

A European integrator that supplies cranes for construction sites and cargo ports.

 

Challenge

To develop an AI-based engineering solution to improve crane operation efficiency, accuracy and safety.

 

Solution

1. Concept Design

To meet this challenge, we decided to develop a LiDAR-camera fusion assistance system for crane operators combining the capabilities of LiDAR sensors and video cameras.

LiDAR technology (light detection and ranging) uses laser beams to measure distances and create detailed 3D maps of the surroundings. Camera technology complements LiDAR by capturing visual information and providing additional contextual information, such as colour, texture, and fine details.​

 

Instant AI-powered real-time alerts for crane operators for their quick response

Instant AI-powered real-time alerts for crane operators for their quick response

 

2. System Design and Its Benefits

We have developed a prototype of the LiDAR-camera fusion assistance system, which will enable new features and benefits:

 

1. Improved safety and risk mitigation by providing comprehensive situational awareness and detecting potential hazards:​

  • Obstacle detection helps detect nearby structures, equipment, or personnel that pose risks during crane operations.
  • Collision avoidance by providing instant alerts to the operator whenever the system detects an imminent collision with an obstacle, such as a structure or a person.​
  • Real-time warnings and proximity monitoring are integrated into the operator's interface, ensuring prompt response.
  • Adverse condition awareness, even in challenging weather or lighting conditions, reduces the risk of accidents caused by lack of visibility.​

 

A system prototype with a camera on the crane operator's cab to detect unauthorised objects

A system prototype with a camera on the crane operator's cab to deteсt unauthorised objects

 

2. Enhanced efficiency and productivity

LiDAR-camera fusion systems reduce human error and streamline operations by taking advantage of data-driven decision-making:​

  • Automated object recognition. Advanced algorithms allow operators to quickly identify structures, equipment, or objects that may impact crane movements.
  • Precise positioning assistance in real-time by leveraging the accurate distance measurements from LiDAR and camera visual feedback. ​
  • Virtual overlays and augmented reality. It includes distance markers, load paths, safety zones, or other relevant data.
This compact LiDAR can be mounted on a crane arm to provide a detailed 3D site map

This compact LiDAR can be mounted on a crane arm to provide a detailed 3D site map

 

3. Real-time data and analytics

The LiDAR-camera fusion system enables monitoring, performance analysis, and operational insights:​

  • Real-time data monitoring with constant data collection on the environment and crane operations for alerts and statistics.
  • Performance metrics and analysis: load positioning accuracy, time taken for specific tasks, or frequency of obstacles encountered.This data helps improve operational efficiency, identify potential bottlenecks and make data-driven decisions.​
  • Workflow optimisation. Insights gained from the sensor fusion system lead to improved task sequencing and better coordination with other activities on the worksite.​
  • Operator behaviour analysis. A system can be equipped with an additional camera looking at the operator to analyse their behaviour, including distraction, bad habits (smoking, phone), and emotional state.

 

Business Value

Our client has received a LiDAR-camera fusion assistance system prototype for crane operators on construction sites and cargo ports.

Our software engineers analysed the user interface of crane operators to seamlessly integrate our platform with real-time alerts, video streaming with an augmented reality layer, and analytics for more efficient operation.

Integrating LiDAR-camera fusion assistance systems can provide a competitive edge and further develop the new system through machine learning and optimising algorithms for data-driven decisions. This technology results in lower costs and a significant increase in safety.

Here is an overview of the features and business value of the new system:

FeatureBenefits
Safety
Detect people in hazardous areas and measure distance to themAvoid injuries and ensures labour safety, reducing the impact of human error
Deteсt obstacles and measure their position, orientation, dimension and the distance to themAvoid collisions of crane's jib and hook with obstacles, like walls, power lines, cars, humans etc.
Night visionProvides safety at night
Construction Time & Costs Minimisation
Detect a load type, and measure its dimension and 3D position

− Detect out-of-reach criteria

− Plan manoeuvre to go from A to B

− Save time for unnecessary hoist

Operator Assistance with Augmented Reality
Draw a 3D scene from various cranes and construction site viewsImprove operator efficiency and safety

− Draw crane projection onto the ground and objects there

− Highlight collision areas and people

Improve operation safety
Autopilot functionPlan route and optimise the manoeuvre time and safety

 

Other Case Studies

Tell us about your project!

All submitted information will be kept confidential.