Industrial Cameras with AI: Сase Study on Quality Analytics
In this article, we will explain how Industrial IoT + AI can automate traditional industries such as retail, agriculture, woodworking, logistics, construction, manufacturing, and many other types of businesses that don’t usually come to mind when you think of tech. We will take a look at one of these projects from the inside so that you can see what problems can be solved and what results can be achieved.
IIoT and Industry 4.0 are not just buzzwords, but the working tools that change the approach to traditional production processes, improve business efficiency, and raise the level of business intelligence for the client. However, not all business owners and managers are yet aware of the real possibilities that new technologies can offer. We’ve decided to remedy this situation by using real-life examples to explain how artificial intelligence and IoT technologies (AIoT) can work in real enterprises.
Contents
Overview of the Stages in an Automation Project
The starting point for any automation project can be determined after an analysis of the company's business processes is performed, but it would be even better if the client of such a project voices the problems from the start. For example: "I'm losing money because I pay employees to perform simple monotonous tasks, as well as on the delivery costs to get the products to my end customers. I want to automate one of these processes.” This would be a great start for the first meeting and gives both parties something to think about. :-)
To find a solution, we bring in a business analyst and a systems analyst. They gather requirements and deal with the client's "pain point" by asking clarifying questions
Our experience shows that getting analysts involved from the first meeting changes our clients’ attitude. They start to look at their business from a new perspective and can even offer solutions themselves. Then we take these proposals and transform them into genuine high-level technological solutions, starting with preparations for a concrete implementation plan.
The entire automation project usually involves the following steps: 
Case Study: Automation in Woodworking
Let’s take an example from our automation practice. In this case, the head of a woodworking company was concerned about a defect in the production process, namely, the periodic deformation of a certain type of wooden beam, the quality of which was monitored by a special employee. We proposed to solve this problem by implementing machine learning and computer vision algorithms to detect such defects automatically.
How does this work? At a certain point of the production process, we place a special device with a visual sensor that captures the process, uses a special software algorithm to analyze the video, and then automatically recognizes the type of beam profile. If the sample is warped or otherwise deformed, the device sends a signal to the flipping and sorting mechanisms.
Pictured: example of an incorrect and a correct beam profile
Aside from the new flipping mechanism, the automation project also features the recording of each instance in the database or CRM, which tallies how many problems were observed during the working day, in what section of the process, and at what time. With this new addition to the process, the client is now able to improve the financial performance of the business and can calculate the financial benefits of the automation. Below, we break down this case step by step:
- First, let's take a look at how things worked "the old-fashioned way" with the implementation of manual labor.
- We offer three technical automation solutions and choose the best one.
- We explain how computer vision systems are developed in a way that nontechnical readers can understand. :-)
- We see the results of the project on the automated conveyor line.
- We’ll touch on the consequences of automation, specifically, the emergence of tech support and possible layoffs.
1. Analyzing the way things worked before automation
Pictured: one of the processing line sections at a woodworking plant
At each of our client's eight production areas, there was a section where two people were working. They visually assessed each beam to identify deformed samples and turned them over either by hand or by pressing a button on a rotary mechanism.
So, it’s safe to say that a reasonable man can make better use of his abilities. This realization helps convince clients that automating routine manual labor and the associated retraining or even layoffs is a necessary measure to further grow both the business and the people working in it.
By the way, if you asked the question "How and why does a warped beam get on the processing line" while you were reading this article, we will answer it in the words of the client: "This happens to even the highest-quality wood dues to the changes in humidity and temperature". This information can be saved for the second stage of automation, where we can install a network of temperature and humidity sensors that monitor the storage and transportation conditions of the products. In the meantime, let's take a look at the technical solutions for the task at hand.
2. Finding technical automation solutions
Generally, we develop multiple engineering solutions to the problem at the design stage. The freedom to choose reduces errors and gives the client flexibility. When implementing one solution, individual functions can be taken from the others and implemented as well.
Here are the technical solutions that would be relevant to our woodworking case study:
Solution 1. Computer vision and machine-learning algorithms that detect visual defects from the video camera data.
Solution 2. Laser rangefinders that determine profile curvature utilizing shape estimation and defect analysis.
If you take a close look at the diagram, you will be able to see the main disadvantage of this solution: if the workpiece is placed unevenly, the sensor “B” will not be activated and this will lead to false detection of a defective beam.
Solution 3. Computer vision with the analysis of the beams’ faces from a single photo camera image: if the geometry differs from the norm to a certain extent, the sample is considered defective.
Below are examples of edge detection with computer vision. Here is an image of St. Paul's Cathedral in London processed with the PST technology:
An example of edge detection in complex environments. Source: Uqam, Wikipedia, CC BY-SA 4.0
An example of edge detection in simple environments. Sources: a still image from the video Computer Vision: Edge detection using canny in OpenCV & Python
You can see from the photos that it would be technically difficult for us to achieve a perfect result in an actual production environment. The solution will only work correctly in "laboratory conditions” where there’s a monochrome background and an absence of visual noise.
This is the format we use to work through each solution: assess the complexity and cost of implementation, the benefits, and risks, and then select the final option together with the client.
3. Development of a computer vision system (explanation for non-technical people)
For rapid prototyping, we use TensorFlow, which is a free open-source machine learning library developed by Google. It allows users to create and train neural networks for automatic pattern recognition. The margin of error for such networks approaches that of human perception.
The following shows how computer vision works. A neural network predicts what a person’s gesture in the video most closely resembles: 
Source: a still image from a YouTube presentation of the Lobe service, which promises to make machine learning accessible to everyone.
In our woodworking case study, we used TensorFlow Light, which is a special library for working with “light” machine learning models. It allows users to create optimized neural network models that work on low-power platforms and mobile devices. This means that we can implement local data processing without necessarily being connected to the Internet (this approach is called edge computing).
Our solution runs on a credit-card-sized Raspberry Pi single-board computer and a special camera for video analytics and neural network processing – all enclosed in a dust and moisture-proof case:
Pictured: assembling a solution using off-the-shelf components. Source: Newark and Luxonis
To train the neural network, we perform data mining at all of the client's production areas: video and photos of differently-shaped wood profiles. The first step is to load all the different profiles without categorizing them. This is our data set. Next, the ML engineer comes in to perform the data partitioning. His or her task is to classify the collected data, i.e. to select correct and incorrect shapes of profiles, using given samples as a basis. This data classification process is one of the first and vital stages of development. In other projects, the process isn’t limited to images but is also performed with sounds, spreadsheets, text, and numbers.
The ML engineer then makes the first predictions and adjusts the data as needed. His or her task is to "teach" the AI algorithms to correctly recognize the type of object (correct/incorrect) in different lighting conditions, from different perspectives, and when foreign objects appear in the frame. The more useful data there is and the more diverse it is, the "smarter" the network becomes, so the ML-engineer increases the data set during the augmentation process:
Examples of different data augmentation methods. Source: https://github.com/aleju/imgaug
In our case, we ran the TensorFlow framework on Amazon's cloud platform, AWS EC2, and used these tools to implement the training and object classification phase.
Then, when we receive an image via the camera, we give it to the already trained neural network, and it determines the correctness or incorrectness of the beam profile. As a result, the rotary mechanism on the conveyor is only activated when a deformed beam is detected, and the necessary record is subsequently made in the database. And all this happens without any human involvement. With such automation technologies, it is possible to achieve an accuracy rate of 95%.
4. How the automated conveyor line works
Our client spent time and money to ensure the standards for occupational safety and health and the enterprise set aside a budget to pay for the salaries of 16 people. After the automation process is implemented, these costs can be reduced.
Pictured: the enterprise’s accounting system
The client also gathers statistics data as part of a single accounting system. As a result, he can monitor the quality of the products remotely, see their dynamics (improvement/reduction) and intervene if necessary. Thus, the company can work more effectively with suppliers since its manager can guarantee that when a shipment of beams is purchased, no more than 5% of them will be defective. This also allows for the prediction of the return on recycled wood that was rejected in the initial selection. The client can see how much such defective product has accumulated in real time and can schedule a shipment at a lower price. This is how different client segments are formed. The additional knowledge lets the client confidently plan the financial performance of the company and improve the efficiency of production.
What's more, the automated system has another incredibly valuable quality – scalability. Once the system is implemented and debugged at one site, it can easily be installed at other sites in the same way.
5. Consequences of automation: tech support and layoffs
It must be noted that progress does not come without consequences:
- The newly-implemented automation will require a technical support service in the form of an in-house specialist or "on-demand" service. However, in our woodworking example, it would be one or two specialists instead of 16 employees.
- The managers of the business will have to take responsibility for downsizing. Of course, this is not an easy thing to do for everyone involved, but our experience shows that managers often try to retain valuable talent by switching them over to other processes. Thus, the employees are retrained and perform more interesting and complex tasks that cannot be automated.
Conclusion
Now that you’ve taken a look behind the scenes of our engineering process, you should have a better understanding of the potential implementation options of artificial intelligence in other types of business as well. You may already have some ideas or questions for your particular industry.
Of course, a single article (or even a few) that features a real-life example is not nearly sufficient to make a definitive decision on whether to launch an automation project in your particular field, so we will conclude our report by turning to international experience.
According to a recent report done by German research company IoT Analytics, the number of business and manufacturing process automation projects based on IIoT continues to grow, with over 1,600 IoT projects that automate 48 different business processes already implemented over the past few years. The ten most popular ones are shown in the diagram below:
Source: https://iot-analytics.com/top-10-iot-use-cases/
In 2021, the average company in the field of large manufacturing, medical, automotive, retail, or energy has implemented eight different IoT projects. Most implementations were realized in enterprise accounting and asset tracking, followed by process automation.
It makes a lot of sense that these directions ended up being the most in-demand since the principle that guides many successful businessmen is “If you fail, do it over. If you succeed, scale it up.” We can attest to this, as our experience shows that in automation projects, economies of scale allow clients to quickly recoup even the most expensive projects. The cost of implementing each subsequent solution is negligible, but the savings are enormous.
We would like to thank you for reading! If you still have questions or ideas after looking through our case study, please feel free to contact us.