Structured Firmware QA for an Industrial Controller That Was Outgrowing Manual Testing

Project in a Nutshell: Promwad delivered a full automated QA framework for a European supplier of industrial embedded controllers used in high-speed rotating machinery. The framework replaces manual and ad-hoc checks with a two-layer test suite in Python and Robot Framework, running on a pool of physical devices inside an Azure DevOps CI/CD pipeline. Broader coverage, faster regression, and critical performance and networking defects intercepted before firmware reached the field

Client & Challenge

Our client develops industrial embedded controllers for high-speed rotating machinery used in critical production environments. As the product matured and firmware complexity grew, manual and semi-manual testing inside the development team stopped keeping up. On critical industrial equipment, that carries direct business risk: a firmware bug can become a plant-floor incident.

Hiring a full-time QA engineer for this one domain was not efficient, and general software testers rarely have hands-on experience with real hardware and industrial protocols on low-resource Linux targets. The client wanted an experienced embedded QA who could plug into their existing process and leave behind a reproducible framework the team could keep using.

Promwad's engineering team was already known to the client, and we matched the profile with an available embedded QA engineer familiar with the tools they had chosen.

Solution

Over the engagement, our engineer built a two-layer automated test framework for the controller's Linux-based firmware stack, integrated into the client's Azure DevOps CI/CD pipeline.

The QA system exercises the device from two directions:

• UI tests drive the web interface as a real operator would, validating end-to-end scenarios.

• Module tests exercise each microservice in isolation to verify its functional zone.

A load-testing block emulates production conditions, captures performance and network stability metrics, and runs automatic analysis of device logs for traces and anomalies. 

Two run configurations cover different needs:a smoke set for fast build validation, and a full regression run for deep pre-release checks. The set is chosen at run time.

Key Engineering Decisions

Automated device recovery via network-controlled relay. Physical test rigs occasionally went offline under stress or during specific failure scenarios, and manual reboots by on-site colleagues introduced delays into every CI run. We connected a network-controlled relay to the test devices and wrote a library that integrates it into the test framework. When a test loses contact with a device, the framework power-cycles it automatically and continues the run.

Physical test device pool with automatic selection. We built a pool of physical test units with automatic selection of a free device at run time. Multiple builds now run in parallel without conflicts or manual scheduling, removing a bottleneck that had been slowing release cadence.

Reusable Modbus and OPC UA library layer. Testing an industrial controller means interacting with hundreds of Modbus registers and OPC UA data points, each with its own meaning and safe usage rules. We built reusable Python libraries that encode the register-map logic once, instead of repeating it inside every test. The libraries were later adopted by the client's development team, cutting duplicate code across their own work.

Test grouping by microservice. The controller's firmware is composed of multiple microservices, each responsible for a functional zone. We structured the test suite to mirror this architecture. When a test fails, the responsible module is identified immediately, and the development team sees exactly what to look at.

Results & Business Value

Optimization of the slowest test paths cut roughly 30 minutes off the full regression run, even as the number of tests kept growing. Load testing then exposed what had previously slipped through: a class of network connectivity dropouts under heavy load, and Modbus register write failures under specific stress conditions. The client's development team classified both as critical and fixed them before firmware reached customer sites. A meaningful share of defects was caught even earlier, during test authoring itself, before those scenarios entered the regression suite.

The framework, libraries, and reporting are designed so that any team member can extend them with new scenarios without deep knowledge of the internals. The client's QA infrastructure is portable across future firmware releases and the products that share this platform.

More of What We Do for QA & Test Automation

• Software Testing Services — QA & Test Automation: Manual and automated testing across mobile, web, desktop, and embedded platforms, with CI/CD integration and hardware-in-the-loop workflows.  

• Embedded Software for Industrial Automation: Firmware development for an industrial gateway module implementing Modbus RTU and HART — an example of the embedded stack our QA practice tests end-to-end.  

• Automated Testing for Embedded Software: CI/CD Pipelines, Simulators, and Hardware-in-the-Loop: How to structure an embedded QA pipeline that combines unit tests, HIL rigs, and CI/CD integration