Why RTOS and Linux AMP Systems Fail at Boot, Reboot, and Recovery
Quick Overview
Problem: An AMP system that runs correctly once both sides are up loses its real-time core, its I/O, or its IPC channel at the edges of its life: after boot, during a Linux reboot or update, or after a crash.
Common failure points: Linux switching off clocks and supplies it does not know the RTOS uses; an unplanned handoff between bootloader and remoteproc; RPMsg channels that never come back after one side restarts; RTOS and Linux images updated out of step.
Where it appears: Industrial controllers and drives with a Linux HMI, robotics controllers, test instruments, gateways with real-time I/O, off-road and vehicle ECUs built on Cortex-A + Cortex-M/R parts.
Engineering focus: Resource ownership that survives late init, an explicit boot and recovery contract, versioned updates across both images, and fault-injection testing of every lifecycle event.
Production Failure Scenario
Wrong Assumption
The assumption is that once each core has its own operating system, each side has its own life: the RTOS starts, Linux starts, RPMsg connects them, done. That holds in steady state. It does not hold at transitions. On a heterogeneous SoC both sides still share the clock tree, power domains, reset logic, and usually the storage the RTOS firmware is loaded from — and Linux, by default, manages all of them as if it were alone. Until resource ownership, boot handoff, restart behavior, and versioning are designed explicitly, the two lifecycles stay coupled in ways the bench never shows.
Why It Fails
Linux reclaims what it does not know about. At late init, the Linux common clock framework switches off clocks that no Linux driver has claimed, and the regulator core does the same for unclaimed supplies, typically about 30 seconds later. Pin controllers and power domains follow similar logic. A timer, an ADC, or a sensor supply used only by the RTOS is “unused” from Linux’s point of view unless the device tree says otherwise. The kernel parameter clk_ignore_unused hides the problem on development images and leaves it in production. Hardware isolation controllers (ETZPC on STM32MP1, RIFSC on STM32MP2, RDC on i.MX 8M, TRDC on i.MX 93) can stop Linux from touching an RTOS peripheral, but they enforce a map — they do not write it. That map is Linux kernel and BSP engineering work, not an application task.
The boot handoff is a design decision, not a default. Products that must drive outputs to a safe state within milliseconds often start the RTOS from the bootloader, long before Linux. Whether Linux then attaches to the running core or stops and reloads it depends on the vendor remoteproc driver, the BSP, and how the bootloader hands over state. Some vendors document automatic attach — ST, for example, describes Linux attaching to a Cortex-M already started by the bootloader. When the handoff is not configured end to end (device-tree properties, the resource table where Linux expects it, a kernel with attach support for that driver), a reset and reload of the real-time core is one of the possible outcomes. It is a risk to rule out on the target, not a universal remoteproc behavior.
Restart is where IPC actually breaks. RPMsg channels are created when the RTOS announces its endpoints after the virtio link comes up. If Linux reboots while the RTOS keeps running, the link is torn down and rebuilt from the Linux side. Unless the RTOS firmware detects that reset, reinitializes its side, and announces its endpoints again, the channel never returns — the control core runs, and Linux cannot see it. The reverse case matters just as much: after an RTOS crash, Linux user space has to notice that its endpoint is gone and wait for a new one rather than writing into nothing.
Two images, one product, independent versions. The RTOS firmware often ships inside the Linux root filesystem under /lib/firmware, while the RTOS itself may be started by the bootloader from a different copy. An update that changes the message format on one side keeps the same endpoint name and passes every connectivity check — and then delivers garbage. The same drift can hit the memory map: the reserved-memory carveout in the device tree, the RTOS linker script, and the resource table must agree, and an update that changes one of them silently corrupts shared data. A firmware update strategy for mission-critical devices has to treat both images as one versioned unit, or the channel has to negotiate a protocol version when it opens. When both images are signed, the secure boot chain has to cover the RTOS firmware too.
These failures rarely show up together, and that is why they survive testing. Each one needs a specific transition — a slow boot, a reboot during motion, an update that touches only one image — and a bench test that powers the board on and runs it never produces any of them. Steady-state timing under Linux load is a different failure class, covered in why mixed RTOS and Linux architectures break determinism under real industrial load.
Hidden System Complexity
power-on → boot ROM → TF-A / OP-TEE → U-Boot (optional early start of the RTOS core) → Linux kernel → late init (unused clocks and supplies off) → remoteproc (attach or load) → resource table → virtio / vrings → RPMsg name service → rpmsg_char / rpmsg_tty → user-space daemon → application … → Linux reboot / update / panic → and back again
A steady-state test exercises the middle of that chain once. Production exercises the ends of it thousands of times. Every arrow is a handoff where one side assumes something about the other: the RTOS assumes its clocks stay on; Linux assumes anything it did not claim can be turned off. Linux assumes it starts the remote core; the bootloader already did. The daemon assumes a channel exists; the RTOS announces it only after it initializes — and, after a Linux restart, only if it was written to announce it again.
Failure Patterns
Scenario 1. Sensor feedback on the real-time core drops to zero roughly 30 seconds after boot, on production images only. The analog front-end supply comes from a PMIC regulator that no Linux driver claims, so the regulator core disables it during late cleanup. Development images survived because an older device tree marked that regulator always-on.
Scenario 2. Linux reboots after an OTA update while the drive is running. The RTOS keeps control, as designed, but the HMI shows the controller offline until the whole unit is power-cycled. The RTOS firmware never handled a virtio reset from the Linux side, so it never re-announced its RPMsg endpoints.
Scenario 3. After an update, some units show nonsense values on the HMI. The new root filesystem carried a newer RTOS firmware with a changed message structure, but units whose RTOS was started from the bootloader’s copy kept running the old one. Neither side checked a protocol version, and the endpoint name was the same.
Embedded Linux and RTOS Integration Engineering
RTOS + Linux AMP failures at boot, reboot, and update are rarely RTOS bugs or kernel bugs. They come from a lifecycle contract nobody owned: who holds which resource, who starts which core, what each side does when the other restarts, and how two images stay in step. Promwad develops MCU firmware on Zephyr and other RTOSes or bare metal, Linux kernels, BSPs, and drivers, and integrates both sides on heterogeneous SoCs from NXP, ST, TI, and AMD.
Engineering Experience Across Heterogeneous SoC Platforms
A Composite Scenario: The Drive That Stayed Safe and Still Lost Its HMI
Illustrative composite, not a single client project. It combines lifecycle failures that recur on i.MX 8M-, STM32MP- and AM6x-class designs.
A drive controller runs its current and position loops on the Cortex-M core under an RTOS. Linux on the Cortex-A cores runs a Qt HMI, logging, and OPC UA. To meet the safe-state requirement, U-Boot starts the RTOS before Linux. Each side was validated on its own, and steady-state behavior was clean.
The transitions told another story. Because the remoteproc configuration did not match the early start, Linux reloaded the firmware on boot, and outputs briefly returned to their reset state. Once that was fixed with attach mode, a Linux reboot during an update left the RTOS running but invisible to the HMI, because the firmware never re-announced its endpoints. And after the next update, the HMI and the RTOS disagreed about the message format, because the rootfs and bootloader copies of the RTOS firmware had drifted apart.
The fix in a case like this is a lifecycle pass, not a rewrite: an attach-mode handoff verified on the target, virtio-reset handling and endpoint re-announcement in the RTOS, one versioned firmware bundle with a protocol handshake, and a fault-injection test matrix for every transition. The control algorithm does not change.
Solution Approach
Step 1: Make resource ownership survive late init. List every clock, regulator, pin, power domain, interrupt, DMA channel, and memory region the RTOS depends on, and give each one an owner. Encode it in the Linux device tree (reserved-memory carveouts, RTOS peripherals disabled for Linux, RTOS clocks and supplies claimed by the remoteproc node or protected in their provider) and in the SoC isolation controller. Then boot the production image without clk_ignore_unused and leave it running well past the regulator cleanup window. If anything stops, the map is incomplete. This is the center of Linux and Android kernel engineering on AMP platforms.
Step 2: Write the lifecycle contract. Decide who starts the real-time core (boot ROM, bootloader, or Linux) and whether Linux attaches or loads. Then define the behavior for each event: Linux reboot, suspend and resume, kernel panic, RTOS crash, and an update of either image. Where Linux supervises the RTOS, use remoteproc recovery and coredump. Where the RTOS must outlive Linux, give it its own reset domain and an independent hardware watchdog. The MCU firmware side — RTOS and bare-metal — has to implement the matching states: safe outputs, a degraded mode, virtio-reset handling, and endpoint re-announcement.
Step 3: Version the pair and test every transition. Ship RTOS and Linux images as one versioned unit, or add a protocol-version handshake when the RPMsg channel opens — preferably both. Then build a fault-injection matrix: Linux reboot under motion, kernel panic, suspend and resume, RTOS crash and watchdog reset, power loss during an update of each image. Run each transition repeatedly on target hardware, record every failure to reach a safe state or to restore the channel, and set pass criteria in advance: outputs stay safe, the channel returns within a defined time, versions match. Steady-state timing is a separate acceptance item. For hard real-time paths it needs an argued worst-case bound, with maximum observed latency and deadline misses recorded over long runs; a percentile alone is not proof. That validation discipline is covered in the industrial AMP determinism article and in QA and test automation.
An AMP system that has only been powered on and left running has not been tested at the points where it fails. Its risks sit at transitions, and transitions have to be produced on purpose.
Real Trade-Offs
RTOS started by the bootloader vs. by Linux. An early start gives a safe state within milliseconds and control that does not depend on Linux. It also requires a verified attach handoff and a separate path for updating the bootloader’s firmware copy. Letting remoteproc load the RTOS keeps updates simple, but the real-time core then starts only after Linux and shares Linux’s fate on every reboot.
Linux-supervised recovery vs. an independent safety path. remoteproc recovery and coredump make RTOS crashes easy to diagnose and restart. They also make Linux part of the recovery path. Where the RTOS must reach a safe state without Linux, an independent watchdog and reset domain are required. Where the safety argument does not tolerate shared silicon at all, a physically separate MCU — as in a dual-MCU railway BMU architecture — is the cleaner answer.
Attach on Linux restart vs. clean reload. Keeping the RTOS running through a Linux reboot preserves control continuity, but the RTOS has to handle a virtio reset correctly and carry its state across. Reloading it gives a known clean state, at the cost of a control gap on every Linux restart.
One firmware bundle vs. independent updates. A single versioned bundle removes version drift, but every RTOS fix becomes a full system update. Independent updates are lighter but demand version negotiation and backward-compatible messages.
RTOS firmware in the rootfs vs. its own partition. Storing it in /lib/firmware ties it to the Linux A/B update scheme. A dedicated partition read by the bootloader supports an early start and an independent rollback, but adds a second update and verification path.
Typical AMP Lifecycle Engineering Tasks
Late-Init Resource Audit
Mapping every clock, supply, pin, and power domain the RTOS uses, and proving the production image runs without clk_ignore_unused or always-on workarounds.
Restart-Safe IPC and Versioning
Handling virtio resets, endpoint re-announcement, protocol-version handshakes, and coordinated firmware updates across both images.
Boot Handoff and remoteproc Setup
Configuring early RTOS startup, remoteproc attach or load, reset-domain separation, recovery, and coredump handling.
Lifecycle Fault-Injection Testing
Testing Linux reboots, panics, suspend/resume, RTOS crashes, and interrupted updates on hardware against defined pass criteria.
Qualifying Symptoms
- The real-time core works with clk_ignore_unused or a debug device tree and fails in the production image.
- A peripheral or sensor supply used by the RTOS goes dead a fixed time after boot.
- Rebooting or updating Linux restarts the RTOS core or drives outputs to their reset state.
- After a Linux reboot the RTOS keeps running, but Linux can no longer talk to it.
- Linux applications keep writing to an RPMsg endpoint after the RTOS has crashed and restarted.
- After an update, the HMI and the RTOS disagree about data formats on some units.
- Nobody on the team can say what the RTOS does if Linux panics.
RTOS and Linux Integration Engineering
At this stage the work is lifecycle engineering across both operating systems, not tuning either one. In practice: an ownership map that survives late init, a written boot and recovery contract, one versioned pair of images, and a fault-injection matrix that produces every transition on purpose.
The RTOS side lives in MCU firmware: RTOS and bare-metal and, where Zephyr is the choice, in Zephyr RTOS-based firmware, which includes OpenAMP support for this kind of split. The Linux side — device tree, remoteproc, drivers — is Linux kernel engineering. Steady-state behavior under load, including memory contention and the IPC coherency model, is covered in why mixed RTOS and Linux architectures break determinism under real industrial load. Where a safety standard applies, the restart and recovery behavior becomes part of the IEC 61508 safety software argument.
Relevant Promwad Engineering Experience
This class of problem shows up most when a Linux HMI or connectivity stack is added to a product whose real-time behavior used to live on a separate MCU — industrial controllers and drives, robotics, test instruments, off-road vehicle electronics — and the boot and update path is scoped for Linux alone. Promwad’s reusable robotics software platform draws the same line across two chips: EtherCAT motion control on a Hilscher netX90, ROS 2 on a Yocto-based Linux image on Qualcomm RB3.
FAQ
What is the difference between AMP and SMP in embedded systems?
Will Linux remoteproc restart an RTOS core that the bootloader already started?
Can the RTOS keep running if Linux crashes or reboots?
How should RTOS firmware be updated together with Linux?
Which SoCs support RTOS and Linux in an AMP configuration?
Can you join an AMP integration that is already in trouble?
Related Engineering Cases
- Reusable Robotics Software Platform: EtherCAT motion control on Hilscher netX90 with ROS 2 on Yocto Linux (Qualcomm RB3): deterministic control and Linux services kept in separate domains.
- Dual-MCU Railway BMU Architecture: Primary control MCU plus an independent safety MCU for monitoring and watchdog supervision, with a SIL2 path.
- Seed Drill Control System for Precision Farming: Real-time ECU control with a separate i.MX 8M Linux/Qt HMI.