AI-Assisted Hardware-in-the-Loop for Embedded Linux

AI coding assistants are genuinely useful for embedded Linux work -- navigating unfamiliar build systems, writing device tree overlays, debugging kernel configs. But they hit a wall the moment physical hardware is involved. The agent can modify source files and run builds, but it cannot flash the board, watch it boot, or read back what happened.

This post covers a practical approach to closing that gap: giving an AI agent direct access to development hardware through a controlled interface, turning it into a proper hardware-in-the-loop participant.

The project

We are porting the Luckfox Pico Ultra SDK from Buildroot to Yocto. The stock Luckfox SDK is Buildroot with a substantial shell script wrapping the entire build and flash process. Yocto gives us reproducible builds, layer-based customisation, and a package feed -- but the porting process involves a lot of iteration. Build a rootfs, flash it, check if it boots, examine what went wrong, adjust, repeat.

The AI agent (Claude Code) runs inside a VirtualBox VM. It can read the SDK sources, modify recipes, and kick off builds. But it has no way to flash the result onto the Luckfox board connected via USB to the host machine. USB passthrough to VMs is unreliable for Rockchip devices -- the chip re-enumerates during the flash process, which breaks the passthrough session.

What does pass through reliably to the VM is a USB serial adapter and a small USB relay board. The agent uses the serial adapter to read console output directly, and the relay board to power-cycle the target and place it into flash mode -- no human hands required. The only piece that cannot live inside the VM is the Rockchip flash tool itself, because of the re-enumeration problem.

The approach

Rather than fighting USB passthrough, we built a small MCP server (gatecmd) that runs on the host and exposes whitelisted commands to the agent over HTTP. The agent connects from inside the VM and can:

• List connected Rockchip devices (upgrade_tool ld)

• Flash loaders, firmware, and partition images

• Read device and storage information

• Manage files in a shared staging directory

Every command is validated against an allowlist with pattern-matched arguments. The agent cannot execute arbitrary commands -- only pre-defined operations with pre-defined argument structures. File paths are restricted to a single shared directory, so the agent cannot read or write arbitrary host files.

Example config:

file_root: "/home/user/vm-shared"

commands:

  - name: upgrade_tool

    binary: /opt/upgrade_tool_v2.17/upgrade_tool

    description: "Rockchip firmware development tool"

    allowed_args:

      - pattern: "ld"

      - pattern: "db {file}"

        file_args: ["file"]

      - pattern: "ul {file}"

        file_args: ["file"]

      - pattern: "uf {file}"

        file_args: ["file"]

      - pattern: "wl {offset} {file}"

        file_args: ["file"]

    timeout_secs: 120

The workflow

The development loop now looks like this:

1. The agent modifies a Yocto recipe or kernel config inside the VM

2. It runs the build

3. It copies the resulting image to the shared folder

4. It triggers the relay to power-cycle the board into flash mode

5. It calls upgrade_tool through the MCP server to flash the board

6. It triggers the relay again to boot normally

7. It reads the serial console to watch the boot process

8. It diagnoses what went wrong and goes back to step 1

The agent handles the entire cycle autonomously -- including the physical steps of power-cycling and entering flash mode. It can attempt a flash, see it fail, check device state, read the kernel panic on the console, and adjust its approach. The same iterative process a developer would follow at a bench, but without needing someone sitting there.

What this actually gives you

The value is not that the AI does anything a developer could not do manually. It is that the feedback loop runs without human intervention. Porting an SDK to Yocto involves a large number of small iterations -- a missing kernel module, a wrong device tree property, a rootfs that mounts but cannot find init. Each cycle is straightforward to diagnose but tedious to execute manually.

Giving the agent hardware access turns these from "wait for a developer to sit down and flash the board" into background tasks that progress on their own. The developer reviews results and makes architectural decisions rather than babysitting the flash-boot-debug loop.

Security considerations

Letting an AI agent run commands on your host machine is exactly as dangerous as it sounds. The security model here is deliberately restrictive:

• Allowlist-only execution -- only commands explicitly defined in the YAML config can run

• Pattern-matched arguments -- the agent cannot inject arbitrary flags or subcommands

• No shell -- commands execute directly, never through sh -c

• Scoped file access -- all file paths must resolve under a single configured directory

• Bearer token auth -- the MCP endpoint requires a shared secret

This is not a general-purpose remote execution framework. It is a narrow interface designed around one specific workflow: an AI agent building and flashing embedded firmware.

Practical notes

Rockchip USB re-enumeration: The Rockchip bootrom, loader, and firmware modes all present as different USB devices. During a flash sequence, the device disappears and reappears on the bus. This is why VM USB passthrough fails -- the host reassigns the device. Running the flash tool on the host avoids this entirely.

Shared folder timing: VirtualBox shared folders can have stale caches. The built-in file management tools (list_files, copy_file) operate on the host side directly, so the agent can verify what is actually present before flashing.

Serial and relay passthrough: Unlike the Rockchip flash interface, simple USB devices like serial adapters and relay boards pass through to a VM without issue -- they do not re-enumerate during use. The agent accesses these directly from inside the VM, so no MCP proxy is needed for power control or console access. This keeps the trust boundary clean: only the flash tool runs on the host.

Timeout management: Flashing a full firmware image can take over a minute. Per-command timeouts in the config prevent hung flash operations from blocking the agent indefinitely.

Source

The MCP server (gatecmd) is written in Rust and available on GitHub. It includes the Rockchip upgrade_tool v2.17 binary and an English translation of the original Chinese user guide.