AI PLC test matrixAI SEO formatted

Make the AI Agent Write the PLC Test Matrix Before It Writes Control Code

Your first PLC AI proof of concept should not be code generation. Test whether the agent can turn one frozen routine and approved requirements into a reviewable matrix that exposes missing cases before FAT.

August 4, 20266 min readManufacturing AI Workflows
A frozen PLC project and approved cause-and-effect sheet feed an engineer-reviewed test matrix that runs in an offline PLC emulator.

Key takeaways

  • A test matrix is easier to review than AI-generated control code and tests whether the agent understands the expected behavior.
  • Start with one non-safety equipment module and an immutable project snapshot.
  • Expected results must come from approved requirements, not from the controller code alone.
  • Execute only engineer-approved rows with a deterministic runner in an offline emulator or isolated bench.
  • Logical emulator evidence never proves physical I/O, wiring, mechanics, process dynamics, or safety performance.
01Better first deliverable

Ask for a test matrix before control code

Your first PLC AI proof of concept should not be code generation. Give the agent one exported routine, one approved cause-and-effect sheet, and an offline emulator. Then measure whether its reviewed test matrix exposes missing cases before FAT.

Most PLC AI pilots start with the most expensive question: can the model write ladder logic or structured text? That creates a review problem before the team has proved that the agent understands the machine's expected behavior.

The agent reads one frozen controller export, maps interlock dependencies, and drafts the cases. A controls engineer approves every expected result. A deterministic runner executes only those approved rows in an offline emulator or isolated test bench.

02Bounded scope

Start with one non-safety module

Choose one pump skid, conveyor section, or equipment module with roughly 15 to 30 ordinary-control interlocks. Exclude E-stops, guards, burner management, safety PLC logic, SIS functions, and anything covered by the site's functional-safety lifecycle.

Do not give the agent a production endpoint, credentials, download permission, online-edit access, or force capability. If two documents disagree, the agent records the conflict instead of guessing.

  • Immutable controller export or project snapshot
  • Approved cause-and-effect sheet, interlock list, or control narrative
  • I/O list plus alarm, reset, and recovery requirements
  • Controller family, firmware, runtime, and software versions
  • A hash for every artifact
03Test contract

Make every row reviewable

Each proposed case needs a deterministic starting state, a traceable requirement, an input transition, an expected result, a timing window, reset behavior, and evidence to retain.

The tag names in the example are placeholders. The expected result and time window must come from an approved requirement.

TC-017 | C&E-12 | Auto selected, stopped, permissives healthy | Ready_FB changes 1 to 0 while RunReq stays 1 | RunCmd drops inside the approved window, trip latches, auto restart stays blocked | restore Ready_FB, apply approved reset | save trace and hashes
  • Test ID and requirement reference
  • Deterministic starting state
  • Input transition or sequence
  • Expected output and internal state
  • Approved timing window
  • Reset and recovery behavior
  • Evidence to retain
04Independent requirements

The agent is not the oracle

Do not let the controller code become the only source of expected behavior. If the agent reads the code, predicts what it will do, and calls matching behavior a pass, the test may only prove that the program agrees with itself.

A controls engineer must approve every expected result against a cause-and-effect sheet, control narrative, equipment specification, or another independent requirement. Ambiguous rows go into a question queue. They do not run.

A 2024 study of LLM-generated PLC test cases found that generated tests could reach strong statement coverage on a small set of open-source function blocks, while assertions often required manual correction. Coverage shows which logic executed. It does not make an invented answer correct.

05Offline execution

Execute approved rows with a deterministic runner

After engineering review, execute only the approved cases against the exact named controller artifact.

  1. 01Load the exact controller artifact named in the packet.
  2. 02Restore a known emulator snapshot before each case.
  3. 03Inject only approved transitions from a separate harness.
  4. 04Capture inputs, outputs, internal states, and timestamps across the required scans.
  5. 05Compare the trace with the engineer-approved result.
  6. 06Store the controller hash, requirement revision, runner version, trace, and reviewer.

Use the right test environment

FactoryTalk Logix Echo and S7-PLCSIM Advanced execute supported controller applications in simulated environments. CODESYS Test Manager provides a test and reporting layer for CODESYS applications. Ignition's Programmable Device Simulator can exercise tag paths, bindings, alarms, and HMI behavior without a physical PLC, but it does not execute the target controller program.

Use an emulator, soft controller, or isolated test PLC when controller behavior is the question. Keep the signal model outside production logic when possible. If simulation code must exist in the controller project, hard-gate it, review the diff, and prove that it is removed or disabled before promotion.

06Coverage

Test the awkward states

  • Timers and debounce across multiple scans
  • One-shots and sequences from a defined starting step
  • Cold start, warm restart, and required retentive states
  • Shared tags and reusable-block cross-effects
  • Reset behavior and prevention of unapproved automatic restart
  • Selected combinations of conflicting permissives
  • Code behavior missing from the approved requirements
07Evidence boundary

Keep logical and physical evidence separate

The pilot can demonstrate logical behavior for modeled inputs. It cannot prove physical I/O, field wiring, calibration, networks, mechanics, real process dynamics, operator response, or safety performance.

Record logical pass in the named emulator or test bench separately from physical pass on the actual installation. Never let the first imply the second.

08One to two weeks

Run a bounded pilot

Continue only if the workflow produces complete traceability, repeatable execution against the same artifact hash, no unapproved oracle, and zero production connections, downloads, online edits, or forces.

This small internal harness will not replace a full virtual-commissioning platform. It can show whether your requirements and test architecture are ready to benefit from one.

  1. 01Freeze one routine and its requirements.
  2. 02Have the engineer write five golden cases manually.
  3. 03Let the agent draft 20 to 40 total rows plus an ambiguity log.
  4. 04Review every expected result.
  5. 05Execute approved rows offline.
  6. 06Rerun from a clean snapshot.
  7. 07Compare preparation time, correction effort, and evidence quality with the manual baseline.
09Learn the workflow

Practical AI Foundations and Ignition AI Skills

Practical AI Foundations is a forthcoming beginner course for manufacturing, IT, and OT teams. It focuses on useful prompting, verification, read-only first passes, human approval, and implementation boundaries. Join the verified launch list on the courses page.

Ignition AI Skills provides reusable, version-aware workflows for Ignition 8.1 and 8.3 work. It is designed to reduce common errors, wrong assumptions, invented functions, Python-version drift, and repeated example-pasting. It does not promise zero hallucinations or production safety.

Keep human review, diffs, read-only discovery, dry runs, Gateway validation, readbacks, logs, backups, and rollback explicit.

Article FAQ

Frequently asked questions

Why should an AI agent write the PLC test matrix before control code?

A matrix is easier to review and tests whether the agent understands the machine's expected behavior before the team accepts generated ladder logic or structured text.

Can the PLC code itself define the expected test result?

Not by itself. Every expected result should be approved against an independent cause-and-effect sheet, control narrative, equipment specification, or other accepted requirement.

Where should AI-generated PLC tests run?

Run only engineer-approved cases in an offline emulator, soft controller, or isolated test bench with a deterministic harness and a known starting snapshot.

Does an emulator test prove the physical machine works?

No. It can prove logical behavior for modeled inputs. Physical I/O, wiring, calibration, networks, mechanics, process dynamics, operator response, and safety need separate evidence on the installation.

What is a reasonable first PLC AI pilot?

Use one non-safety routine with about 15 to 30 ordinary-control interlocks, five engineer-written golden cases, 20 to 40 total rows, an ambiguity log, offline execution, and a clean rerun.

Sources and notes

Documentation referenced

Lifetime membership

Want the toolkit behind this workflow?

The one-time Ignition AI Toolkit membership includes the Web Dev API runner, skill files, setup docs, versioned downloads, and released Ignition 8.3 resources for lifetime members. Founder pricing is available while the first 50 spots last.

Founder lifetime access - $99