The proving ground

Four layers, stacked like strata

1. 1
   The unit suite — ~750 pytest tests, everything mocked.

   Catches logic, protocol parsing, reconcile and state machinery.

2. 2
   The device zoo — contract invariants over cluster combinations.

   Catches whole classes of device-mapping bugs at once.

3. 3
   The virtual fleet — 15+ genuine Matter devices, built from matter.js.

   Catches real commissioning, subscriptions and command round-trips.

4. 4
   Real hardware, live server — shipping devices on the production path.

   Catches everything else — including what Indigo itself actually does.

The unit suite

cd indigo-matter && python3 -m pytest -q

No Indigo server and no matter-server needed: the indigo module is mocked (tests/conftest.py) and matter-server is faked at the WebSocket frame layer (tests/fakes.py), so the real client, reconcile and dispatch code paths run against recorded protocol shapes. tests/test_golden_real.py pins parsing against frames captured from a live matter-server — a wire-format drift fails loudly.

The device zoo

Matter devices arrive with cluster combinations nobody predicted — the spec requires some surprises. A Matter 1.2+ fan must expose OnOff alongside FanControl. An Extended Color Light need only implement XY and colour temperature — HueSaturation is optional. The zoo (tests/test_device_zoo.py) is a contract harness for exactly this: a table of synthetic nodes run through the real handler registry, with five invariants asserted over every entry:

1. each endpoint maps to exactly the expected Indigo device types;
2. at most one actuator device per endpoint — two would fight over one physical device;
3. every spec’s device type exists in Devices.xml;
4. every seeded initial state is declared or built-in;
5. every sensor carries explicit display props — Indigo derives the device-list display from creation props, never from XML statics (issue #56).

Track record

Caught — first run  Caught — same week

On its very first run the zoo found a colour light without LevelControl producing a duplicate relay — a case nobody had predicted. Days later it caught the button handler declaring display props without merging them into its creation spec. Invariants find the bugs you didn’t think to write a test for.

The virtual fleet

Real Matter devices, minus the shopping. The matter.js examples plus small custom scripts compose genuine commissionable Matter nodes on the LAN — the plugin and matter-server cannot tell them from shipping hardware. The development fleet runs fifteen-plus devices:

• relay
• dimmer
• extended colour light
• temp / humidity
• thermostat
• fan
• window covering
• door lock
• valve
• button
• smoke / CO
• air quality ×4
• pressure / flow
• energy plug
• battery sensor

Scripts live in /tmp/matter-test/*.mjs; commissioned identities persist in ~/.matter/<id>, so a relaunch keeps the node:

pgrep -fl "node .*\.mjs"                        # what's running
cd /tmp/matter-test && node fan.mjs >> fan.log 2>&1 &

Hard-won rig rules

Each of these cost a debugging session (war stories in HANDOVER.md):

• One UDP port per device — two devices on one port send PASE to the wrong socket.
• Distinct discriminators with different top nibbles — the 11-digit manual code only carries the 4-bit short discriminator.
• Commission within ~15 minutes of first launch, then the device stops advertising.
• After cycling many device instances, restart matter-server to clear its mDNS cache.
• Custom devices must enable their cluster features explicitly, and must not set server-managed attributes.

That colour light is an ExtendedColorLightDevice with default features — XY, no HueSaturation. The plugin sent RGB as MoveToHueAndSaturation; the device rejected every colour command with UNSUPPORTED_COMMAND. A maximal test device would have hidden that for years.

Live validation on a production server

Some truths only real Indigo knows — several of its API behaviours were established empirically here: that Supports* creation props rule the device-list display and XML statics never apply to API-created devices; that with both display props explicitly false, the XML UiDisplayStateId applies after all; that replacePluginPropsOnServer does re-derive the display — proven by deploying a fix over a fleet of pre-fix devices and watching two reconciles.

The method, reproducible by any plugin developer with a test server:

1. Deploy. rsync the build into the live plugin bundle (updates only — first installs must be double-click installed), restart the plugin.
2. Read the event log. The plugin is written so the log is the evidence: self-heals log what they changed and verify persistence by reading props back. An unverified write logs as a warning — never as success.
3. Restart again. The steady state must be quiet. Anything still healing or warning on pass two is a finding.
4. Instrument if needed. For display/UI questions, a temporary INFO line on the deployed copy answers in one restart — and never gets committed.

Real hardware, production path

The FP300 test also delivered the bug report that became issue #56 — external testers running real devices are part of the methodology, not an afterthought.