FreeDMR/docs/test-harness-design.md

# FreeDMR End-to-End Packet Test Harness Design

For concise commands to run these tests, see [testing.md](testing.md).

## Scope

FreeDMR needs two complementary packet test layers:

1. An in-process deterministic harness for fast, isolated tests of decoded packet
   handling in `bridge_master.py`.
2. A black-box UDP integration harness for realistic process, socket, login,
   authentication and packet-cadence tests.

Both layers now exist as test-only code under `tests/`. The current
implementation is intentionally small: it establishes the harness architecture,
packet builders, captures, dependency isolation, and one smoke scenario per
layer. The scenario set should be expanded incrementally without changing
production behaviour.

## Current Implementation

The harness code is split as follows:

- `tests/harness/deterministic.py`
  - `PacketSpec`: synthetic `DMRD` packet builder and decoded argument adapter.
  - `parse_dmr_fields()`: shared parser for captured `DMRD` payload assertions.
  - `PacketCapture` and `CapturedPacket`: in-process send capture.
  - `ReportCapture`, `FakeClock`, `FakeReactor`, and `FakeTransport`: test
    doubles for FreeDMR runtime boundaries.
  - `DeterministicScenario`: isolated in-process scenario setup for
    `bridge_master.py` globals and real `routerHBP` / `routerOBP` instances.
  - `minimal_config()`, `active_bridge()`, and `add_openbridge_system()`:
    helpers for small test topologies.
- `tests/harness/udp_blackbox.py`
  - `DependencySandbox`: chooses an interpreter for starting FreeDMR, or
    bootstraps a venv from `requirements.txt` when explicitly enabled.
  - `write_bridge_master_config()`: emits a loopback-only subprocess config,
    including optional OpenBridge/FBP peer sections.
  - `FreeDmrProcess`: starts and stops `bridge_master.py`.
  - `HbpRepeater`: UDP HBP client emulator with login, ping, packet send, stream
    send, and capture support.
  - `FbpPeer`: UDP FBP v5 peer emulator with signed packet sends, keepalive,
    version negotiation, STUN and source-quench control helpers.
  - `UdpBlackBoxScenario`: process plus two-master loopback topology with
    optional FBP peers.
- `tests/test_deterministic_harness.py`
  - Packet builder smoke coverage.
  - In-process HBP static TG routing smoke coverage, skipped when runtime
    dependencies needed to import `bridge_master.py` are unavailable.
  - Dial-a-TG TS1 private-call control and status reporting of TS2 reflector
    state, including reserved target no-op behavior.
- `tests/test_udp_blackbox_harness.py`
  - Opt-in subprocess UDP coverage for two registered repeaters and static TG 91
    routing.
  - Opt-in dial-a-TG prompt coverage for a reserved control private call,
    asserting local TG9 TS2 announcement packets and no inter-master UDP leak.
  - Opt-in FBP v5 coverage for HBP-to-FBP and FBP-to-HBP static TG routing,
    source-quench suppression, and network-ID rejection.

The concise run commands live in [testing.md](testing.md).

## Layer 1: In-Process Deterministic Harness

The deterministic harness bypasses UDP sockets and DMR 30 ms slot timing. The
implemented scenario path uses the router seam and supports a parser seam:

- Parser seam: feed raw `DMRD` bytes directly to
  `HBSYSTEM.master_datagramReceived()` or `OPENBRIDGE.datagramReceived()` with a
  fake source address and fake transport via `DeterministicScenario.inject_datagram()`.
  This tests packet parsing and transport gates without binding sockets. `DMRE`
  packet-builder support is planned but not implemented yet.
- Router seam: inject already-decoded packet metadata at the smallest safe seam
  around `bridge_master.py`.

The router seam is implemented and is the default for most scenarios:

- HBP traffic enters at `routerHBP.dmrd_received(peer_id, rf_src, dst_id, seq,
  slot, call_type, frame_type, dtype_vseq, stream_id, data)`.
- OpenBridge traffic enters at `routerOBP.dmrd_received(peer_id, rf_src, dst_id,
  seq, slot, call_type, frame_type, dtype_vseq, stream_id, data, hash, hops,
  source_server, ber, rssi, source_rptr)`.
- Outbound traffic is captured by replacing each test system's `send_system()`
  method. Production routing calls `systems[target].send_system(...)` after
  applying intended rewrites, making it the narrow outbound observation point.

The harness owns test-only state setup:

- Build a minimal `CONFIG`, `BRIDGES`, `SUB_MAP`, aliases and `systems` map per
  scenario.
- Instantiate real `routerHBP` and `routerOBP` objects.
- Replace network-facing sends and report sends with capture objects.
- Provide a fake clock by monkeypatching `bridge_master.time`.
- Generate synthetic `DMRD` payloads while preserving original bytes for
  comparison. Recorded fixture loading and `DMRE` fixture generation are planned
  extensions.

This layer treats timing as explicit scenario input. A test can advance the fake
clock by 0.030 seconds per frame, by several seconds for hangtime, or by minutes
for rule timeout checks without sleeping.

### Bugs Layer 1 Can Detect

- Packet parsing bugs when tests use the parser seam: incorrect source,
  destination, slot, call type, frame type, dtype/voice sequence or stream ID
  extraction from raw bytes.
- Routing-rule mistakes: wrong target system, duplicate routing, missed bridge,
  wrong active/inactive bridge selection.
- Dial-a-TG state transition bugs: reflector bridge creation, activation,
  deactivation, timer reset and single-mode behaviour.
- Dial-a-TG system-scope bugs: control calls on one master should only mutate
  that receiving master's TS2 reflector state, not another master's entry in the
  same bridge.
- Dial-a-TG control-slot bugs: private calls from TS1 or TS2 should control the
  TS2 reflector state so TS1 can disconnect or retune TS2 when TS2 RF is busy.
  Status query `5000` follows the same rule and reports TS2 reflector state from
  either RF slot.
- Dial-a-TG status-report bugs: private call `5000` should report one active
  TS2 reflector for the receiving master. It does not repair inconsistent
  multi-active reflector state.
- Dial-a-TG reserved-target bugs: private calls to local/control targets such as
  `5`, `6`, `7`, and `9`, and reserved control-range targets `4001..4999`,
  should not create, activate or retune reflector state. The `4001..4999`
  range should report busy rather than announce a successful link.
- Dial-a-TG AllStar-control bugs: private call `8` is an AllStar mode control
  target. When AllStar is disabled it reports busy; when enabled it enters
  AllStar mode and schedules reset. It should not create or retune reflector
  state or announce a dial-a-TG link.
- Dial-a-TG default-reflector configuration bugs: startup and live options reload
  should use the same prohibited default-reflector targets. Reserved/control
  targets such as `6`, `7`, and AllStar control target `8` should not create an
  active default TS2 reflector at startup. The FreeDMR policy cap should match
  RF dial-a-TG handling: `999999` is valid and higher default reflectors are
  rejected. Invalid default-reflector options should disable any existing
  default reflector for the current session rather than preserving stale TS2 TG9
  reflector state. Invalid startup defaults should be logged and should not
  create bridge state; the in-memory effective default should normalize to `0`
  without writing back to the config file. System-wide defaults are intended for
  sparing use; client requested settings are preferential.
- Static TG configuration bugs: startup and live options reload should reject
  prohibited local/control TGs consistently on both TS1 and TS2 after parsing the
  configured TG strings to integers. Invalid IDs at or above `16777215` should
  be rejected consistently. Simple whitespace should be normalized away. Invalid
  tokens inside one static TG list should be skipped without blocking other
  valid tokens or the other slot's valid list. A prohibited TS1 static TG should
  not be logged as ignored and then created anyway.
- Client options parsing bugs: malformed independent numeric fields such as
  `IDENTTG=A`, `VOICE=A`, or `SINGLE=A` should not abort otherwise valid session
  options in the same string. `VOICE` and `SINGLE` accept only `0` or `1`.
  Empty `DIAL` / `DEFAULT_REFLECTOR` is equivalent to `0` and means no default
  reflector. Invalid `TIMER` values should be logged and should not block valid
  static TG changes, which should use the current effective timer.
- Voice ident override bugs: `OVERRIDE_IDENT_TG` should be parsed before packet
  generation. Valid positive TGs below all-call are used as the destination;
  empty or false override values use all-call; malformed, out-of-range, or
  local/control TG values are logged and fall back to all-call.
- Voice prompt stream scoping bugs: generated prompt helpers should use the
  router instance's `self._system` for stream bookkeeping. A stale module-level
  `system` loop variable must not cause prompt packets for one master to mutate
  another master's status.
- Voice ident lifecycle bugs: generated ident playback should use the same
  prompt token/cancellation lifecycle as other generated voice helpers, so an
  interrupted ident cannot leave stale cancel state that blocks later idents.
- Bridge reset lifecycle bugs: resetting a master should leave that master
  represented by an inactive bridge entry, preserve unrelated bridge entries,
  and keep reflector activation triggers such as `ON=[235]` for `#235` bridges.
- HBP reset/reload admission bugs: packets should be admitted when lifecycle
  flags are absent or false, and should be dropped with a one-shot log while the
  receiving master is actively resetting or reloading options.
- Data packet reporting bugs: HBP unit data forwarded to OBP should report on
  the OBP target system and reporting must not raise after the packet has
  already been sent.
- Data packet metadata bugs: HBP unit data forwarded to OpenBridge/FBP should
  preserve BER/RSSI send metadata just like the group/voice path, while
  DATA-GATEWAY remains a protocol-v1 SMS/GPS path and must not be evaluated as
  an FBP peer.
- OBP unit-data FBP metadata bugs: unit data forwarded from one FBP peer to
  another should preserve source server, source repeater for protocol versions
  that support it, hops, BER and RSSI without treating lower protocol versions
  as if they carried every field.
- OpenBridge parser bugs: truncated `DMRE` datagrams should be logged and
  discarded before fixed-offset metadata parsing, so malformed UDP input cannot
  raise out of the parser.
- OpenBridge bridge-control bugs: generated `BCST` STUN packets should validate
  against the same signed bytes on receive and set the traffic gate that
  production OpenBridge send/receive paths already check.
- OpenBridge source-quench bugs: dial-a-TG reflector forwarding from HBP to FBP
  should apply `BCSQ` using the reflector TG carried on FBP, not local TG9.
- HBP parser bugs: truncated `DMRD` datagrams from connected peers should be
  logged and discarded before fixed-offset header parsing reaches decoded packet
  handling.
- Data packet HBP target-slot reporting bugs: unit data forwarded to HBP via
  `SUB_MAP` should report the explicit target slot `_d_slot`, matching the
  captured packet slot bits.
- Group-addressed data reporting bugs: group data headers and data continuation
  blocks routed over TG bridges should be reported as data, not as `GROUP VOICE`
  lifecycle events. Timeout cleanup should not create voice end events for
  data-only state.
- Voice LC rewrite boundary bugs: embedded-LC rewrite should apply only to voice
  bursts B-E and must not mutate data-sync/control payload bytes while forwarding
  over HBP or FBP paths.
- HBP group/VCSBK rate-control bugs: same-timestamp packet bursts should not
  raise during local packet-rate calculation before duplicate/drop handling can
  run.
- OBP group voice rate-control bugs: per-stream packet-rate protection should
  use elapsed stream duration, not the absolute stream start timestamp.
- OBP voice lifecycle/report-coupling bugs: terminators should mark streams
  finished even when live reporting is disabled, so late same-stream packets do
  not route because a dashboard option is off.
- OBP voice rewrite error-path bugs: missing target LC state should log with the
  correct router name and should not crash while handling malformed or
  inconsistent stream state.
- HBP voice lifecycle bugs: terminators should mark the slot stream finished so
  late same-stream voice bursts do not route or reopen the ended stream, and
  new-stream classification must not inherit stale data state from the previous
  slot occupant. Terminator-only first observations on idle slots should still
  close the stream locally.
- HBP/OBP voice packet-control bugs: DMRD sequence numbers are one-byte
  modulo-256 values. The deterministic harness verifies streams continue after
  wrap with packet loss and that sequence `0` duplicates are still rejected.
- HBP stale duplicate-state bugs: new HBP streams should reset per-stream
  duplicate state such as `lastSeq` and `lastData` so a stream after a timeout
  is not judged against the previous slot occupant.
- OpenBridge target lifecycle bugs: forwarded voice terminators should mark OBP
  target streams finished so timeout cleanup only handles missing terminators,
  not streams that already ended normally.
- HBP VCSBK reporting bugs: specific VCSBK block data reports should not be
  duplicated by generic `OTHER DATA` fallback reports; unknown VCSBK types
  should still use the fallback and should not create voice lifecycle reports.
- OBP unit-data loop-control bugs: same-timestamp duplicate OBP sources should
  not crash diagnostic packet-rate calculations while first-source loop-control
  ignores the later source.
- Enhanced OpenBridge sendability bugs: enhanced OBP targets require recent
  `_bcka` keepalive state before receiving forwarded traffic. Missing or stale
  keepalive state should suppress HBP-originated voice/data and OBP-originated
  data without mutating packet bytes.
- Config/startup support bugs: config booleans should be parsed as booleans
  across both `hblink.py` admission and `bridge_master.py` forwarding layers,
  alias reload timing should use the already-normalized seconds value, alias
  reloads should update both module globals and shared `CONFIG` dictionaries,
  and bridge reset should tolerate session keys removed by hblink disconnect
  lifecycle.
- Protocol-version-sensitive metadata: packet metadata/options and argument
  ordering must be asserted against the protocol version actually in use for the
  session. FBP expectations must not be applied to protocol v1 DATA-GATEWAY
  traffic.
- Data packet protocol model: data packets are packet-oriented rather than
  AMBE2+ audio-style streams, and may be unit addressed or group addressed to a
  talkgroup.
- Dial-a-TG echo-target regressions: private call `9990` is intentionally
  linkable as an echo/test target, while `9991..9999` remain information
  services.
- Dial-a-TG information-service regressions: private calls to `9991..9999`
  schedule the requested on-demand AMBE file and also keep the existing generic
  silence speech scheduling. They should not create or retune reflector state.
- Dial-a-TG policy-range regressions: the current FreeDMR dial-a-TG link policy
  caps link targets at `999999`. `999999` remains linkable; higher private-call
  targets should report busy rather than announcing a successful link.
- Private voice lifecycle bugs: private unit calls such as dial-a-TG and AMI
  control should not be timed out as `GROUP VOICE` RX lifecycle events.
- Dial-a-TG FBP target regressions: when a linkable reflector target is created,
  matching OpenBridge/FBP systems are intentionally added as active route
  targets. OpenBridge protocol versions greater than 1 are termed FBP,
  FreeDMR Bridge Protocol. The current reflector creation rule excludes
  `9990..9999` from FBP target creation. FBP route target lifetime follows
  FreeDMR's "everything everywhere" principle: retuning or disconnecting a local
  master reflector entry does not deactivate already-created FBP route targets;
  source quench provides the selective behavior, and `rule_timer_loop()` clears
  disconnected FBP-only route targets.
- Slot handling bugs after decoded metadata is available: wrong target slot,
  incorrect slot-bit rewrite and incorrect slot-specific `STATUS` updates.
- Packet rewrite bugs in `bridge_master.py`: destination TG rewrite, stream ID
  preservation, source ID preservation, LC rewrite regions and unintended byte
  mutation.
- Stream lifecycle bugs in router state: duplicate detection, terminator
  handling, stale stream trimming and source-timeout logic when driven by a fake
  clock.
- Data-call routing bugs that depend on `SUB_MAP`, configured peer systems and
  bridge state.

### Bugs Layer 1 Cannot Detect

- UDP socket binding, address-family, packet loss or process startup issues.
- Repeater login/authentication handshake bugs.
- Socket-level UDP receive bugs. Parser-seam tests can cover malformed payload
  handling, but they do not prove the OS socket path delivers those bytes.
- Real scheduling bugs caused by Twisted reactor timing, OS buffering or packets
  arriving at true 30 ms cadence.
- Interoperability bugs with real clients that depend on exact UDP source
  address, port reuse, NAT behaviour or keepalive timing.
- Bugs in final transport serialization performed by production
  `send_peers()`, `send_master()` or OpenBridge `send_system()` after the
  deterministic capture point.

## Layer 2: Black-Box UDP Integration Harness

The UDP harness starts FreeDMR as a subprocess with a generated test
configuration and interacts only through UDP and observable outputs. The current
implementation emulates HBP repeaters/clients and FBP/OpenBridge peer servers.

Implemented:

- One or more HBP repeaters/clients, including registration/config handshake and
  keepalive ping.
- One or more FBP v5 peer servers, including signed `DMRE` packet sends,
  signed `BCKA`, `BCVE`, `BCSQ` and `BCST` bridge-control packets, and capture
  of outbound `DMRE` traffic.
- Synthetic `DMRD` packet sends using the shared `PacketSpec`.
- Synthetic FBP v5 packets derived from `PacketSpec`, with the OpenBridge
  transport envelope, timestamp, source server, source repeater, hop count,
  BER/RSSI and BLAKE2b hash generated by the harness.
- Synthetic FBP v4 packets derived from `PacketSpec`, using the older metadata
  layout without a source-repeater field. This is characterization/deprecation
  coverage; v4 is historical and is not expected to remain a long-term protocol
  contract.
- Synthetic signed v1 OpenBridge `DMRD` packets derived from `PacketSpec`, for
  protocol-refusal tests on enhanced/FBP-configured links.
- Recorded packet fixtures loaded from hex-encoded UDP payload files. Replay
  preserves bytes and leaves all parsing, routing and mutation to FreeDMR.
- Reusable `StreamProfile` helpers for realistic 30 ms voice-over packet
  sequences with optional headers and terminators.
- Optional fixed stream cadence through `HbpRepeater.send_stream(...,
  cadence_seconds=...)`, including realistic 0.030 second spacing.
- Deterministic `LinkImpairment` scheduling for fake endpoint sends. It can
  model drops, duplicates, jitter, fixed/random delay and explicit per-packet
  delay, while keeping runs reproducible through a seed. This is sender-side
  UDP impairment only; the harness does not implement a receive-side jitter
  buffer.
- Named `ImpairmentProfiles` for common patterns such as clean links, provider
  VXLAN-style reordering, mobile flutter drops, burst loss and duplicated UDP
  datagrams.
- UDP capture and parsed assertions for received packets.
- Subprocess stdout capture for optional warning/error log assertions.
- Loopback-only generated FreeDMR config with reports, API, AllStar, voice ident
  and alias downloads disabled. The generated config supports scenario-level
  knobs for global ACL fields, static TG lists and optional FBP peers.
- Black-box HBP coverage for static routing, global ACL startup parsing,
  data/control payload preservation, sequence wrap, duplicate sequence `0`
  suppression, terminator lifecycle suppression, recorded fixture replay, burst
  loss and duplicate UDP profiles, and local generated prompt output for
  dial-a-TG reserved controls.
- Black-box FBP coverage for enhanced keepalive/version setup, static TG routing
  from HBP to FBP and from FBP to HBP, BCKA gating of enhanced HBP-to-FBP
  forwarding, BCSQ source-quench suppression, invalid BCSQ rejection, BCST STUN
  gating of OpenBridge send/receive traffic, BCVE downgrade/unsupported/invalid
  handling, historical FBP v4 inbound packet layout characterization, signed v1
  OBP refusal on a v5-configured link, and rejection of inbound FBP packets with
  a mismatched network ID.
- Black-box unreliable-link coverage for HBP and FBP delayed/out-of-order
  packet arrival. Current tests delay sequence `1` behind sequence `2` at a
  realistic 30 ms cadence and assert FreeDMR forwards `0,2` while discarding the
  late `1`. The FBP case also verifies a following stream on the same trunk
  still routes after the impaired stream.
- Black-box multi-stream trunk coverage for HBP-to-FBP output: one stream is
  reordered and drops its late packet while a second clean stream on another TG
  still traverses the same FBP peer.
- Black-box generated-prompt interruption coverage: a local TG9 TS2 generated
  prompt is observed, then real HBP voice is injected and must route to another
  master rather than being blocked by the prompt.
- Black-box hostile/negative packet coverage: malformed short HBP `DMRD`,
  malformed short FBP `DMRE`, bad FBP hashes, stale FBP timestamps and max-hop
  FBP packets are exercised against the subprocess. Bad or malformed packets
  must not leak to HBP targets; stale and max-hop FBP packets must return BCSQ
  source-quench for the affected TG/stream. Selected negative tests assert the
  subprocess log messages as well as packet behavior.
- Runtime dependency resolution through current Python, `FREEDMR_UDP_PYTHON`, or
  an opt-in venv bootstrap. The venv bootstrap installs `requirements.txt` into
  the test venv and does not modify production code.

Planned:

- Additional unreliable-link scenarios: whole-trunk impairment warning and more
  simultaneous FBP streams with different impairment profiles.
- Voice-ident interruption coverage with `VOICE_IDENT` enabled, once a reliable
  short-trigger mechanism is added to the generated test config. Production
  currently starts the ident loop after a fixed 914 second interval, so a fast
  subprocess test would need a test hook or a long-running opt-in mode.
- A third opt-in Docker/proxy integration layer for packaged deployments that
  run the hotspot proxy by default. Proxy/firewall tests should avoid modifying
  real host firewall state unless isolated by Docker or a fake command runner.
  Related firewall code may live outside this repo and should be inspected only
  when network access is explicitly needed.

The UDP harness should capture outbound UDP packets using local sockets bound to
the emulated client or peer addresses. Assertions should parse captured UDP
payloads and compare observable behaviour:

- Which emulated endpoint received traffic.
- Packet counts, order and timing windows.
- Header fields, slot bit, source, destination, stream ID, BER/RSSI and OBP
  metadata.
- Keepalive, registration and source-quench behaviour.
- Absence of unintended traffic to real network addresses.

The subprocess config must bind only to loopback and ephemeral or test-reserved
ports. Test config files should disable production reports, API, voice ident,
AllStar and external alias downloads unless a scenario explicitly covers them.

The UDP harness can run FreeDMR under:

- the current Python interpreter, when all runtime dependencies are already
  installed;
- an explicit interpreter selected with `FREEDMR_UDP_PYTHON=/path/to/python`;
- an opt-in virtualenv created by the harness when
  `FREEDMR_UDP_BOOTSTRAP_VENV=1` is set.

When bootstrapping is enabled, dependencies are installed from `requirements.txt`
inside the venv. Set `FREEDMR_UDP_VENV_DIR=/path/to/venv` to reuse a persistent
test venv; otherwise a temporary venv is used for the scenario.

### Bugs Layer 2 Can Detect

- UDP parsing and raw packet validation bugs in `hblink.py`.
- Authentication, registration, keepalive and peer timeout bugs.
- HMAC/BLAKE2 hash handling for OpenBridge versions.
- Transport serialization bugs after `bridge_master.py` calls `send_system()`.
- Bugs caused by FreeDMR startup config, process lifecycle, Twisted reactor
  scheduling or socket binding.
- Cadence-sensitive bugs: packet-rate limiting, duplicate/out-of-order handling
  under realistic arrival spacing and jitter.
- Regressions against FreeDMR's real-time discard model: delayed packets should
  not be re-emitted in corrected order or override loop-control/source-quench
  decisions.
- Robustness bugs in malformed/hostile UDP handling: short datagrams, bad FBP
  hashes, stale timestamps and max-hop enforcement should be logged/ignored or
  quenched without crashing or forwarding invalid traffic.
- Bridge-control state bugs visible over UDP: missing enhanced keepalive should
  suppress enhanced target forwarding, invalid BCSQ must not suppress streams,
  and valid BCST STUN should block OpenBridge traffic without being confused
  with unrelated HBP-to-HBP routing.
- Version-negotiation bugs visible over UDP: BCVE downgrade, unsupported version
  or invalid hash must not mutate the configured outbound behavior, and v4
  packet fixtures characterize the historical v4 metadata layout.
- Known protocol-version issues can be carried as expected-failure black-box
  tests until runtime behavior is changed: unsupported embedded `DMRE` versions
  are currently not rejected, and the v4 send layout currently carries the
  module default version byte instead of the configured `PROTO_VER` value. v4
  is historical/deprecation context, not a desired long-term compatibility
  target.
- Protocol-refusal bugs visible over UDP: signed v1 OBP packets on a
  v5-configured link should produce BCVE and should not leak to HBP targets.
  v1 itself remains supported as an open OBP interop protocol, especially for
  external network bridge instances through `bridge.py`; direct
  `bridge_master.py` FBP tests only assert refusal when a link is configured for
  v5.
- `bridge.py` backport checks are intentionally narrower than the
  `bridge_master.py` harness. Current coverage verifies source-level shared
  sequence arithmetic and uses `py_compile` for syntax; full packet-path
  behavior remains covered through the main deterministic and UDP harnesses
  unless a dedicated bridge-instance runtime harness is added.
- Observable interoperability regressions between emulated repeaters, clients
  and peer servers.
- Generated voice prompt/ident regressions that are externally visible as
  blocked or missing real HBP traffic.

### Bugs Layer 2 Cannot Detect

- Internal state transitions that have no observable UDP effect unless extra
  reporting or logs are asserted.
- Exact branch-level causes for routing decisions without coupling tests to
  logs or report streams.
- RF-side behaviour outside the UDP protocol, such as real radio timing,
  repeater firmware quirks and modem-level DMR slot contention.
- AMBE recovery, terminal late entry, MMDVM jitter buffering or RF-path stream
  recovery decisions. FreeDMR-owned stream IDs and UDP/IP impairment are the
  model under test here.
- Rare internet or NAT behaviour unless the harness is extended beyond loopback.
- Proxy packaging behaviour, hotspot-proxy multiplexing and firewall/iptables
  integration until a third Docker/proxy harness is added.

## Shared Packet and Fixture Model

Both layers share packet builders and capture parsing today, and should share
fixture readers once recorded fixtures are added:

- `PacketSpec` represents intent: client/repeater identity, slot, source ID,
  destination TG or unit ID, stream ID, sequence, frame type, call type,
  dtype/voice sequence, payload bytes and optional frame delay.
- Synthetic fixtures build canonical `DMRD` payload bytes from `PacketSpec`.
- Recorded fixture support is not implemented yet. When added, fixtures should
  keep raw bytes plus sidecar metadata describing expected decoded fields and
  allowed rewrite regions.
- Transport simulation and protocol mutation are separate. Builders may create
  valid transport envelopes; only production code may perform route-driven
  rewrites. Tests compare original and captured bytes with explicit allowed
  rewrite ranges.

## Capture and Assertions

The deterministic harness captures calls to `send_system()` before real network
traffic. The UDP harness captures datagrams at socket boundaries. Both should
produce a common capture record where possible:

- Target system or endpoint.
- Exact packet bytes at that layer.
- Parsed DMR fields: peer/network ID, source, destination, slot, call type,
  frame type, dtype/voice sequence and stream ID.
- Transport metadata when present: source server, source repeater, hops, BER,
  RSSI, hash/version fields and UDP address.
- Scenario time or wall-clock receive time.

Assertions should be grouped by intent:

- Routing assertions: recipient set, non-recipient set, count and order.
- Byte preservation assertions: unchanged bytes outside allowed rewrite ranges.
- Rewrite assertions: TG, slot bit, LC and transport envelope changes.
- State assertions: `STATUS`, bridge `ACTIVE`, timers, `SUB_MAP`, report events.
- Timing assertions: deterministic fake-clock checks in layer 1, wall-clock
  windows in layer 2.

## Risks and Limitations

- `bridge_master.py` relies heavily on module globals. Deterministic scenarios
  must isolate and restore `CONFIG`, `BRIDGES`, `SUB_MAP`, aliases, `AMIOBJ` and
  `hblink.systems`.
- Direct `dmrd_received()` injection bypasses transport gates by design. Any
  test claiming login, HMAC, UDP parsing or real cadence coverage belongs in the
  UDP layer.
- Minimal synthetic voice payloads may not be sufficient for scenarios that
  assert full LC encoding. Recorded fixtures or carefully generated payloads
  should be used for those cases.
- Embedded LC can carry information such as embedded GPS and talker alias. The
  current harness protects against accidental mutation of data/control packets,
  but it does not yet verify future source-to-destination embedded-LC carry-over.
- Generated prompt interruption is covered in both layers for state and
  UDP-visible routing. The harness still does not prove RF-side audio behavior
  or how a physical repeater/radio reacts to an abandoned prompt without a
  terminator.
- Fake-clock tests can hide real scheduling issues. UDP cadence tests should
  cover packet-rate and timeout behaviour before relying on the harness for
  release confidence.
- Black-box UDP tests are slower and more brittle. They should cover a small
  number of high-value flows, while the deterministic layer carries most routing
  and rewrite coverage.

## Current Scenario Coverage

- `PacketSpec` builds parseable `DMRD` payloads.
- Deterministic HBP group packet routes to an active static TG target.
- Deterministic cross-slot routing tests verify that TS1-to-TS2 routing rewrites
  only the slot bit while preserving source ID, destination TG, peer ID, stream
  ID and packet bytes outside the expected header bit.
- Deterministic dial-a-TG tests verify that TS1 private calls create, retune,
  disconnect and query TS2 reflector state without emitting network traffic.
- Deterministic generated-prompt tests verify first-packet prompt state, prompt
  cancellation when real HBP voice wins the same slot, and embedded-LC rewrite
  for late entry after cancellation.
- Deterministic status tests verify that `5000` reports one active reflector for
  the receiving master even if stale multi-active state exists.
- Deterministic dial-a-TG scope tests verify that disconnecting or retuning on
  one master does not mutate another master's reflector entry.
- Deterministic reserved-target tests verify that TS1 private calls to `5`, `6`,
  `7` and `9` do not create or retune reflector bridges.
- Deterministic AllStar-control tests verify that target `8` reports busy when
  AllStar is disabled, enters AllStar mode when enabled, and never creates or
  retunes dial-a-TG reflector state.
- Deterministic reserved control-range tests verify that TS1 private calls to
  `4001..4999` do not create or retune reflector bridges and report busy rather
  than linked.
- Deterministic echo-target tests verify that TS1 private call `9990` creates
  and activates the TS2 reflector state and announces a link.
- Deterministic information-service tests verify that `9991..9999` schedules
  both the requested AMBE file and the generic silence prompt without creating a
  reflector.
- Deterministic policy-range tests verify that `999999` is still linkable while
  `1000000` does not create or retune reflector state and reports busy rather
  than linked.
- Deterministic default-reflector tests verify that startup rejects reserved
  control targets `6`, `7`, and `8`, while still allowing a linkable default
  reflector target to create an active TS2 reflector. They also verify
  `999999` remains valid and startup/options reject default reflector targets
  above that policy cap, with invalid options disabling any active default
  reflector state and invalid startup defaults producing a warning while
  normalizing runtime state to `0`.
- Deterministic static-TG configuration tests verify that startup rejects
  prohibited TS1 and TS2 static TGs after integer parsing, rejects invalid IDs
  at or above `16777215`, and that options reload rejects prohibited or
  out-of-range TS1 static TGs rather than creating them after logging the
  prohibition. They also verify whitespace normalization and token-level
  skipping of invalid static TG tokens while valid tokens still apply.
- Deterministic options parser tests verify malformed independent numeric fields
  do not block valid DIAL/static fields, boolean-like options reject values other
  than `0` or `1`, empty `DIAL` disables default reflector state, and invalid
  `TIMER` values are logged without blocking valid static TG changes.
- Deterministic voice-ident tests verify override destination selection for
  valid string TGs, empty/false overrides, malformed values, control TGs, and
  all-call.
- Deterministic FBP-target tests verify that linkable dial-a-TG reflector
  creation adds active FBP route targets where the current production rule
  permits it, and that those route targets remain active across local master
  retunes and disconnects until `rule_timer_loop()` removes disconnected
  FBP-only reflector bridges.
- UDP black-box HBP repeaters register with FreeDMR and observe static TG 91
  routing over real UDP.
- UDP black-box dial-a-TG tests verify that a reserved control private call
  emits a local TG9 TS2 prompt without sending traffic to another master.
- UDP black-box FBP bridge-control tests verify that enhanced targets require
  BCKA before HBP-to-FBP forwarding, invalid BCSQ does not suppress a stream,
  and valid BCST STUN blocks OpenBridge traffic in both directions.
- UDP black-box FBP version tests verify that BCVE downgrade, unsupported
  version and invalid hash do not change outbound packet version, and that
  historical v4 packet fixtures currently route using the older metadata layout.
  This v4 coverage is characterization/deprecation context.
- UDP black-box OBP-v1 refusal tests verify that a signed v1 packet received on
  a v5-configured link receives BCVE and does not route onward.

## Next Deterministic Scenario Tests

1. HBP group voice routes to another HBP master on the same TG.
   The current smoke test covers a single packet. Extend it to a header, burst
   and terminator stream and assert expected LC rewrite regions.

2. HBP slot rewrite when bridge targets a different slot.
   Build `MASTER-A` active on TG 91 slot 1 and `MASTER-B` active on TG 91 slot
   2. Inject a slot 1 packet from `MASTER-A`. Assert captured traffic to
   `MASTER-B` has the slot bit flipped to slot 2 while source ID and stream ID
   remain unchanged.

3. Dial-a-TG timeout lifecycle.
   Build one master system with default UA timer enabled and an active TS2
   reflector bridge. Advance fake time and run the timer path to assert the
   bridge deactivates without emitting network traffic.