FreeDMR/docs/testing.md

# Testing

FreeDMR currently has two packet test harness layers under `tests/`.

## Deterministic Harness

The deterministic harness runs in-process. It bypasses UDP sockets and captures
calls that would otherwise send network traffic.

Run it with:

```bash
PYTHONDONTWRITEBYTECODE=1 python -m unittest tests.test_deterministic_harness -v
```

If FreeDMR runtime dependencies are not installed, tests that import
`bridge_master.py` are skipped. Pure harness tests still run.

The deterministic suite includes static TG routing and packet rewrite coverage.
It verifies cross-slot TS1-to-TS2 routing changes only the expected slot bit
while preserving packet identity fields and bytes outside that header bit.

API controller coverage verifies the experimental HTTP/JSON API performs only
small in-memory control-plane operations, returns a clear no-options response,
preserves caller-supplied `OPTIONS` strings unchanged, validates peer/system
keys, and exposes JSON error responses without requiring Spyne.

Auxiliary utility coverage verifies small non-packet helpers used by optional
operations: AMI client factories keep per-command state on protocol instances,
report receiver CLI flags parse `0` as false, and the SQL report client uses
the factory-held database object with parameterized inserts. It also covers the
hotspot proxy environment boolean parser so Docker settings such as
`FDPROXY_IPV6=0` disable the feature.

Standalone DMR codec coverage verifies the new `freedmr_dmr_codec.py` helpers
before they are wired into packet forwarding. These tests cover MMDVMHost-style
embedded LC encode/decode, Hamming(16,11,4) single-bit correction,
uncorrectable error rejection, checksum-checked round trips, DMR payload
embedded-LC slice helpers, full LC header/terminator BPTC generation, RS(12,9)
LC parity masks, group/unit LC classification, and Golay(20,8,7) slot type
encode/decode correction. Synthetic group voice LC generation defaults to
normal service options (`0x00`) and keeps the HBLink `0x20` value only as an
explicit legacy constant. The full LC, routing-style embedded LC, voice
header/terminator and voice burst fixtures are fixed byte/bit vectors captured
from known-good behaviour, so these tests no longer need `dmr_utils3`.
Runtime LC generation and the voice header/terminator and burst decode helpers
used by `bridge_master.py` and `bridge.py` now use compatibility functions in
`freedmr_dmr_codec.py`. Active runtime helper functions such as `bytes_3()`,
`bytes_4()`, `int_id()` and `get_alias()` are provided by FreeDMR `utils.py`.

The deterministic suite includes dial-a-TG coverage. It verifies that private
calls from TS1 create, retune, disconnect and query TS1 reflector state, while
private calls from TS2 control TS2 reflector state. TS1 no longer controls TS2.
Default dial-a-TG startup and OPTIONS handling is covered through canonical
per-slot `DEFAULT_DIAL_TS1` and `DEFAULT_DIAL_TS2`; deprecated
`DEFAULT_REFLECTOR`, `DIAL` and `StartRef` remain TS2 compatibility aliases.
It also covers late-entry synthetic LC fallback: streams without a decodable
voice header fabricate normal group voice LC bytes, while streams with a real
voice header preserve the decoded LC service-options byte unchanged.
It verifies these state changes are scoped to the receiving master system.
Status query `5000` reports one active reflector and does not repair stale
multi-active state.
It also verifies reserved local/control targets do not create or retune reflector
bridges, and that reserved control-range targets `4001..4999` report busy rather
than announcing a successful link. Target `8` is covered as an AllStar control
target, not a dial-a-TG link target. Private call `9990` is covered as an
intentional echo/test link target. Information-service targets `9991..9999`
are covered as scheduling both the requested AMBE file and the generic silence
prompt without creating reflector state. The current FreeDMR dial-a-TG policy
cap is covered: `999999` remains linkable and higher targets report busy rather
than linked. Private dial-a-TG timeout coverage verifies private unit calls do
not emit unmatched `GROUP VOICE,END,RX` lifecycle events. Startup default-dial handling is covered so reserved/control
targets `6`, `7`, and `8` are rejected, `999999` is accepted, and higher targets
are rejected. Invalid default-dial options disable the effective slot TG9
default reflector for the session; invalid startup defaults are logged and do
not create bridge state, with the in-memory effective default normalized to `0`.
A linkable target can still create an active per-slot default reflector. Static TG startup and options handling is
covered so prohibited local/control TGs are rejected consistently on both TS1
and TS2, and invalid IDs at or above `16777215` are rejected while linkable
static TGs are still created. Static TG option parsing also covers simple
whitespace normalization and token-level skipping of invalid tokens so valid
tokens in the same TS1 or TS2 list still apply. Client options parsing covers
malformed independent numeric fields so valid default-dial/static fields still apply,
`VOICE` and `SINGLE` accept only `0`/`1`, and empty `DIAL` disables the default
TS2 reflector. Invalid `TIMER` values are logged and static TG changes continue
using the current effective timer. Voice ident override coverage verifies valid
override TGs are used, empty/false overrides use all-call, and malformed,
control, or all-call override values are logged and fall back to all-call.
Generated voice prompt helper coverage verifies prompt stream state is attached
to the router instance sending the prompt, even if a stale module-level
`system` variable names another master. This protects dial-a-TG prompts, idents,
disconnected announcements and on-demand files from cross-system status
corruption. Prompt lifecycle coverage also verifies the first generated packet
records prompt activity, real HBP voice cancels a generated prompt instead of
being blocked by it, and late-entry embedded LC rewrite still occurs for the
real voice burst after cancellation. Voice ident lifecycle coverage verifies an
interrupted ident does not leave stale prompt-cancel state that blocks a later
ident.
Bridge reset coverage verifies a reset master remains represented by its own
bridge entry, unrelated master and FBP entries are not duplicated or rewritten,
and `#` reflector activation triggers survive reset.
HBP packet admission coverage verifies reset/reload lifecycle flags are optional
false-default booleans: packets continue when both flags are false or absent,
and packets are dropped with one log record while either lifecycle flag is true.
Data packet coverage verifies HBP unit data forwarded to OBP systems still
emits reporting on the OBP target when reporting is enabled, without changing
the captured packet destination or raising from a reporting side effect. It also
verifies HBP unit data preserves BER/RSSI send metadata when forwarded to
OpenBridge/FBP targets; DATA-GATEWAY remains present for protocol-v1 SMS/GPS
handling and is not treated as an FBP peer. OBP-originated unit data forwarded
to another FBP peer is covered for source server, source repeater, hops, BER and
RSSI metadata preservation.
The OpenBridge parser seam is covered for truncated `DMRE` packets so malformed
UDP input is logged and discarded before fixed-offset parsing can raise.
Enhanced OpenBridge bridge-control coverage verifies a valid generated `BCST`
STUN packet sets the global `STUN` traffic gate.
Dial-a-TG source-quench coverage verifies HBP-to-FBP reflector forwarding checks
`BCSQ` against the reflector TG visible on FBP, not the local TG9 control path.
The HBP master parser seam is covered for truncated `DMRD` packets from a
connected peer so malformed client traffic is discarded before decoded packet
handling.
Unit data forwarded to HBP via `SUB_MAP` is covered for both HBP and OBP
sources; captured packet slot bits and TX report slot metadata must both match
the target HBP slot.
Group-addressed data reporting is covered for HBP and OBP sources; group data
headers and data continuation blocks must emit data `RX/TX` events and must not
generate `GROUP VOICE` timeout lifecycle events, while ordinary group voice still emits
voice start/end reports. HBP group data rate-drop coverage verifies
same-timestamp packet bursts do not divide by zero before duplicate/drop
handling can run.
Data-sync control payload preservation is covered across HBP-to-HBP, HBP-to-FBP,
FBP-to-HBP and FBP-to-FBP forwarding so voice embedded-LC rewrite does not mutate
VCSBK/control payload bytes.
Voice embedded-LC preservation is covered across HBP-to-HBP, HBP-to-FBP,
FBP-to-HBP and FBP-to-FBP same-TG forwarding. Those tests assert voice bursts
B-E keep their DMR payload bytes unless the target TG is intentionally rewritten.
Embedded-LC observability coverage verifies accepted in-call Talker Alias and
GPS LC cycles are decoded through the standalone validated codec to system log
messages without changing packet routing or mutation behaviour.
OBP group voice rate-drop coverage verifies per-stream packet-rate protection is
calculated from elapsed stream duration rather than the absolute stream start
timestamp. OBP voice lifecycle coverage verifies a voice terminator marks the
stream finished even when live reporting is disabled, so late packets with the
same stream ID are suppressed independently of dashboard configuration.
OBP voice rewrite error-path coverage verifies missing target embedded-LC state
is logged with the handling router name and does not crash packet processing.
HBP voice lifecycle coverage verifies a voice terminator marks the slot stream
finished, so late same-stream voice bursts are suppressed instead of reopening
or routing the ended stream. It also verifies a new voice terminator observed
after a group data packet uses the current packet's voice classification, not
stale data state from the previous slot occupant, and that an idle-slot
terminator-only voice packet still marks the stream finished.
HBP and OBP voice packet-control coverage verifies DMRD sequence numbers are
handled as modulo-256 values: a stream can route through `254`, `255`, then `2`
with the missing post-wrap packets counted as loss rather than being rejected
as out-of-order. Sequence `0` duplicate handling is also covered. HBP
new-stream duplicate-state coverage verifies a stream following a timed-out
prior stream does not inherit `lastSeq`/`lastData` and false packet loss from
the previous slot occupant.
The `bridge.py` conference-bridge backport has a lightweight source-level test
for the shared modulo-256 sequence helper. Full runtime coverage remains in the
`bridge_master.py` deterministic and UDP harnesses because importing `bridge.py`
requires the deployed FreeDMR runtime dependencies.
OpenBridge target lifecycle coverage verifies forwarded voice terminators mark
target streams finished for HBP-to-OBP and OBP-to-OBP paths, preventing the
timeout trimmer from later emitting duplicate `GROUP VOICE,END,RX` events for
streams that already ended normally.
HBP VCSBK reporting verifies specific VCSBK block RX events are not duplicated
by the generic `OTHER DATA` fallback, while unknown VCSBK types still use the
fallback event. Unknown VCSBK reports are covered for HBP and OBP sources and
must not generate `GROUP VOICE` lifecycle events.
OBP unit-data loop-control coverage verifies same-timestamp duplicate OBP
sources do not raise from diagnostic packet-rate calculation and still mark the
later source as loop-controlled.
Enhanced OpenBridge keepalive coverage verifies missing or stale `_bcka` state
suppresses forwarding to enhanced OBP targets for HBP-originated voice/data and
OBP-originated data, while recent keepalive state permits forwarding.
Config/startup support coverage verifies `GLOBAL.USE_ACL: False` is parsed as a
boolean false, alias stale days are converted to seconds exactly once, periodic
alias reload updates both `bridge_master.py` globals and the shared `CONFIG`
alias dictionaries read by `hblink.py`, and bridge reset tolerates a missing
session `OPTIONS` key after HBP disconnect/timeout lifecycle cleanup.
Linkable dial-a-TG reflector creation is covered for FBP route targets;
OpenBridge protocol
versions greater than 1 are termed FBP, FreeDMR Bridge Protocol. FBP route
targets follow FreeDMR's "everything everywhere" principle and remain active
across local master retunes and disconnects; source quench provides selective
behavior, and `rule_timer_loop()` clears disconnected FBP-only route targets.

## Black-Box UDP Harness

The UDP harness starts `bridge_master.py` as a subprocess with a generated
loopback-only test config. It emulates HBP repeaters and FBP/OpenBridge peer
servers over UDP, performs HBP login, sends signed packets/control messages, and
captures outbound UDP packets. The HBP login helper retries the initial login
challenge for a short startup window because `bridge_master.py` can spend time
loading aliases, keys and voice assets before Twisted has bound the loopback UDP
sockets.

Current UDP scenarios cover HBP registration/config handshake, static TG
routing, global `USE_ACL: False` startup parsing observed through packet
admission, data-sync/control payload preservation, same-TG voice embedded-LC
payload preservation, in-call Talker Alias/GPS log observability,
modulo-256 voice sequence wrap, sequence `0` duplicate suppression, voice
terminator suppression of late
same-stream packets, recorded HBP fixture replay, and a dial-a-TG reserved
control private call that emits a local TG9 TS2 prompt without leaking traffic
to another master. They now also cover FBP v5 static TG routing in both
directions, FBP keepalive/version control setup, BCKA gating of enhanced
HBP-to-FBP forwarding, BCSQ source-quench suppression of HBP-to-FBP forwarding,
rejection of invalid BCSQ, BCVE downgrade/unsupported/invalid-version handling,
current FBP v5 packet handling, historical FBP v4 characterization, and signed
v1 OBP packet refusal on a v5-configured link. v1 remains an important open OBP
interop protocol for external network bridge instances, primarily through
`bridge.py`; the `bridge_master.py` UDP test here only verifies that a
v5-configured FBP link refuses v1 traffic. They also cover rejection of FBP
packets carrying the wrong OpenBridge network ID. Valid BCST STUN is covered as
an OpenBridge traffic gate in both directions; ordinary HBP-to-HBP routing is
not the target of that gate. The UDP
harness also includes deterministic `LinkImpairment` scheduling for fake
endpoint sends; current scenarios use it to delay sequence `1` behind sequence
`2` at a 30 ms cadence, model burst loss and duplicate UDP datagrams, and assert
that late out-of-order or duplicate HBP and FBP packets are discarded rather
than buffered or replayed. Reusable `StreamProfile` and `ImpairmentProfiles`
helpers provide named stream and link patterns for more real-world scenarios.
Current coverage also includes a multi-stream HBP-to-FBP trunk case where one
stream is reordered while another clean stream on the same FBP trunk still
routes, plus a generated prompt interruption case where real HBP voice routes
after a local TG9 TS2 prompt has started. Negative-path coverage includes
malformed short HBP `DMRD`, malformed short FBP `DMRE`, bad FBP BLAKE2b hashes,
stale FBP timestamps and max-hop FBP packets; these must not leak traffic to HBP
targets, and stale/max-hop FBP packets must produce a source-quench response.
Selected malformed packet tests also assert subprocess warning logs.

Two UDP tests are marked as expected failures because they document current
protocol-version issues rather than fixed behavior: unsupported embedded `DMRE`
packet versions are not yet rejected, and the historical v4 send layout
currently carries the module default version byte instead of the configured
`PROTO_VER` value. v4 is characterization/deprecation context, not a long-term
protocol contract.

UDP integration tests are opt-in:

```bash
FREEDMR_RUN_UDP_TESTS=1 \
PYTHONDONTWRITEBYTECODE=1 \
python -m unittest tests.test_udp_blackbox_harness -v
```

If dependencies are already installed in another Python, point the harness at it:

```bash
FREEDMR_RUN_UDP_TESTS=1 \
FREEDMR_UDP_PYTHON=/path/to/python \
PYTHONDONTWRITEBYTECODE=1 \
python -m unittest tests.test_udp_blackbox_harness -v
```

To let the harness create a virtualenv and install `requirements.txt`:

```bash
FREEDMR_RUN_UDP_TESTS=1 \
FREEDMR_UDP_BOOTSTRAP_VENV=1 \
PYTHONDONTWRITEBYTECODE=1 \
python -m unittest tests.test_udp_blackbox_harness -v
```

The venv bootstrap installs `requirements.txt` into the test virtualenv when the
selected Python does not already have the FreeDMR runtime dependencies.

To reuse a persistent test virtualenv:

```bash
FREEDMR_RUN_UDP_TESTS=1 \
FREEDMR_UDP_BOOTSTRAP_VENV=1 \
FREEDMR_UDP_VENV_DIR=/tmp/freedmr-blackbox-venv \
PYTHONDONTWRITEBYTECODE=1 \
python -m unittest tests.test_udp_blackbox_harness -v
```

## Full Test Discovery

Run all tests with:

```bash
PYTHONDONTWRITEBYTECODE=1 python -m unittest discover -v
```

The black-box UDP tests still skip unless `FREEDMR_RUN_UDP_TESTS=1` is set.

See [test-harness-design.md](test-harness-design.md) for the harness design and
coverage tradeoffs.