FreeDMR/docs/testing.md

Testing

FreeDMR currently has two packet test harness layers under tests/.

Deterministic Harness

The deterministic harness runs in-process. It bypasses UDP sockets and captures calls that would otherwise send network traffic.

Run it with:

PYTHONDONTWRITEBYTECODE=1 python -m unittest tests.test_deterministic_harness -v

If FreeDMR runtime dependencies are not installed, tests that import bridge_master.py are skipped. Pure harness tests still run.

The deterministic suite includes static TG routing and packet rewrite coverage. It verifies cross-slot TS1-to-TS2 routing changes only the expected slot bit while preserving packet identity fields and bytes outside that header bit.

API controller coverage verifies the experimental HTTP/JSON API performs only small in-memory control-plane operations, returns a clear no-options response, preserves caller-supplied OPTIONS strings unchanged, validates peer/system keys, and exposes JSON error responses without requiring Spyne.

Auxiliary utility coverage verifies small non-packet helpers used by optional operations: AMI client factories keep per-command state on protocol instances, report receiver CLI flags parse 0 as false, and the SQL report client uses the factory-held database object with parameterized inserts. It also covers the hotspot proxy environment boolean parser so Docker settings such as FDPROXY_IPV6=0 disable the feature.

Standalone DMR codec coverage verifies the new freedmr_dmr_codec.py helpers before they are wired into packet forwarding. These tests cover MMDVMHost-style embedded LC encode/decode, Hamming(16,11,4) single-bit correction, uncorrectable error rejection, checksum-checked round trips, DMR payload embedded-LC slice helpers, full LC header/terminator BPTC generation, RS(12,9) LC parity masks, group/unit LC classification, and Golay(20,8,7) slot type encode/decode correction. Synthetic group voice LC generation defaults to normal service options (0x00) and keeps the HBLink 0x20 value only as an explicit legacy constant. The full LC, routing-style embedded LC, voice header/terminator and voice burst fixtures are fixed byte/bit vectors captured from known-good behaviour, so these tests no longer need dmr_utils3. Runtime LC generation and the voice header/terminator and burst decode helpers used by bridge_master.py and bridge.py now use compatibility functions in freedmr_dmr_codec.py. Active runtime helper functions such as bytes_3(), bytes_4(), int_id() and get_alias() are provided by FreeDMR utils.py.

The deterministic suite includes dial-a-TG coverage. It verifies that private calls from TS1 create, retune, disconnect and query TS1 reflector state, while private calls from TS2 control TS2 reflector state. TS1 no longer controls TS2. Default dial-a-TG startup and OPTIONS handling is covered through canonical per-slot DEFAULT_DIAL_TS1 and DEFAULT_DIAL_TS2; deprecated DEFAULT_REFLECTOR, DIAL and StartRef remain TS2 compatibility aliases. It also covers late-entry synthetic LC fallback: streams without a decodable voice header fabricate normal group voice LC bytes, while streams with a real voice header preserve the decoded LC service-options byte unchanged. It verifies these state changes are scoped to the receiving master system. Status query 5000 reports one active reflector and does not repair stale multi-active state. It also verifies reserved local/control targets do not create or retune reflector bridges, and that reserved control-range targets 4001..4999 report busy rather than announcing a successful link. Target 8 is covered as an AllStar control target, not a dial-a-TG link target. Private call 9990 is covered as an intentional echo/test link target. Information-service targets 9991..9999 are covered as scheduling both the requested AMBE file and the generic silence prompt without creating reflector state. The current FreeDMR dial-a-TG policy cap is covered: 999999 remains linkable and higher targets report busy rather than linked. Private dial-a-TG timeout coverage verifies private unit calls do not emit unmatched GROUP VOICE,END,RX lifecycle events. Startup default-dial handling is covered so reserved/control targets 6, 7, and 8 are rejected, 999999 is accepted, and higher targets are rejected. Invalid default-dial options disable the effective slot TG9 default reflector for the session; invalid startup defaults are logged and do not create bridge state, with the in-memory effective default normalized to 0. A linkable target can still create an active per-slot default reflector. Static TG startup and options handling is covered so prohibited local/control TGs are rejected consistently on both TS1 and TS2, and invalid IDs at or above 16777215 are rejected while linkable static TGs are still created. Static TG option parsing also covers simple whitespace normalization and token-level skipping of invalid tokens so valid tokens in the same TS1 or TS2 list still apply. Client options parsing covers malformed independent numeric fields so valid default-dial/static fields still apply, VOICE and SINGLE accept only 0/1, and empty DIAL disables the default TS2 reflector. Invalid TIMER values are logged and static TG changes continue using the current effective timer. Voice ident override coverage verifies valid override TGs are used, empty/false overrides use all-call, and malformed, control, or all-call override values are logged and fall back to all-call. Generated voice prompt helper coverage verifies prompt stream state is attached to the router instance sending the prompt, even if a stale module-level system variable names another master. This protects dial-a-TG prompts, idents, disconnected announcements and on-demand files from cross-system status corruption. Prompt lifecycle coverage also verifies the first generated packet records prompt activity, real HBP voice cancels a generated prompt instead of being blocked by it, and late-entry embedded LC rewrite still occurs for the real voice burst after cancellation. Voice ident lifecycle coverage verifies an interrupted ident does not leave stale prompt-cancel state that blocks a later ident. Bridge reset coverage verifies a reset master remains represented by its own bridge entry, unrelated master and FBP entries are not duplicated or rewritten, and # reflector activation triggers survive reset. HBP packet admission coverage verifies reset/reload lifecycle flags are optional false-default booleans: packets continue when both flags are false or absent, and packets are dropped with one log record while either lifecycle flag is true. Data packet coverage verifies HBP unit data forwarded to OBP systems still emits reporting on the OBP target when reporting is enabled, without changing the captured packet destination or raising from a reporting side effect. It also verifies HBP unit data preserves BER/RSSI send metadata when forwarded to OpenBridge/FBP targets; DATA-GATEWAY remains present for protocol-v1 SMS/GPS handling and is not treated as an FBP peer. OBP-originated unit data forwarded to another FBP peer is covered for source server, source repeater, hops, BER and RSSI metadata preservation. The OpenBridge parser seam is covered for truncated DMRE packets so malformed UDP input is logged and discarded before fixed-offset parsing can raise. Enhanced OpenBridge bridge-control coverage verifies a valid generated BCST STUN packet sets the global STUN traffic gate. Dial-a-TG source-quench coverage verifies HBP-to-FBP reflector forwarding checks BCSQ against the reflector TG visible on FBP, not the local TG9 control path. The HBP master parser seam is covered for truncated DMRD packets from a connected peer so malformed client traffic is discarded before decoded packet handling. Unit data forwarded to HBP via SUB_MAP is covered for both HBP and OBP sources; captured packet slot bits and TX report slot metadata must both match the target HBP slot. Group-addressed data reporting is covered for HBP and OBP sources; group data headers and data continuation blocks must emit data RX/TX events and must not generate GROUP VOICE timeout lifecycle events, while ordinary group voice still emits voice start/end reports. HBP group data rate-drop coverage verifies same-timestamp packet bursts do not divide by zero before duplicate/drop handling can run. Data-sync control payload preservation is covered across HBP-to-HBP, HBP-to-FBP, FBP-to-HBP and FBP-to-FBP forwarding so voice embedded-LC rewrite does not mutate VCSBK/control payload bytes. Voice embedded-LC preservation is covered across HBP-to-HBP, HBP-to-FBP, FBP-to-HBP and FBP-to-FBP same-TG forwarding. Those tests assert voice bursts B-E keep their DMR payload bytes unless the target TG is intentionally rewritten. Embedded-LC observability coverage verifies accepted in-call Talker Alias and GPS LC cycles are decoded through the standalone validated codec to system log messages without changing packet routing or mutation behaviour. OBP group voice rate-drop coverage verifies per-stream packet-rate protection is calculated from elapsed stream duration rather than the absolute stream start timestamp. OBP voice lifecycle coverage verifies a voice terminator marks the stream finished even when live reporting is disabled, so late packets with the same stream ID are suppressed independently of dashboard configuration. OBP voice rewrite error-path coverage verifies missing target embedded-LC state is logged with the handling router name and does not crash packet processing. HBP voice lifecycle coverage verifies a voice terminator marks the slot stream finished, so late same-stream voice bursts are suppressed instead of reopening or routing the ended stream. It also verifies a new voice terminator observed after a group data packet uses the current packet's voice classification, not stale data state from the previous slot occupant, and that an idle-slot terminator-only voice packet still marks the stream finished. HBP and OBP voice packet-control coverage verifies DMRD sequence numbers are handled as modulo-256 values: a stream can route through 254, 255, then 2 with the missing post-wrap packets counted as loss rather than being rejected as out-of-order. Sequence 0 duplicate handling is also covered. HBP new-stream duplicate-state coverage verifies a stream following a timed-out prior stream does not inherit lastSeq/lastData and false packet loss from the previous slot occupant. The bridge.py conference-bridge backport has a lightweight source-level test for the shared modulo-256 sequence helper. Full runtime coverage remains in the bridge_master.py deterministic and UDP harnesses because importing bridge.py requires the deployed FreeDMR runtime dependencies. OpenBridge target lifecycle coverage verifies forwarded voice terminators mark target streams finished for HBP-to-OBP and OBP-to-OBP paths, preventing the timeout trimmer from later emitting duplicate GROUP VOICE,END,RX events for streams that already ended normally. HBP VCSBK reporting verifies specific VCSBK block RX events are not duplicated by the generic OTHER DATA fallback, while unknown VCSBK types still use the fallback event. Unknown VCSBK reports are covered for HBP and OBP sources and must not generate GROUP VOICE lifecycle events. OBP unit-data loop-control coverage verifies same-timestamp duplicate OBP sources do not raise from diagnostic packet-rate calculation and still mark the later source as loop-controlled. Enhanced OpenBridge keepalive coverage verifies missing or stale _bcka state suppresses forwarding to enhanced OBP targets for HBP-originated voice/data and OBP-originated data, while recent keepalive state permits forwarding. Config/startup support coverage verifies GLOBAL.USE_ACL: False is parsed as a boolean false, alias stale days are converted to seconds exactly once, periodic alias reload updates both bridge_master.py globals and the shared CONFIG alias dictionaries read by hblink.py, and bridge reset tolerates a missing session OPTIONS key after HBP disconnect/timeout lifecycle cleanup. Linkable dial-a-TG reflector creation is covered for FBP route targets; OpenBridge protocol versions greater than 1 are termed FBP, FreeDMR Bridge Protocol. FBP route targets follow FreeDMR's "everything everywhere" principle and remain active across local master retunes and disconnects; source quench provides selective behavior, and rule_timer_loop() clears disconnected FBP-only route targets.

Black-Box UDP Harness

The UDP harness starts bridge_master.py as a subprocess with a generated loopback-only test config. It emulates HBP repeaters and FBP/OpenBridge peer servers over UDP, performs HBP login, sends signed packets/control messages, and captures outbound UDP packets. The HBP login helper retries the initial login challenge for a short startup window because bridge_master.py can spend time loading aliases, keys and voice assets before Twisted has bound the loopback UDP sockets.

Current UDP scenarios cover HBP registration/config handshake, static TG routing, global USE_ACL: False startup parsing observed through packet admission, data-sync/control payload preservation, same-TG voice embedded-LC payload preservation, in-call Talker Alias/GPS log observability, modulo-256 voice sequence wrap, sequence 0 duplicate suppression, voice terminator suppression of late same-stream packets, recorded HBP fixture replay, and a dial-a-TG reserved control private call that emits a local TG9 TS2 prompt without leaking traffic to another master. They now also cover FBP v5 static TG routing in both directions, FBP keepalive/version control setup, BCKA gating of enhanced HBP-to-FBP forwarding, BCSQ source-quench suppression of HBP-to-FBP forwarding, rejection of invalid BCSQ, BCVE downgrade/unsupported/invalid-version handling, current FBP v5 packet handling, historical FBP v4 characterization, and signed v1 OBP packet refusal on a v5-configured link. v1 remains an important open OBP interop protocol for external network bridge instances, primarily through bridge.py; the bridge_master.py UDP test here only verifies that a v5-configured FBP link refuses v1 traffic. They also cover rejection of FBP packets carrying the wrong OpenBridge network ID. Valid BCST STUN is covered as an OpenBridge traffic gate in both directions; ordinary HBP-to-HBP routing is not the target of that gate. The UDP harness also includes deterministic LinkImpairment scheduling for fake endpoint sends; current scenarios use it to delay sequence 1 behind sequence 2 at a 30 ms cadence, model burst loss and duplicate UDP datagrams, and assert that late out-of-order or duplicate HBP and FBP packets are discarded rather than buffered or replayed. Reusable StreamProfile and ImpairmentProfiles helpers provide named stream and link patterns for more real-world scenarios. Current coverage also includes a multi-stream HBP-to-FBP trunk case where one stream is reordered while another clean stream on the same FBP trunk still routes, plus a generated prompt interruption case where real HBP voice routes after a local TG9 TS2 prompt has started. Negative-path coverage includes malformed short HBP DMRD, malformed short FBP DMRE, bad FBP BLAKE2b hashes, stale FBP timestamps and max-hop FBP packets; these must not leak traffic to HBP targets, and stale/max-hop FBP packets must produce a source-quench response. Selected malformed packet tests also assert subprocess warning logs.

Two UDP tests are marked as expected failures because they document current protocol-version issues rather than fixed behavior: unsupported embedded DMRE packet versions are not yet rejected, and the historical v4 send layout currently carries the module default version byte instead of the configured PROTO_VER value. v4 is characterization/deprecation context, not a long-term protocol contract.

UDP integration tests are opt-in:

FREEDMR_RUN_UDP_TESTS=1 \
PYTHONDONTWRITEBYTECODE=1 \
python -m unittest tests.test_udp_blackbox_harness -v

If dependencies are already installed in another Python, point the harness at it:

FREEDMR_RUN_UDP_TESTS=1 \
FREEDMR_UDP_PYTHON=/path/to/python \
PYTHONDONTWRITEBYTECODE=1 \
python -m unittest tests.test_udp_blackbox_harness -v

To let the harness create a virtualenv and install requirements.txt:

FREEDMR_RUN_UDP_TESTS=1 \
FREEDMR_UDP_BOOTSTRAP_VENV=1 \
PYTHONDONTWRITEBYTECODE=1 \
python -m unittest tests.test_udp_blackbox_harness -v

The venv bootstrap installs requirements.txt into the test virtualenv when the selected Python does not already have the FreeDMR runtime dependencies.

To reuse a persistent test virtualenv:

FREEDMR_RUN_UDP_TESTS=1 \
FREEDMR_UDP_BOOTSTRAP_VENV=1 \
FREEDMR_UDP_VENV_DIR=/tmp/freedmr-blackbox-venv \
PYTHONDONTWRITEBYTECODE=1 \
python -m unittest tests.test_udp_blackbox_harness -v

Full Test Discovery

Run all tests with:

PYTHONDONTWRITEBYTECODE=1 python -m unittest discover -v

The black-box UDP tests still skip unless FREEDMR_RUN_UDP_TESTS=1 is set.

See test-harness-design.md for the harness design and coverage tradeoffs.