3.3 KiB

Raw Permalink Blame History

Testing and Release Gates

FreeDMR 2 must preserve behaviour through tests before changing architecture. The existing deterministic harness, UDP black-box harness, codec tests, support tests, and future live RF validation form the release gate structure.

Test Commands

General test run:

python -m unittest discover -v

Focused support/codec tests:

python -m unittest tests.test_freedmr_dmr_codec tests.test_utils -v

UDP black-box tests:

FREEDMR_RUN_UDP_TESTS=1 python -m unittest tests.test_udp_blackbox_harness -v

UDP black-box tests with venv bootstrap:

FREEDMR_RUN_UDP_TESTS=1 \
FREEDMR_UDP_BOOTSTRAP_VENV=1 \
FREEDMR_UDP_VENV_DIR=/tmp/freedmr-blackbox-venv3 \
PYTHONDONTWRITEBYTECODE=1 \
python -m unittest tests.test_udp_blackbox_harness -v

Release Gates

Level 0: Codec/unit/config/support tests. Must pass for every commit.

Level 1: Deterministic packet/state harness. Must pass before merge to main.

Level 2: Black-box UDP harness. Must pass before release candidate.

Level 3: Live RF / real repeater / real peer validation. Required before changing protocol-visible behaviour.

What Each Layer Proves

Deterministic harness:

Good for packet parsing seams, routing state, dial-a-TG state, fake-clock expiry, rewrite boundaries, LC/embedded LC tests, data-vs-voice classification, and reporting event generation.
Bypasses real UDP, socket binding, subprocess startup, and Twisted timing.

UDP black-box harness:

Good for subprocess startup, HBP login, UDP parsing, FBP signing, bridge-control, malformed/hostile packets, cadence, packet ordering, and link impairment.
Cannot prove RF-side modem/radio behaviour.

Live RF validation:

Required for protocol-visible changes, prompt/ident behaviour, late entry, OVCM/LC options, repeater/radio compatibility, and real terminal quirks.

Required Assertions

Tests should assert:

Route recipients and non-recipients.
Packet byte preservation outside allowed rewrite regions.
Explicit rewrite ranges.
Stream lifecycle.
Subscription state.
Reporting events.
Source quench/STUN behaviour.
Absence of unintended traffic.

Worker/process-boundary Release Gates

Before moving packet-plane behaviour across a process boundary:

Deterministic in-process behaviour must already be covered.
The same scenario must be covered through message-boundary tests.
The same scenario must be covered through UDP black-box tests where observable.
Packet bytes must be compared before and after crossing the process boundary.
Route recipient/non-recipient sets must match.
Allowed rewrite regions must match.
Source quench/STUN/loop-control behaviour must match.
Failure injection must prove worker crash/restart does not replay stale packets.
Reporting/control worker backpressure must not block packet routing.
Live RF validation is required for protocol-visible behaviour.

Suggested future test categories:

Reporting worker crash during active call.
Global exporter outage.
API worker unavailable during normal traffic.
Routing worker restart while stream active.
Routing worker backpressure.
Queue overflow from packet process to reporting worker.
Stale PacketReceived replay prevention.
Duplicate packet prevention after worker restart.
Stream ownership handoff/drain test.
Coordinator restart test, if a coordinator is introduced.

3.3 KiB Raw Permalink Blame History