Simon
1 month ago
parent
af0e1faf4d
commit
865d5a314f
File diff suppressed because it is too large
Load Diff
@ -1,660 +0,0 @@
|
||||
# FreeDMR 2.0 Architecture Decisions
|
||||
|
||||
This file records architectural decisions, requirements, assumptions and open
|
||||
questions driven out during design discussion. It is intended as source material
|
||||
for a later formal FreeDMR 2.0 design document.
|
||||
|
||||
## Project Philosophy
|
||||
|
||||
FreeDMR is open-source, open, intentionally understandable and intentionally
|
||||
simple enough to encourage community implementation, experimentation and
|
||||
operation by radio amateurs.
|
||||
|
||||
HBLink proved that a DMR server could be written in an open, readable way
|
||||
without DMR being gatekept by commercial vendors. FreeDMR takes the next step:
|
||||
it proves that a DMR network can be built this way without central control.
|
||||
Before HBLink and FreeDMR, DMR server software and server-level network
|
||||
membership were typically closed, gatekept or dependent on personal/team
|
||||
approval. FreeDMR exists in part to lower that barrier and give radio amateurs
|
||||
choice and freedom to experiment with global-scale ROIP networking.
|
||||
FreeDMR does not need to gatekeep all private experimentation. The project
|
||||
controls public listing: the process by which servers are shared with Pi-Star
|
||||
and other HBP hotspots as legitimate public access servers. A sysop can run a
|
||||
private server under their own DMR ID and arrange gatewaying with an existing
|
||||
sysop, who effectively vouches for that traffic. Public listing has additional
|
||||
requirements such as connectivity quality, sysop contactability and basic
|
||||
operational expectations.
|
||||
|
||||
The FreeDMR mesh design is influenced by the late Bob Bruninga's APRS ideas,
|
||||
Spanning Tree Protocol and related distributed-network approaches. The project
|
||||
also has a social purpose: bringing together communities and people connected
|
||||
to earlier amateur-radio networking work. FreeDMR is therefore both a technical
|
||||
system and a diplomacy project; design choices must respect operational
|
||||
autonomy, interoperability and trust between independent sysops.
|
||||
|
||||
FreeDMR is successful because it works in the amateur-radio sense: it is best
|
||||
effort, experimental, approachable and deployable on ordinary low-cost systems
|
||||
such as cheap VPS instances and Raspberry Pi-class hardware. It is not intended
|
||||
to be a safety-assured commercial system. FreeDMR 2.0 should improve quality,
|
||||
clarity and scalability without losing the ham-spirit/hacker-philosophy traits
|
||||
that made the network useful and welcoming.
|
||||
|
||||
Design implications:
|
||||
|
||||
- Prefer clear, inspectable protocols over opaque mechanisms.
|
||||
- Keep the implementation understandable by competent sysops and contributors.
|
||||
- Keep the barrier to compatible implementations low where possible.
|
||||
- Preserve low-cost deployment and modest hardware requirements.
|
||||
- Avoid architectural choices that make FreeDMR dependent on heavyweight
|
||||
infrastructure for ordinary single-server operation.
|
||||
- Treat reliability as best-effort resilience appropriate to amateur radio, not
|
||||
as commercial safety assurance.
|
||||
- Preserve server autonomy and local policy.
|
||||
- Avoid unnecessary central control.
|
||||
- Distinguish private operation, vouched/gatewayed traffic and public listing.
|
||||
- Security should protect authenticity and network integrity without hiding
|
||||
amateur-radio traffic.
|
||||
|
||||
## Protected Model
|
||||
|
||||
The protected asset is the FreeDMR operating model, not the old HBLink-derived
|
||||
object structure.
|
||||
|
||||
Preserve:
|
||||
|
||||
- packet model and protocol behaviour
|
||||
- dial-a-TG semantics
|
||||
- TG/DMR-ID centric routing
|
||||
- loop control
|
||||
- source quench
|
||||
- mesh behaviour
|
||||
- practical RF/network tolerance learned from live servers and real RF links
|
||||
- "everything everywhere" principle, subject to documented exceptions
|
||||
|
||||
Replace or redesign where useful:
|
||||
|
||||
- configured `MASTER` stanza as primary runtime identity
|
||||
- proxy-mediated client fan-out
|
||||
- global mutable `BRIDGES` structure as authoritative state
|
||||
- custom dashboard/reporting socket protocol
|
||||
- packet-path coupling to dashboard/API/report consumers
|
||||
|
||||
## Layer Model
|
||||
|
||||
FreeDMR 2.0 should be described as layered:
|
||||
|
||||
- **Access layer**: client/server access protocols such as HBP today and
|
||||
possible future non-trunk client protocols. Owns login/auth/options/keepalive,
|
||||
client sessions, slot state and RF-facing TG presentation.
|
||||
- **Subscription layer**: talkgroup conference membership. Owns direct TG
|
||||
subscriptions, dial-a-TG subscriptions, static/default/user-activated
|
||||
subscriptions, expiry and RF-visible TG to conference TG mapping.
|
||||
- **Mesh layer**: inter-server FBP/OBP/trunk-style behaviour. Owns loop control,
|
||||
source quench, hop/version handling and inter-server conference traffic.
|
||||
- **Reporting layer**: local dashboard, API observers, logs, global lastheard
|
||||
export and state snapshots. Reporting is observational and must not steer
|
||||
packet handling.
|
||||
|
||||
## Reactor and Runtime Migration
|
||||
|
||||
Do not replace Twisted as part of the first FreeDMR 2.0 architecture work.
|
||||
|
||||
Decision:
|
||||
|
||||
- Keep Twisted's single-threaded reactor as a safety boundary initially.
|
||||
- Extract and test the protocol/routing/subscription core behind deterministic
|
||||
interfaces.
|
||||
- Introduce explicit process/message boundaries only after the state model is
|
||||
clear.
|
||||
- Consider asyncio or another event loop only once Twisted has become a thin
|
||||
transport shell around tested core logic.
|
||||
|
||||
Rationale:
|
||||
|
||||
- The current packet behaviour is subtle and validated through real RF/network
|
||||
deployment.
|
||||
- Replacing the event loop while also replacing the state model would mix too
|
||||
many sources of behavioural change.
|
||||
- Twisted's single-threaded reactor helps preserve current ordering assumptions
|
||||
while bridge/subscription and reporting boundaries are made explicit.
|
||||
- The first migration target is architectural clarity and scalability, not event
|
||||
loop novelty.
|
||||
|
||||
## Identity Model
|
||||
|
||||
The configured master/listener is not the client identity.
|
||||
|
||||
FreeDMR 2.0 should move toward:
|
||||
|
||||
- listener identity: UDP socket/service instance
|
||||
- client identity: DMR peer/client ID
|
||||
- subscription identity: client ID + slot + RF-visible TG + conference TG
|
||||
- mesh identity: server/peer/network ID
|
||||
|
||||
Server identity hierarchy:
|
||||
|
||||
- FreeDMR server IDs are 4-digit DMR IDs.
|
||||
- Server sub-IDs are 5-digit IDs derived from the server ID space.
|
||||
- Each sysop/server identity may therefore cover up to 10 server sub-IDs for
|
||||
backend components, larger deployments, failover or fault-tolerant layouts.
|
||||
- Identity verification should cover the base server ID and its authorized
|
||||
sub-IDs rather than requiring unrelated credentials for each sub-ID.
|
||||
|
||||
A single master/listener UDP port should serve an arbitrary number of clients
|
||||
directly, replacing the proxy where possible.
|
||||
|
||||
## Talkgroup Subscription Model
|
||||
|
||||
Conceptually, each TG is a conference bridge. Clients subscribe to conference
|
||||
TGs. FreeDMR does not primarily decide where to send user traffic; users choose
|
||||
the traffic they want to hear by subscription.
|
||||
|
||||
Subscriptions can be:
|
||||
|
||||
- direct TG: RF-visible TG equals conference TG
|
||||
- dial-a-TG: RF-visible TG is currently TG9, conference TG is the selected TG
|
||||
- alias/rewrite: RF-visible TG may be any configured TG, conference TG is the
|
||||
FreeDMR network identity
|
||||
|
||||
Example:
|
||||
|
||||
```python
|
||||
TalkgroupSubscription(
|
||||
client_id=2345001,
|
||||
slot=2,
|
||||
conference_tg=4400,
|
||||
rf_tg=9,
|
||||
mode="dial",
|
||||
active=True,
|
||||
)
|
||||
```
|
||||
|
||||
The invariant is:
|
||||
|
||||
```text
|
||||
conference_tg = FreeDMR network/conference identity
|
||||
rf_tg = client-facing RF presentation identity
|
||||
```
|
||||
|
||||
This makes arbitrary TG rewrites possible without making TG9 structurally
|
||||
special.
|
||||
|
||||
## Bridge Table Replacement
|
||||
|
||||
The legacy `BRIDGES` dict should be replaced internally by subscription-oriented
|
||||
state and indexes. The `"#"` reflector naming convention does not need to be
|
||||
preserved internally; it can be a compatibility/export detail.
|
||||
|
||||
Recommended hot-path structures:
|
||||
|
||||
- `dict` / `set` for O(1)-style local lookups
|
||||
- `typing.NamedTuple` keys for readable hash keys
|
||||
- `dataclass(slots=True)` records for mutable subscription/session state
|
||||
- `heapq` for expiry timers using lazy invalidation
|
||||
|
||||
Recommended indexes:
|
||||
|
||||
```python
|
||||
subscriptions_by_conference_tg[conference_tg] -> set[SubscriptionKey]
|
||||
subscription_by_rf[(client_id, slot, rf_tg)] -> SubscriptionKey
|
||||
subscriptions_by_client_slot[(client_id, slot)] -> set[SubscriptionKey]
|
||||
expiry_heap -> (expires_at, generation, SubscriptionKey)
|
||||
```
|
||||
|
||||
Packet handlers should not scan all subscriptions/bridges to find routing
|
||||
targets.
|
||||
|
||||
## Packet Plane vs Control Plane
|
||||
|
||||
The packet plane is delay-sensitive.
|
||||
|
||||
Packet-plane rules:
|
||||
|
||||
- local in-memory hot state only
|
||||
- no external database round trips
|
||||
- no blocking API/dashboard/report calls
|
||||
- no cross-process lock waits
|
||||
- no dependency on reporting consumers being connected
|
||||
|
||||
External stores may be used for:
|
||||
|
||||
- config distribution
|
||||
- API/dashboard state
|
||||
- control-plane coordination
|
||||
- snapshots
|
||||
- global lastheard export
|
||||
- optional clustering/multi-process coordination
|
||||
|
||||
General performance principle:
|
||||
|
||||
- Expensive processing should be considered for offload to separate processes
|
||||
because CPython execution is constrained by the GIL for CPU-bound Python code.
|
||||
- Offload is appropriate for reporting fanout, global export, dashboard
|
||||
aggregation, historical database writes, heavy analytics, expensive
|
||||
transcoding/codec experiments and non-critical maintenance jobs.
|
||||
- Offload boundaries must be asynchronous from the packet path. If an offload
|
||||
worker is slow or unavailable, packet handling must continue with local state.
|
||||
- Do not offload hot-path routing decisions if doing so would add inter-process,
|
||||
network or lock waits to every packet.
|
||||
|
||||
## DMR Data Packet Policy
|
||||
|
||||
FreeDMR must maintain DMR data packet forwarding support.
|
||||
|
||||
Decision:
|
||||
|
||||
- FreeDMR should forward supported DMR data packets according to the same
|
||||
conference/subscription and mesh principles as other traffic.
|
||||
- There must be no regression in existing data packet forwarding support.
|
||||
- FreeDMR core should not become an application-level DMR data processor.
|
||||
- GPS, SMS and similar application processing should be implemented by systems
|
||||
connected via FBP or another mesh/access-adjacent interface.
|
||||
- `DATA_GATEWAY` is understood as an earlier expression of this model: an FBP
|
||||
link that carries data-oriented traffic rather than ordinary voice traffic.
|
||||
- Existing `SUB_MAP` behaviour is intentional: data addressed to a DMR ID can be
|
||||
routed toward the last known HBP/client location for that DMR ID.
|
||||
|
||||
Core FreeDMR may inspect/classify data packets only as needed for:
|
||||
|
||||
- packet admission and protocol validation
|
||||
- routing/subscription decisions
|
||||
- loop control and source quench
|
||||
- reporting/logging
|
||||
- preserving packet bytes and metadata across FBP/HBP boundaries
|
||||
- maintaining the subscriber location map needed for data-client routing
|
||||
|
||||
Possible narrow exceptions:
|
||||
|
||||
- dial-a-TG control via DMR SMS
|
||||
- DMR SMS alerts from a server to a sysop
|
||||
|
||||
Any such exceptions must be explicit control-plane features and must not turn
|
||||
FreeDMR core into a general GPS/SMS application processor.
|
||||
|
||||
## Mesh Peer Authentication
|
||||
|
||||
FreeDMR should only accept mesh/FBP traffic from servers that can be validated
|
||||
as legitimate members of the network.
|
||||
|
||||
Core principle:
|
||||
|
||||
- FreeDMR may sign/authenticate traffic and control messages, but should not
|
||||
encrypt amateur-radio traffic or mesh traffic by default.
|
||||
- Amateur radio is public in most jurisdictions and encryption is often not
|
||||
permitted. FreeDMR users may also carry IP backhaul over amateur radio links.
|
||||
- FreeDMR's security model is authenticity, integrity, membership validation and
|
||||
local policy enforcement, not secrecy.
|
||||
- This follows the existing FreeDMR principle, agreed historically by project
|
||||
maintainers, that the network has nothing to hide and should remain cleartext.
|
||||
|
||||
Identity/listing distinction:
|
||||
|
||||
- Signed mesh identity should prove a server/sysop identity or a vouching
|
||||
relationship. It should not automatically imply public listing.
|
||||
- Public listing is a directory/discovery decision for clients and HBP hotspots.
|
||||
- A public access server may need stronger operational requirements than a
|
||||
private or gatewayed server.
|
||||
- Local sysops may still choose whether to carry/vouch for traffic from private
|
||||
servers, even when those servers are not publicly listed.
|
||||
- If an individual 7-digit DMR ID is used as a server identity, traffic may pass
|
||||
when a directly connected/listed sysop chooses to allow and gateway it.
|
||||
- The vouching sysop is accountable to their peers for traffic they forward. If
|
||||
that traffic harms the network, peers may choose to stop peering with the
|
||||
vouching server. This preserves a self-policing social mechanism without
|
||||
requiring central control for all private experimentation.
|
||||
|
||||
Analogue network bridges:
|
||||
|
||||
- Analogue ROIP/network bridges commonly connect as if they are DMR clients via
|
||||
HBP.
|
||||
- FreeDMR permits this and is generally more permissive than many other DMR
|
||||
networks.
|
||||
- FreeDMR works with/supports the DVSwitch community on this. DVSwitch provides
|
||||
a common mechanism by which analogue networks can be bridged into DMR-style
|
||||
access.
|
||||
- These bridges are operationally sensitive: technical limitations can make
|
||||
them effectively listen-only, consuming CPU and bandwidth while adding little
|
||||
value if they do not contribute actual two-way user activity.
|
||||
- Analogue bridges are often implemented using audio mixing/conference style
|
||||
behaviour. This is a poor fit for DMR and similar digital modes, which enforce
|
||||
one audio source at a time and rely on stream, hang-time and contention
|
||||
behaviour rather than mixed audio.
|
||||
- This mismatch comes partly from analogue repeater heritage: analogue systems
|
||||
may maintain a continuous transmit carrier and mix notification sounds such as
|
||||
pips, CWID and courtesy tones into the output audio. Analogue systems also
|
||||
often have little or no strong source identity, whereas DMR traffic carries a
|
||||
DMR ID.
|
||||
- A common failure mode is that a feed from an analogue repeater keeps the DMR
|
||||
stream open between analogue overs, plays courtesy/notification tones and then
|
||||
carries the next analogue user in the same held stream. This can hold the TG
|
||||
open and prevent a digital station from breaking in until the analogue
|
||||
repeater times out and its carrier drops.
|
||||
- Analogue bridges should therefore be subject to local sysop policy, public
|
||||
listing expectations and peer accountability. Permitted does not mean
|
||||
automatically valuable or immune from peering/listing consequences.
|
||||
|
||||
Other digital network bridges:
|
||||
|
||||
- Digital voice networks such as YSF and NXDN are generally a better technical
|
||||
match for DMR than analogue networks because they also use AMBE-family vocoder
|
||||
audio.
|
||||
- AMBE-to-AMBE interworking can be lossless at the codec level and avoids
|
||||
transcoding artifacts.
|
||||
- Transcoding from analogue or unlike codecs can degrade audio quality
|
||||
significantly and should be treated carefully.
|
||||
|
||||
Desired direction:
|
||||
|
||||
- Add PKI-backed mesh peer admission to the Bridge Control (`BCXX`) mechanism.
|
||||
- A peer server presents public identity material signed by a FreeDMR network
|
||||
master key or trusted network CA.
|
||||
- The authenticated identity must bind at least:
|
||||
- server ID
|
||||
- authorized server sub-IDs
|
||||
- public key
|
||||
- validity period
|
||||
- permitted protocol/features where useful
|
||||
- Runtime admission should bind the authenticated server identity to the
|
||||
observed transport endpoint, including IP address.
|
||||
- If the observed IP address changes, the FBP peer must perform a new key
|
||||
exchange/authentication step before its traffic is forwarded.
|
||||
- Network membership should be represented by a signed sysop/server key that is
|
||||
issued when the sysop/server joins the network and revoked when they leave or
|
||||
are compromised. Runtime endpoint/session bindings are renewed separately and
|
||||
do not require re-signing the long-lived membership key.
|
||||
- One successful verification of the signed identity should authorize the
|
||||
covered server ID and declared/authorized sub-IDs for that sysop, subject to
|
||||
local policy and endpoint/session binding.
|
||||
|
||||
Packet-plane rule:
|
||||
|
||||
- Expensive signature/certificate validation happens during control-plane
|
||||
admission or re-admission, not for every DMR packet.
|
||||
- Per-packet mesh traffic should use a cached authenticated peer/session state
|
||||
check keyed by server ID and endpoint.
|
||||
|
||||
Initial conceptual flow:
|
||||
|
||||
```text
|
||||
FBP peer connects/sends keepalive
|
||||
-> BC auth exchange presents signed server identity/public key
|
||||
-> FreeDMR validates signature against trusted network key
|
||||
-> FreeDMR binds server_id + endpoint + protocol features to peer session
|
||||
-> DMR traffic is accepted only while that authenticated binding is valid
|
||||
```
|
||||
|
||||
Security requirements:
|
||||
|
||||
- Reject unauthenticated FBP traffic by default once this mode is enabled.
|
||||
- Reject traffic where server ID, key identity and source endpoint do not match
|
||||
the authenticated binding.
|
||||
- Expire authenticated bindings and require renewal.
|
||||
- Support soft renewal: when an authenticated binding reaches its renewal
|
||||
timestamp, schedule asynchronous re-authentication while allowing a bounded
|
||||
grace period so in-flight voice is not interrupted purely by renewal timing.
|
||||
- Hard-stop forwarding only for explicit authentication failure, revoked
|
||||
identity/key, endpoint mismatch outside policy, expired grace period, or
|
||||
policy requiring immediate re-authentication.
|
||||
- Log authentication failure reasons clearly without leaking private material.
|
||||
- Provide a controlled transition mode for existing networks while PKI is rolled
|
||||
out.
|
||||
|
||||
Open questions:
|
||||
|
||||
- Whether to use X.509 certificates, raw Ed25519 public keys with signed
|
||||
metadata, or another compact identity format.
|
||||
- How network master keys/CAs are generated, rotated and revoked.
|
||||
- Whether peer authorization policy should live in config, MQTT/control-plane
|
||||
state, or a signed network membership list.
|
||||
- How to handle legitimate dynamic-IP servers without weakening endpoint
|
||||
binding.
|
||||
- What renewal and grace-period defaults best preserve voice continuity without
|
||||
weakening mesh admission.
|
||||
|
||||
### Distributed Key Gossip Option
|
||||
|
||||
FreeDMR may also use a peer-to-peer signed-key dissemination mechanism over the
|
||||
Bridge Control (`BCXX`) out-of-band channel.
|
||||
|
||||
Concept:
|
||||
|
||||
- Each server periodically advertises the signed server public keys/membership
|
||||
documents it knows to its direct FBP peers.
|
||||
- Peers validate the signatures and build a local table of legitimate server
|
||||
identities as knowledge propagates through the mesh.
|
||||
- Each server uses its local signed-key table and local policy to decide whether
|
||||
to route or reject packets that originated from a given source server, even
|
||||
when that source server is not directly connected.
|
||||
|
||||
Rationale:
|
||||
|
||||
- FreeDMR is a peer network, not hub-and-spoke or master/slave.
|
||||
- Servers are autonomous and independently operated.
|
||||
- Direct FBP peers should not be blindly trusted to make correct routing
|
||||
decisions on behalf of the local server.
|
||||
- Open-source, human-readable code deliberately lowers the barrier to
|
||||
modification, so each server must be able to protect itself from incorrect or
|
||||
malicious upstream forwarding decisions.
|
||||
|
||||
Security requirements for key gossip:
|
||||
|
||||
- Only signed membership documents are accepted; peers cannot create trust by
|
||||
merely repeating a key.
|
||||
- Membership documents need issuer, subject server ID, public key fingerprint,
|
||||
authorized sub-IDs, validity period, serial/version and signature.
|
||||
- Revocation data must propagate by the same or a stronger mechanism.
|
||||
- Each server must enforce local policy after validation. A valid signed key
|
||||
proves membership, not mandatory carriage.
|
||||
- Key gossip must be rate-limited and bounded so it cannot become a BCXX flood
|
||||
or memory-growth vector.
|
||||
- Received membership data must be replay-resistant enough to handle expiry,
|
||||
superseded serials and revoked keys.
|
||||
- The packet path must use cached key/policy state; signature validation and
|
||||
gossip processing are control-plane work.
|
||||
|
||||
This complements direct-peer endpoint authentication. Direct-peer auth proves
|
||||
the connected FBP peer is legitimate for this session; distributed signed-key
|
||||
knowledge lets the local server make autonomous decisions about traffic whose
|
||||
source server is elsewhere in the mesh.
|
||||
|
||||
## Reporting Protocol Decision
|
||||
|
||||
FreeDMR 2.0 should define a structured reporting event protocol and use MQTT as
|
||||
the preferred external live reporting transport.
|
||||
|
||||
Rationale:
|
||||
|
||||
- MQTT is already familiar in DMR network dashboard/reporting contexts.
|
||||
- BrandMeister uses MQTT, providing a useful precedent for dashboard consumers.
|
||||
- MQTT topics map naturally to server/client/subscription/call state.
|
||||
- Retained messages are useful for current state snapshots.
|
||||
- Last Will and Testament can represent server/reporting disconnects.
|
||||
- MQTT-over-WebSocket allows browser dashboards to subscribe directly when the
|
||||
broker supports it.
|
||||
|
||||
Constraints:
|
||||
|
||||
- MQTT publishing must be asynchronous from the packet worker.
|
||||
- Packet routing must continue if the MQTT broker/dashboard is down.
|
||||
- Event generation must be state-change/summary oriented, not per DMR frame.
|
||||
- The event schema is the compatibility contract; internal Python objects are
|
||||
not.
|
||||
- Local live dashboard and central global lastheard remain separate paths.
|
||||
- Voice stability takes precedence over reporting completeness. If the system
|
||||
must choose between dropping/reporting-losing events and delaying packet
|
||||
handling, it must drop or coalesce reporting events.
|
||||
|
||||
Implementation requirement:
|
||||
|
||||
```text
|
||||
packet path -> non-blocking local event queue -> MQTT publisher worker
|
||||
```
|
||||
|
||||
The packet path must not call an MQTT broker synchronously. The local event
|
||||
queue should be bounded. On overflow, the publisher layer should drop or
|
||||
coalesce low-priority events and emit a later reporting-health event rather than
|
||||
blocking packet handling.
|
||||
|
||||
Suggested event priority:
|
||||
|
||||
- retain/coalesce latest state: server/client/slot/subscription state
|
||||
- keep best effort: call start/end summaries
|
||||
- drop first under pressure: high-volume debug/warning/statistical updates
|
||||
|
||||
MQTT publishing should support reconnect with exponential backoff and should
|
||||
refresh retained state after reconnect so a dashboard can recover even if
|
||||
transient events were missed.
|
||||
|
||||
Suggested MQTT namespace:
|
||||
|
||||
```text
|
||||
freedmr/v2/{server_id}/state
|
||||
freedmr/v2/{server_id}/client/{client_id}/state
|
||||
freedmr/v2/{server_id}/client/{client_id}/slot/{slot}/activity
|
||||
freedmr/v2/{server_id}/subscription/{subscription_id}/state
|
||||
freedmr/v2/{server_id}/call/{stream_id}/start
|
||||
freedmr/v2/{server_id}/call/{stream_id}/end
|
||||
freedmr/v2/{server_id}/mesh/{peer_id}/state
|
||||
freedmr/v2/{server_id}/event
|
||||
```
|
||||
|
||||
Use retained messages for current state:
|
||||
|
||||
```text
|
||||
server state
|
||||
client state
|
||||
slot activity
|
||||
subscription state
|
||||
mesh peer state
|
||||
```
|
||||
|
||||
Use non-retained messages for transient events:
|
||||
|
||||
```text
|
||||
call start/end
|
||||
loop-control event
|
||||
source-quench event
|
||||
packet-rate/loss summary
|
||||
warnings
|
||||
```
|
||||
|
||||
Example event:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 2,
|
||||
"event_id": 1849281,
|
||||
"type": "call.started",
|
||||
"timestamp": 1710000000.123,
|
||||
"server_id": 234099,
|
||||
"client_id": 2345001,
|
||||
"slot": 2,
|
||||
"conference_tg": 4400,
|
||||
"rf_tg": 9,
|
||||
"source_id": 2351234,
|
||||
"stream_id": 16909060,
|
||||
"access": "hbp"
|
||||
}
|
||||
```
|
||||
|
||||
Dashboard delivery options:
|
||||
|
||||
- preferred: dashboard subscribes to MQTT over WebSocket
|
||||
- alternative: local reporting sidecar translates MQTT to SSE/HTTP
|
||||
- control actions should use authenticated HTTP APIs unless a future UI needs
|
||||
bidirectional streaming
|
||||
|
||||
## Local Dashboard vs Global Lastheard
|
||||
|
||||
Each FreeDMR server has its own local live dashboard. The global lastheard
|
||||
service is centrally hosted and non-real-time.
|
||||
|
||||
Local dashboard:
|
||||
|
||||
- consumes local MQTT live state/events
|
||||
- displays current client/repeater traffic
|
||||
- must tolerate reconnects and missed transient events by reloading retained
|
||||
state topics
|
||||
|
||||
Global lastheard:
|
||||
|
||||
- consumes call summaries or batched exports
|
||||
- should not depend on packet-plane or dashboard delivery
|
||||
- should tolerate central outage via spool/retry
|
||||
|
||||
Possible MQTT global feed:
|
||||
|
||||
- Each server publishes local live dashboard topics to a local broker or local
|
||||
reporting service.
|
||||
- Prefer a separate exporter process for the curated global feed. The exporter
|
||||
subscribes to the same local real-time MQTT feed as the dashboard, filters and
|
||||
summarizes what is needed, then publishes to the network MQTT broker or writes
|
||||
to the global collector.
|
||||
- The exporter publishes only summary topics needed for the 30-day database,
|
||||
such as call end summaries, client/server presence, selected mesh health and
|
||||
selected subscription changes.
|
||||
- Raw packet events and high-volume live slot updates should not be exported to
|
||||
the global broker by default.
|
||||
- Central broker, global dashboard or exporter failure must not back up into
|
||||
local packet processing or local dashboard state.
|
||||
|
||||
Preferred flow:
|
||||
|
||||
```text
|
||||
FreeDMR core -> local MQTT feed -> local dashboard
|
||||
-> global-exporter process -> network MQTT/collector
|
||||
```
|
||||
|
||||
Core publishing invariant:
|
||||
|
||||
- FreeDMR core emits each reporting event once to its configured local MQTT
|
||||
broker/publisher queue.
|
||||
- Fanout to dashboards, exporters, automation and global collectors is handled
|
||||
by the MQTT broker and separate subscriber processes.
|
||||
- Adding more reporting consumers must not increase FreeDMR packet-process work
|
||||
beyond the single local event emission.
|
||||
|
||||
Suggested global MQTT subjects:
|
||||
|
||||
```text
|
||||
freedmr/v2/global/{server_id}/call/end
|
||||
freedmr/v2/global/{server_id}/client/state
|
||||
freedmr/v2/global/{server_id}/server/state
|
||||
freedmr/v2/global/{server_id}/mesh/state
|
||||
```
|
||||
|
||||
## Reporting Event Types
|
||||
|
||||
Initial event families:
|
||||
|
||||
```text
|
||||
server.started
|
||||
server.stopping
|
||||
client.connected
|
||||
client.disconnected
|
||||
client.options_changed
|
||||
subscription.activated
|
||||
subscription.deactivated
|
||||
subscription.expired
|
||||
call.started
|
||||
call.ended
|
||||
call.lost
|
||||
mesh.peer_up
|
||||
mesh.peer_down
|
||||
mesh.source_quench
|
||||
loop.detected
|
||||
packet.rate_limited
|
||||
```
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Which MQTT broker should be packaged by default: Mosquitto, EMQX, NATS MQTT
|
||||
compatibility, or another option?
|
||||
- Should MQTT be mandatory for FreeDMR 2.0 dashboards, or optional with an
|
||||
embedded/local fallback?
|
||||
- What authentication/authorization model should protect MQTT topics and
|
||||
dashboard control APIs?
|
||||
- What retained-topic expiry policy should be used to prevent stale state?
|
||||
- Should global lastheard consume MQTT directly or use a separate HTTP/queue
|
||||
exporter fed from reporting events?
|
||||
- Should FreeDMR expose a legacy `BRIDGES` compatibility view during migration?
|
||||
@ -1,65 +0,0 @@
|
||||
# FreeDMR 2 Glossary
|
||||
|
||||
This glossary defines FreeDMR 2 terms. Legacy terms such as `SYSTEM`, `MASTER`, `BRIDGES`, and `#` reflector names describe current implementation details or compatibility views, not the primary FreeDMR 2 model.
|
||||
|
||||
**Access layer**: The part of FreeDMR that accepts client/repeater protocols such as HBP. It owns login, authentication, options, keepalive, access sessions, RF-facing slots, and RF-visible TG presentation.
|
||||
|
||||
**Access session**: A live connection-like relationship between FreeDMR and a client/repeater. It is identified by client DMR ID and transport/session metadata, not by a configured listener stanza.
|
||||
|
||||
**Listener**: A UDP socket/service endpoint that accepts one or more access sessions. A configured `MASTER` is a legacy listener-like concept, not the client identity.
|
||||
|
||||
**Client/repeater**: An HBP hotspot, repeater, gateway, bridge, or future access-side system connected to FreeDMR.
|
||||
|
||||
**Client DMR ID**: The DMR ID used by a client/repeater access session. Future routing should key primarily on client DMR ID, slot, and RF-visible TG.
|
||||
|
||||
**RF-visible TG**: The talkgroup number seen by the RF terminal or access-side device. In dial-a-TG this may be TG9 while the network conference TG is different.
|
||||
|
||||
**Conference TG**: The FreeDMR network talkgroup identity. Conceptually this is the conference bridge to which clients subscribe.
|
||||
|
||||
**Subscription**: Membership of a client slot/RF TG presentation in a conference TG.
|
||||
|
||||
**Static subscription**: A subscription created by configuration or policy and normally present for the session.
|
||||
|
||||
**Dial-a-TG subscription**: A subscription created by dial-a-TG control. It maps an RF-visible TG, traditionally TG9, to a selected conference TG.
|
||||
|
||||
**Default reflector**: A configured or client-requested default dial-a-TG style subscription for a session. Empty string, integer 0, or boolean false means no default reflector.
|
||||
|
||||
**Stream**: A voice call flow identified by stream metadata. AMBE voice is a stream; DMR data packets are packet-oriented and should not be treated as AMBE streams.
|
||||
|
||||
**Source server**: The server that originated or advertised the packet into the mesh, according to the FBP/OBP protocol version in use.
|
||||
|
||||
**Source repeater**: The access-side repeater/client identity carried when the protocol version supports it.
|
||||
|
||||
**Mesh peer**: Another FreeDMR/OpenBridge/FBP peer server connected through the mesh layer.
|
||||
|
||||
**Server ID**: A FreeDMR server identity. Server IDs are treated separately from client DMR IDs.
|
||||
|
||||
**Server sub-ID**: A subordinate server identity authorized under a server/sysop identity, for example backend or fault-tolerant deployments.
|
||||
|
||||
**Bridge-control message**: An out-of-band FBP/OBP control message, such as source quench or STUN.
|
||||
|
||||
**Packet plane**: The delay-sensitive path that receives, parses, routes, mutates where necessary, and sends DMR packets.
|
||||
|
||||
**Control plane**: Authenticated configuration, API, bridge-control, admission, and policy operations.
|
||||
|
||||
**Reporting plane**: Observational events and state export for dashboards, logs, lastheard, monitoring, and operators.
|
||||
|
||||
**Compatibility/export state**: A derived view that presents old FreeDMR/HBLink/dashboard shapes to consumers. It is not authoritative state.
|
||||
|
||||
**HBP**: Homebrew Protocol, used by many hotspots/repeaters on the access side.
|
||||
|
||||
**OBP**: OpenBridge Protocol version 1. It remains important as an open interop path where intentionally configured.
|
||||
|
||||
**FBP**: FreeDMR Bridge Protocol. Any OpenBridge-derived peer protocol version higher than 1 is termed FBP for FreeDMR clarity.
|
||||
|
||||
**DATA-GATEWAY**: A historical/early expression of a data-oriented FBP link. FreeDMR 2 should preserve data forwarding without making the core a GPS/SMS application processor.
|
||||
|
||||
**Source quench / BCSQ**: A bridge-control hint asking a peer to suppress a stream/TG toward us. It is optional and per stream/TG.
|
||||
|
||||
**STUN / BCST**: A broader bridge-control gate intended to stop all FBP traffic from a peer under the current conceptual model.
|
||||
|
||||
**OVCM**: Open Voice Channel Mode. ETSI service option bit 0x04 when explicitly used. HBLink legacy 0x20 is compatibility history, not standards-clean OVCM.
|
||||
|
||||
**Synthetic LC**: A generated Link Control value used when FreeDMR has to create fallback LC information.
|
||||
|
||||
**Real inbound LC**: LC decoded from received traffic. It should be preserved unchanged unless FreeDMR deliberately rewrites it.
|
||||
@ -1,41 +0,0 @@
|
||||
# FreeDMR 2 System Model
|
||||
|
||||
FreeDMR 2 is a layered system. The layers are design boundaries, not necessarily separate processes at first.
|
||||
|
||||
## Access Layer
|
||||
|
||||
The access layer owns HBP and future client/repeater protocols. It handles login, authentication, options, keepalive, access sessions, RF-facing slot state, and RF-visible TG presentation.
|
||||
|
||||
A configured listener is not the client identity. A single listener should eventually support multiple clients directly, replacing proxy-mediated fan-out where possible.
|
||||
|
||||
## Subscription Layer
|
||||
|
||||
The subscription layer owns talkgroup conference membership. It handles direct TG subscriptions, dial-a-TG subscriptions, static subscriptions, default reflectors, user/API/SMS activated subscriptions, expiry, and RF-visible TG to conference TG mapping.
|
||||
|
||||
Packet routing should consume subscription state. It should not need to know whether a subscription came from static config, dial-a-TG, API, SMS, or a future UI.
|
||||
|
||||
## Mesh Layer
|
||||
|
||||
The mesh layer owns FBP/OBP/trunk-style inter-server traffic. It handles loop control, source quench, hop/version handling, bridge control, source server/repeater metadata, and conference traffic between servers.
|
||||
|
||||
FreeDMR remains a peer network, not hub-and-spoke. Local sysops retain local routing and policy autonomy.
|
||||
|
||||
## Packet/Stream Layer
|
||||
|
||||
The packet/stream layer owns packet parsing, stream lifecycle, sequence handling, terminators, LC/embedded LC handling, data-vs-voice classification, and packet mutation boundaries.
|
||||
|
||||
Raw packet bytes are immutable input until an explicit named rewrite operation occurs.
|
||||
|
||||
## Reporting Layer
|
||||
|
||||
The reporting layer is observational only. It emits state and events to local dashboards, global lastheard exporters, logs, and monitoring consumers.
|
||||
|
||||
Reporting must not steer packet routing.
|
||||
|
||||
## Control/API Layer
|
||||
|
||||
The control/API layer provides explicit authenticated operations for sysop and control-plane actions. It should operate on access sessions, subscriptions, mesh peers, and reporting state without blocking the packet path.
|
||||
|
||||
## Critical Invariant
|
||||
|
||||
Reporting, dashboards, APIs, databases, exporters, and monitoring consumers must not block or steer packet handling.
|
||||
@ -1,251 +0,0 @@
|
||||
# FreeDMR 2 State Model
|
||||
|
||||
The FreeDMR 2 state model separates listener identity, client identity, subscription state, stream state, mesh peer state, and reporting/export views. The legacy `BRIDGES` dict is not the authoritative FreeDMR 2 model.
|
||||
|
||||
Recommended hot-path structures:
|
||||
|
||||
- `NamedTuple` or tuple keys for hot dict/set indexes.
|
||||
- `dataclass(slots=True)` for mutable state records.
|
||||
- `heapq` expiry queue with lazy invalidation.
|
||||
- Local in-memory packet-plane state.
|
||||
- No packet-path dependency on external databases or reporting consumers.
|
||||
|
||||
Suggested indexes:
|
||||
|
||||
```python
|
||||
subscriptions_by_conference_tg[conference_tg] -> set[SubscriptionKey]
|
||||
subscription_by_rf[(client_id, slot, rf_tg)] -> SubscriptionKey
|
||||
subscriptions_by_client_slot[(client_id, slot)] -> set[SubscriptionKey]
|
||||
expiry_heap -> (expires_at, generation, SubscriptionKey)
|
||||
```
|
||||
|
||||
## AccessSession
|
||||
|
||||
Purpose: Represents a live client/repeater session.
|
||||
|
||||
Owner: Access layer.
|
||||
|
||||
Key: Client DMR ID plus listener/session endpoint data.
|
||||
|
||||
Mutable fields: Authentication state, options, keepalive time, endpoint, active slots, supported protocol features.
|
||||
|
||||
Expiry/timer behaviour: Keepalive/session timeout expires the session and resets session-scoped options to system defaults.
|
||||
|
||||
Packet-plane rules: Read for packet admission, option interpretation, slot state, and source identity. Writes only minimal hot state such as last packet/keepalive.
|
||||
|
||||
Control-plane rules: API may read and update bounded session options.
|
||||
|
||||
Reporting/export view: Client connected/disconnected/options/state events.
|
||||
|
||||
Compatibility mapping: Current configured `MASTER`/`SYSTEM` session fields and peer status entries.
|
||||
|
||||
## Listener
|
||||
|
||||
Purpose: Owns a UDP socket/service endpoint.
|
||||
|
||||
Owner: Access layer or transport shell.
|
||||
|
||||
Key: Listener name or bind address/port.
|
||||
|
||||
Mutable fields: Socket state, configured admission policy, active access sessions.
|
||||
|
||||
Expiry/timer behaviour: None beyond transport lifecycle.
|
||||
|
||||
Packet-plane rules: Receives and sends packets; should not be treated as client identity.
|
||||
|
||||
Control-plane rules: Configuration and lifecycle only.
|
||||
|
||||
Reporting/export view: Listener up/down and session counts.
|
||||
|
||||
Compatibility mapping: Current `MASTER` stanza.
|
||||
|
||||
## ClientSlotState
|
||||
|
||||
Purpose: Tracks per-client per-slot RF-facing state.
|
||||
|
||||
Owner: Access and subscription layers.
|
||||
|
||||
Key: `(client_id, slot)`.
|
||||
|
||||
Mutable fields: RF-visible TG activity, current stream, hang/timeout state, default reflector, static and dial subscriptions.
|
||||
|
||||
Expiry/timer behaviour: Stream timers, dial/default reflector expiry, session reset on disconnect.
|
||||
|
||||
Packet-plane rules: Read for RF-visible TG mapping and stream admission. Writes current stream and observed activity.
|
||||
|
||||
Control-plane rules: API may activate/deactivate subscriptions and defaults.
|
||||
|
||||
Reporting/export view: Slot activity and active subscription state.
|
||||
|
||||
Compatibility mapping: Existing slot options, dial-a-TG state, timeout fields.
|
||||
|
||||
## TalkgroupSubscription
|
||||
|
||||
Purpose: Represents membership of a client slot/RF TG in a conference TG.
|
||||
|
||||
Owner: Subscription layer.
|
||||
|
||||
Key: Stable `SubscriptionKey`, likely `(client_id, slot, rf_tg, conference_tg, mode)`.
|
||||
|
||||
Mutable fields: Active flag, source, expiry, generation, priority/policy metadata.
|
||||
|
||||
Expiry/timer behaviour: Static subscriptions normally session-bound; dial/default/user subscriptions may expire.
|
||||
|
||||
Packet-plane rules: Read heavily for routing. Writes should be explicit activation/deactivation/expiry only.
|
||||
|
||||
Control-plane rules: API and bridge control may create/remove/update subscriptions subject to policy.
|
||||
|
||||
Reporting/export view: Subscription activated/deactivated/expired events.
|
||||
|
||||
Compatibility mapping: Current `BRIDGES` entries and `#` reflector export names.
|
||||
|
||||
## StreamState
|
||||
|
||||
Purpose: Tracks voice stream lifecycle and packet ordering.
|
||||
|
||||
Owner: Packet/stream layer.
|
||||
|
||||
Key: Stream ID plus source identity and direction namespace where needed.
|
||||
|
||||
Mutable fields: Source ID, destination/conference TG, RF-visible TG when relevant, slot, last sequence, last packet time, LC state, source server/repeater metadata, loop-control state.
|
||||
|
||||
Expiry/timer behaviour: Explicit terminator is strong end; timeout is softer, especially on HBP.
|
||||
|
||||
Packet-plane rules: Read/write in packet path. Must be local and fast.
|
||||
|
||||
Control-plane rules: Normally read-only, except explicit reset/debug operations.
|
||||
|
||||
Reporting/export view: Call started/ended/lost events.
|
||||
|
||||
Compatibility mapping: Current stream tracking dicts and report socket call state.
|
||||
|
||||
## MeshPeerState
|
||||
|
||||
Purpose: Tracks a peer server/link.
|
||||
|
||||
Owner: Mesh layer.
|
||||
|
||||
Key: Peer/server ID and authenticated endpoint/session where available.
|
||||
|
||||
Mutable fields: Protocol version, endpoint, auth state, last seen, stun/quench state, supported metadata layout, send/receive counters.
|
||||
|
||||
Expiry/timer behaviour: Peer keepalive/control timeout; auth renewal timers.
|
||||
|
||||
Packet-plane rules: Read for admission, protocol layout, and source metadata. Writes last-seen counters and cached safety state only.
|
||||
|
||||
Control-plane rules: API/BCXX may stun, clear stun, authenticate, or update policy.
|
||||
|
||||
Reporting/export view: Peer up/down/stun/source-quench events.
|
||||
|
||||
Compatibility mapping: Current OBP/FBP peer entries.
|
||||
|
||||
## BridgeControlState
|
||||
|
||||
Purpose: Holds bridge-control effects such as source quench, STUN, and authentication state.
|
||||
|
||||
Owner: Mesh/control layer.
|
||||
|
||||
Key: Peer ID plus control scope, for example `(peer_id, stream_id, conference_tg)` for BCSQ.
|
||||
|
||||
Mutable fields: Active flag, reason, expiry, generation, authenticated issuer.
|
||||
|
||||
Expiry/timer behaviour: Source quench and soft controls should expire; hard policy blocks may persist until cleared.
|
||||
|
||||
Packet-plane rules: Read for admission/suppression. Writes only when handling bridge-control packets.
|
||||
|
||||
Control-plane rules: API/BCXX may set or clear controls.
|
||||
|
||||
Reporting/export view: Mesh control events.
|
||||
|
||||
Compatibility mapping: Current BCSQ/BCST handling.
|
||||
|
||||
## ReportingState
|
||||
|
||||
Purpose: Tracks reporting pipeline health and retained current state.
|
||||
|
||||
Owner: Reporting layer.
|
||||
|
||||
Key: Event family or retained state identity.
|
||||
|
||||
Mutable fields: Queue depth, dropped counts, publisher connected state, last emitted state.
|
||||
|
||||
Expiry/timer behaviour: Retained state refresh and reconnect backoff.
|
||||
|
||||
Packet-plane rules: Packet path may enqueue non-blocking events only.
|
||||
|
||||
Control-plane rules: API may read reporting health.
|
||||
|
||||
Reporting/export view: Native v2 reporting state.
|
||||
|
||||
Compatibility mapping: Current dashboard socket state, preferably through a sidecar adapter.
|
||||
|
||||
## CompatibilityExportState
|
||||
|
||||
Purpose: Derived legacy-shaped view for old dashboard/API/HBLink-compatible consumers.
|
||||
|
||||
Owner: Compatibility adapter.
|
||||
|
||||
Key: Consumer-specific.
|
||||
|
||||
Mutable fields: Cached translated state.
|
||||
|
||||
Expiry/timer behaviour: Follows source state; may drop stale export entries.
|
||||
|
||||
Packet-plane rules: Must not be read by packet routing.
|
||||
|
||||
Control-plane rules: May expose old-compatible admin views if required.
|
||||
|
||||
Reporting/export view: Legacy compatibility only.
|
||||
|
||||
Compatibility mapping: `BRIDGES`, `SYSTEM`, `MASTER`, and `#` reflector names.
|
||||
|
||||
## Worker Ownership Considerations
|
||||
|
||||
Authoritative packet-plane state must have one owner. Reporting/export state is derived and must not drive routing. External stores may distribute snapshots or control-plane updates, but they are not per-packet routing dependencies.
|
||||
|
||||
Process boundaries must preserve the same state ownership rules as in-process modules.
|
||||
|
||||
AccessSession:
|
||||
|
||||
- Classification: Access/session state with packet-plane admission impact.
|
||||
- Likely owner: Transport/listener process initially.
|
||||
- Future ownership: May be assigned to a routing worker once admitted.
|
||||
|
||||
ClientSlotState:
|
||||
|
||||
- Classification: Packet-plane authoritative state.
|
||||
- Likely owner: Single routing owner for that client/slot.
|
||||
|
||||
TalkgroupSubscription:
|
||||
|
||||
- Classification: Packet-plane authoritative state.
|
||||
- Likely owner: Single routing/subscription owner.
|
||||
|
||||
StreamState:
|
||||
|
||||
- Classification: Packet-plane authoritative state.
|
||||
- Likely owner: Single stream owner; all packets for a given stream should be handled by one owner.
|
||||
|
||||
MeshPeerState:
|
||||
|
||||
- Classification: Split transport/session state and routing policy state.
|
||||
- Likely owner: Transport/session owner for socket/auth/session facts; routing policy owner for cached packet decisions.
|
||||
- Rule: Authenticated peer/session state must be cached locally for packet decisions.
|
||||
|
||||
BridgeControlState:
|
||||
|
||||
- Classification: Control-plane input with packet-plane effect.
|
||||
- Likely owner: Relevant packet/routing owner for active BCSQ/STUN effects.
|
||||
- Rule: BCSQ/STUN state used by the packet path must be local to the relevant packet owner.
|
||||
|
||||
ReportingState:
|
||||
|
||||
- Classification: Reporting/export snapshot and event state.
|
||||
- Likely owner: Reporting worker.
|
||||
- Rule: Not authoritative for packet routing.
|
||||
|
||||
CompatibilityExportState:
|
||||
|
||||
- Classification: Derived compatibility state.
|
||||
- Likely owner: Compatibility adapter/export worker.
|
||||
- Rule: Never authoritative.
|
||||
@ -1,66 +0,0 @@
|
||||
# FreeDMR 2 Subscription Model
|
||||
|
||||
The subscription model is the centrepiece of FreeDMR 2.
|
||||
|
||||
Conceptually, each TG is a conference bridge. Client systems subscribe to conference TGs. FreeDMR routes traffic according to active subscriptions, not according to the legacy shape of the `BRIDGES` dict.
|
||||
|
||||
Definitions:
|
||||
|
||||
- `conference_tg`: FreeDMR network/conference identity.
|
||||
- `rf_tg`: Client-facing RF presentation identity.
|
||||
|
||||
Examples:
|
||||
|
||||
Direct TG:
|
||||
|
||||
```text
|
||||
rf_tg == conference_tg
|
||||
```
|
||||
|
||||
Dial-a-TG:
|
||||
|
||||
```text
|
||||
rf_tg == 9
|
||||
conference_tg == selected reflector/TG
|
||||
```
|
||||
|
||||
Alias/rewrite:
|
||||
|
||||
```text
|
||||
rf_tg may differ from conference_tg by policy/configuration
|
||||
```
|
||||
|
||||
Example subscription:
|
||||
|
||||
```python
|
||||
TalkgroupSubscription(
|
||||
client_id=2345001,
|
||||
slot=2,
|
||||
rf_tg=9,
|
||||
conference_tg=4400,
|
||||
mode="dial",
|
||||
active=True,
|
||||
)
|
||||
```
|
||||
|
||||
## Routing Invariant
|
||||
|
||||
Packet routing should not need to know whether a subscription came from static config, default reflector, dial-a-TG, API, SMS control, or a future UI action. Those are subscription sources, not routing modes.
|
||||
|
||||
## Dial-a-TG Rationale
|
||||
|
||||
Dial-a-TG exists so terminal users can access arbitrary FreeDMR TGs without programming every TG into the terminal/codeplug. It is an amateur-radio usability feature and should be evaluated against that goal, not only against commercial DMR fleet assumptions.
|
||||
|
||||
Control of dial-a-TG from TS1 as well as TS2 is intentional. If TS2 is blocked by unwanted traffic, a user can transmit private-call control on TS1 to disconnect or change the TS2 reflector/TG state.
|
||||
|
||||
Voice prompts should remain RF-visible as TG9 slot 2 unless that policy is deliberately changed.
|
||||
|
||||
## FreeDMR Routing Model
|
||||
|
||||
- TGs are conference groups.
|
||||
- DMR IDs are like phone numbers.
|
||||
- Timeslots are access/capacity paths, more like phone lines.
|
||||
- FreeDMR is intended to be relatively timeslot agnostic.
|
||||
- TS1 control affecting TS2 reflector state is consistent with the FreeDMR PBX/line model.
|
||||
|
||||
This model also allows future arbitrary RF TG aliases, not only the traditional TG9 dial-a-TG rewrite.
|
||||
@ -1,61 +0,0 @@
|
||||
# Packet and Stream Model
|
||||
|
||||
## Packet Mutation Boundaries
|
||||
|
||||
Raw DMR packet bytes should be treated as immutable input until an explicit rewrite operation. Transport simulation and protocol mutation must remain separate.
|
||||
|
||||
Packet mutation must be named, explicit, and testable. FreeDMR should preserve packet bytes unless it intentionally rewrites them.
|
||||
|
||||
Protocol-sensitive rewrite areas include:
|
||||
|
||||
- Slot bit rewrite.
|
||||
- TG rewrite.
|
||||
- Stream ID preservation.
|
||||
- Source ID preservation.
|
||||
- Voice header LC rewrite.
|
||||
- Terminator LC rewrite.
|
||||
- Embedded LC rewrite.
|
||||
|
||||
Voice header/terminator LC and embedded LC must be handled carefully. Embedded LC rewrite should apply only to voice bursts B-E, not data/control packets.
|
||||
|
||||
Same-TG voice forwarding should preserve embedded LC payloads where possible. TG-mapped forwarding may regenerate embedded LC for routing correctness.
|
||||
|
||||
Data/control packets are packet-oriented and not AMBE voice streams. Group-addressed data is valid and can be routed as data, not reported as voice. Data/control classification must remain separate from group-vs-unit addressing.
|
||||
|
||||
Unit/private calls are control-plane only in FreeDMR. Do not introduce general private voice routing unless project policy changes.
|
||||
|
||||
## Sequence and Lifecycle Principles
|
||||
|
||||
DMRD sequence numbers are one byte and modulo-256.
|
||||
|
||||
- Delta `0`: duplicate.
|
||||
- Delta `1`: normal progress.
|
||||
- Delta `2..127`: forward progress with loss.
|
||||
- Delta `128..255`: stale or out-of-order.
|
||||
|
||||
Explicit voice terminator is a strong end-of-stream signal. Timeout without terminator is softer and may remain recoverable on HBP to preserve audio continuity.
|
||||
|
||||
HBP should be more tolerant because it is RF-facing and real deployments include imperfect terminals, repeaters, RF paths, cellular links, and RF IP links. FBP/OBP can be stricter because it is server-to-server, but should still preserve audio where possible on unreliable links.
|
||||
|
||||
Loop-control safety must not be overridden by tolerance for delayed or out-of-order packets.
|
||||
|
||||
## LC and OVCM
|
||||
|
||||
For DMR Group Voice Channel User LC, the first bytes are:
|
||||
|
||||
- FLCO
|
||||
- FID
|
||||
- Service Options
|
||||
|
||||
Normal synthetic group voice LC should use service options `0x00`.
|
||||
|
||||
OVCM is `0x04` if explicitly required.
|
||||
|
||||
HBLink legacy `0x20` should be documented as legacy/compatibility only. It is not standards-clean OVCM and should not be used as a new synthetic/system-generated traffic marker.
|
||||
|
||||
Decoded real inbound LC must be preserved unchanged unless there is a deliberate reason to rewrite. Synthetic/fallback LC generation must be explicit and tested. FreeDMR routing metadata should be used for routing state, not magic bits in synthetic LC.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Exact live RF behaviour after long HBP gaps still needs validation with real repeaters and terminals.
|
||||
- Some prompt/late-entry behaviour may need live testing because terminal interpretation of DMR standards can be loose or incomplete.
|
||||
@ -1,15 +0,0 @@
|
||||
# Data Packet Policy
|
||||
|
||||
No regression of DMR data support is permitted.
|
||||
|
||||
FreeDMR should forward supported DMR data packets according to conference/subscription and mesh rules. The FreeDMR core should not become a general GPS/SMS application processor.
|
||||
|
||||
GPS, SMS, and similar application processing should be implemented by systems connected via FBP or another mesh/access-adjacent interface. `DATA-GATEWAY` is understood as an earlier expression of this model.
|
||||
|
||||
Existing `SUB_MAP` / last-known-location behaviour is intentional: data addressed to a DMR ID can be routed toward the last known HBP/client location.
|
||||
|
||||
Narrow exceptions may exist for SMS-based dial-a-TG control or DMR SMS alerts to the sysop.
|
||||
|
||||
Data/control classification must be separate from group-vs-unit addressing. A group-addressed data packet is not automatically a voice stream.
|
||||
|
||||
The packet layer may inspect data packets for admission, routing, loop/source-quench safety, reporting, and metadata preservation. Application-level GPS/SMS semantics should live outside the core unless a specific control-plane feature requires it.
|
||||
@ -1,40 +0,0 @@
|
||||
# Mesh Model
|
||||
|
||||
FreeDMR is a peer network, not hub-and-spoke. Local sysops retain policy autonomy.
|
||||
|
||||
The guiding principle remains "everything everywhere", subject to source quench, STUN, ACLs, local policy, authentication, loop-control, and documented exceptions.
|
||||
|
||||
## Loop Control and Bridge Control
|
||||
|
||||
Loop control, source selection, duplicate suppression, source quench, and STUN are packet-plane safety mechanisms.
|
||||
|
||||
Source quench is a control hint to suppress a stream/TG toward a peer. It is optional and scoped per stream/TG.
|
||||
|
||||
STUN is a broader FBP/OpenBridge traffic gate. Under the current conceptual model, `BCST`/STUN applies to all FBP traffic from that peer until cleared or expired by policy.
|
||||
|
||||
`BCSQ` is per stream/TG.
|
||||
|
||||
`BCST`/STUN is all FBP traffic.
|
||||
|
||||
## Protocol Versions and Metadata
|
||||
|
||||
Source server and source repeater metadata must be preserved according to the protocol version actually in use for that session.
|
||||
|
||||
OBP/FBP protocol version controls metadata layout and option order.
|
||||
|
||||
Protocol v1 OBP remains an important open interop path where intentionally configured. FBP v5 is the current richer peer-server protocol target. FBP v4 is historical/deprecation context unless explicitly retained.
|
||||
|
||||
## TG Namespace Rule
|
||||
|
||||
HBP/RF-visible TG and FBP/OBP-visible conference TG can intentionally differ, especially with dial-a-TG.
|
||||
|
||||
Source quench must use the TG namespace visible to the peer sending or receiving the quench.
|
||||
|
||||
For HBP-to-FBP dial-a-TG, `BCSQ` should use the FBP/reflector TG, not local RF TG9.
|
||||
|
||||
For OBP-source traffic, `BCSQ` should use the inbound OBP TG because that is the source-server namespace.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Final FBP v5 identity/auth fields still need a concrete wire-format decision.
|
||||
- STUN recovery policy needs an operator workflow, likely API-driven.
|
||||
@ -1,203 +0,0 @@
|
||||
# Reporting Model
|
||||
|
||||
Decision: FreeDMR 2 replaces the legacy dashboard/report socket model with a new structured reporting event model.
|
||||
|
||||
The existing dashboard is not a compatibility constraint for the FreeDMR 2 core. It must be updated separately or supported by an optional out-of-process adapter.
|
||||
|
||||
Reporting is observational only. Packet routing must not depend on dashboard/report consumers.
|
||||
|
||||
The v2 event schema is the compatibility contract. Raw `BRIDGES`/`SYSTEM` state should not be exposed as the primary v2 API. Old dashboard/report socket event names should not shape the FreeDMR 2 core.
|
||||
|
||||
Any old dashboard compatibility must live in a sidecar/adapter, not inside packet routing. FreeDMR 1.x/current code remains live until FreeDMR 2 is ready, so FreeDMR 2 can make a clean reporting break.
|
||||
|
||||
## Preferred Transport
|
||||
|
||||
MQTT is the preferred external live reporting transport.
|
||||
|
||||
Architecture:
|
||||
|
||||
```text
|
||||
packet path -> non-blocking bounded local event queue -> MQTT publisher worker -> local broker/feed
|
||||
```
|
||||
|
||||
Constraints:
|
||||
|
||||
- MQTT publishing must be asynchronous from the packet worker.
|
||||
- Use a bounded queue.
|
||||
- The bounded local event queue is the only coupling from packet path to reporting worker.
|
||||
- Drop or coalesce low-priority events under pressure.
|
||||
- Emit a later reporting-health event rather than blocking packet handling.
|
||||
- Voice stability takes precedence over reporting completeness.
|
||||
- Reconnect with exponential backoff.
|
||||
- Refresh retained state after reconnect.
|
||||
- Reporting backpressure must be visible through reporting-health events but must not delay DMR packets.
|
||||
|
||||
Reporting is the first major candidate for out-of-process execution. The MQTT publisher should be an independent worker or sidecar where practical. The global lastheard exporter should be a separate process. Dashboard aggregation should not run in the packet hot path.
|
||||
|
||||
Reporting worker crash must not affect packet routing. Reporting worker restart should refresh retained state after reconnect.
|
||||
|
||||
## Local Dashboard and Global Lastheard
|
||||
|
||||
Local dashboard:
|
||||
|
||||
- Consumes local MQTT live state/events.
|
||||
- Displays live client/repeater/server traffic.
|
||||
- Recovers from retained state after reconnect.
|
||||
|
||||
Global lastheard:
|
||||
|
||||
- Central/non-real-time.
|
||||
- Consumes summaries, not packet-plane traffic.
|
||||
- Should preferably be fed by a separate exporter process.
|
||||
- Central outage must not affect local packet handling or local dashboard.
|
||||
|
||||
Preferred flow:
|
||||
|
||||
```text
|
||||
FreeDMR core -> local MQTT feed -> local dashboard
|
||||
-> global-exporter process -> network MQTT/collector
|
||||
```
|
||||
|
||||
## Initial Event Families
|
||||
|
||||
- `server.started`
|
||||
- `server.stopping`
|
||||
- `client.connected`
|
||||
- `client.disconnected`
|
||||
- `client.options_changed`
|
||||
- `subscription.activated`
|
||||
- `subscription.deactivated`
|
||||
- `subscription.expired`
|
||||
- `call.started`
|
||||
- `call.ended`
|
||||
- `call.lost`
|
||||
- `mesh.peer_up`
|
||||
- `mesh.peer_down`
|
||||
- `mesh.source_quench`
|
||||
- `mesh.stun`
|
||||
- `loop.detected`
|
||||
- `packet.rate_limited`
|
||||
- `reporting.queue_overflow`
|
||||
- `reporting.publisher_disconnected`
|
||||
- `reporting.publisher_reconnected`
|
||||
- `reporting.events_dropped`
|
||||
|
||||
## Suggested MQTT Topics
|
||||
|
||||
```text
|
||||
freedmr/v2/{server_id}/state
|
||||
freedmr/v2/{server_id}/client/{client_id}/state
|
||||
freedmr/v2/{server_id}/client/{client_id}/slot/{slot}/activity
|
||||
freedmr/v2/{server_id}/subscription/{subscription_id}/state
|
||||
freedmr/v2/{server_id}/call/{stream_id}/start
|
||||
freedmr/v2/{server_id}/call/{stream_id}/end
|
||||
freedmr/v2/{server_id}/mesh/{peer_id}/state
|
||||
freedmr/v2/{server_id}/event
|
||||
```
|
||||
|
||||
Use retained messages for current state and non-retained messages for transient events.
|
||||
|
||||
## Example Events
|
||||
|
||||
```json
|
||||
{
|
||||
"event": "server.started",
|
||||
"server_id": "2345",
|
||||
"version": "2.0-dev",
|
||||
"time": "2026-05-24T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"event": "client.connected",
|
||||
"server_id": "2345",
|
||||
"client_id": 2345001,
|
||||
"listener": "hbp-public",
|
||||
"endpoint": "198.51.100.10:62031",
|
||||
"time": "2026-05-24T12:00:01Z"
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"event": "subscription.activated",
|
||||
"server_id": "2345",
|
||||
"subscription_id": "2345001-2-9-4400",
|
||||
"client_id": 2345001,
|
||||
"slot": 2,
|
||||
"rf_tg": 9,
|
||||
"conference_tg": 4400,
|
||||
"mode": "dial",
|
||||
"source": "dial-a-tg",
|
||||
"time": "2026-05-24T12:00:02Z"
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"event": "call.started",
|
||||
"server_id": "2345",
|
||||
"stream_id": 12345678,
|
||||
"client_id": 2345001,
|
||||
"slot": 2,
|
||||
"source_id": 2345678,
|
||||
"rf_tg": 9,
|
||||
"conference_tg": 4400,
|
||||
"source": "hbp",
|
||||
"time": "2026-05-24T12:00:03Z"
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"event": "call.ended",
|
||||
"server_id": "2345",
|
||||
"stream_id": 12345678,
|
||||
"reason": "terminator",
|
||||
"duration_ms": 18420,
|
||||
"packets": 614,
|
||||
"time": "2026-05-24T12:00:21Z"
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"event": "call.lost",
|
||||
"server_id": "2345",
|
||||
"stream_id": 12345678,
|
||||
"reason": "timeout",
|
||||
"last_seen_ms_ago": 7000,
|
||||
"time": "2026-05-24T12:00:28Z"
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"event": "mesh.source_quench",
|
||||
"server_id": "2345",
|
||||
"peer_id": "2350",
|
||||
"stream_id": 12345678,
|
||||
"conference_tg": 4400,
|
||||
"reason": "duplicate-source",
|
||||
"time": "2026-05-24T12:00:04Z"
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"event": "reporting.queue_overflow",
|
||||
"server_id": "2345",
|
||||
"dropped_events": 42,
|
||||
"queue_limit": 2048,
|
||||
"policy": "drop-low-priority",
|
||||
"time": "2026-05-24T12:00:05Z"
|
||||
}
|
||||
```
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Broker packaging and defaults: Mosquitto, embedded broker, external broker, or optional dependency.
|
||||
- Exact retained-state expiry policy.
|
||||
- Authentication model for MQTT clients.
|
||||
- Whether legacy dashboard compatibility is a supplied sidecar or a separate dashboard migration task.
|
||||
@ -1,52 +0,0 @@
|
||||
# API and Control Model
|
||||
|
||||
The current HTTP/JSON API is experimental and should be treated as a starting point, not a fixed FreeDMR 2 contract.
|
||||
|
||||
## Current API Principles
|
||||
|
||||
- Local administration and automation.
|
||||
- Not for public internet exposure.
|
||||
- Small in-memory operations only.
|
||||
- Bounded request bodies.
|
||||
- No expensive live serialization of internal state.
|
||||
- No blocking packet path.
|
||||
- User-level authentication by connected peer/client session key.
|
||||
- System-level authentication by system API key.
|
||||
|
||||
The API should bind to localhost by default unless explicitly configured otherwise.
|
||||
|
||||
## FreeDMR 2 Direction
|
||||
|
||||
API operations should be bounded control-plane operations over access sessions, subscriptions, mesh peers, and reporting state.
|
||||
|
||||
Destructive system actions such as kill, resetall, and STUN clear should be separately enableable. Audit logs are required. Keys and secrets must never be logged.
|
||||
|
||||
Dashboard controls should use authenticated HTTP API operations unless a future UI genuinely needs bidirectional streaming.
|
||||
|
||||
Compatibility with the old API should be an adapter concern if needed.
|
||||
|
||||
The API may eventually run as a separate control-plane worker or sidecar. API requests should become `ControlCommand` messages sent to the owner of the relevant state.
|
||||
|
||||
The API must not directly mutate packet-plane state it does not own. Destructive operations must go through explicit owner-handled commands. API worker failure should not stop existing packet routing, although new control actions may fail until the worker recovers.
|
||||
|
||||
## Suggested v2 Operations
|
||||
|
||||
```text
|
||||
GET /api/v2/health
|
||||
GET /api/v2/version
|
||||
GET /api/v2/state
|
||||
GET /api/v2/client/{client_id}
|
||||
GET /api/v2/client/{client_id}/slot/{slot}
|
||||
POST /api/v2/client/{client_id}/slot/{slot}/subscriptions
|
||||
DELETE /api/v2/client/{client_id}/slot/{slot}/subscriptions/{subscription_id}
|
||||
POST /api/v2/mesh/{peer_id}/stun
|
||||
DELETE /api/v2/mesh/{peer_id}/stun
|
||||
POST /api/v2/system/reset
|
||||
POST /api/v2/system/stop
|
||||
```
|
||||
|
||||
## Packet-Path Rule
|
||||
|
||||
API handlers must not perform work that delays packet routing. Expensive state export, dashboard compatibility, global reporting, and administrative analysis should use snapshots, bounded queues, or separate processes.
|
||||
|
||||
No API path should force expensive live serialization of packet-plane state.
|
||||
@ -1,56 +0,0 @@
|
||||
# Security Model
|
||||
|
||||
Core principle: FreeDMR may sign/authenticate traffic and control messages. FreeDMR should not encrypt amateur-radio or mesh traffic by default.
|
||||
|
||||
The security model is authenticity, integrity, membership validation, and local policy, not secrecy. Amateur radio is public, and users may provide IP backhaul over amateur-radio links where encryption rules matter.
|
||||
|
||||
## Mesh Authentication
|
||||
|
||||
Preferred direction:
|
||||
|
||||
- PKI-backed FBP peer admission through Bridge Control / BCXX.
|
||||
- Signed server/sysop identity.
|
||||
- Bind server ID, authorized sub-IDs, public key, validity, and features where useful.
|
||||
- Bind authenticated identity to observed endpoint/IP.
|
||||
- If endpoint changes, peer must re-authenticate.
|
||||
- Expensive signature/cert validation is control-plane work.
|
||||
- Packet-plane uses cached authenticated session state.
|
||||
- Soft renewal should avoid interrupting in-flight voice when safe.
|
||||
- Hard stop on revocation, explicit failure, endpoint mismatch outside policy, grace expiry, or local policy.
|
||||
|
||||
## Identity and Listing
|
||||
|
||||
Signed identity proves membership/identity, not mandatory carriage. Public listing is separate from mesh identity.
|
||||
|
||||
Local sysops may choose whether to carry or vouch for traffic. A valid signed key does not override local policy.
|
||||
|
||||
Vouching sysop accountability is part of FreeDMR's social trust model. A sysop allowing problematic traffic onto the mesh may see other peers stop peering with them.
|
||||
|
||||
One verification of a key may cover the server ID and authorized sub-IDs for that sysop/server deployment.
|
||||
|
||||
## Distributed Key Gossip Option
|
||||
|
||||
Signed membership documents may be gossiped over bounded/rate-limited BCXX.
|
||||
|
||||
Peers validate signatures and build local key tables. Revocation, expiry, serials, and replay protection are required.
|
||||
|
||||
Key gossip cannot create trust by mere repetition. The packet path must use cached key/policy state.
|
||||
|
||||
This supports autonomous routing decisions for packets that originated from a server even when that source server is not directly connected.
|
||||
|
||||
## Analogue and Digital Bridge Policy
|
||||
|
||||
Analogue ROIP bridges may connect as HBP clients. Permitted does not mean automatically valuable.
|
||||
|
||||
Analogue bridges can be operationally sensitive because mixed or continuous analogue audio is a poor fit for DMR one-source-at-a-time stream behaviour. They may hold a TG open, play tones, or prevent digital users from breaking in until a carrier/timer drops.
|
||||
|
||||
Analogue bridges should be subject to local policy, listing expectations, and peer accountability.
|
||||
|
||||
YSF/NXDN and other AMBE-family networks are often a better technical match than analogue or unlike-codec transcoding, because they can avoid lossy audio translation.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- X.509 certificates versus simpler Ed25519 signed membership documents.
|
||||
- Exact revocation and renewal distribution process.
|
||||
- Default grace period for soft re-authentication.
|
||||
- How much key gossip should be enabled by default.
|
||||
@ -1,69 +0,0 @@
|
||||
# Runtime and Concurrency
|
||||
|
||||
Decision: Do not replace Twisted as the first FreeDMR 2 architecture move.
|
||||
|
||||
## Rationale
|
||||
|
||||
Current packet behaviour is subtle. Twisted's single-threaded reactor is currently a safety boundary. Replacing the event loop and the state model at the same time mixes too many changes.
|
||||
|
||||
The first goal is architectural clarity and testability, not event-loop novelty.
|
||||
|
||||
## Immediate Runtime Strategy
|
||||
|
||||
- Keep Twisted initially as the transport shell.
|
||||
- Use Twisted's single-threaded reactor as a safety boundary while the core is extracted.
|
||||
- Do not replace the event loop and state model at the same time.
|
||||
- Extract the protocol/routing/subscription core behind deterministic interfaces.
|
||||
- Keep packet-plane state local and deterministic.
|
||||
- No blocking work in reactor callbacks.
|
||||
- No dashboard/API/database/MQTT waits in the packet path.
|
||||
- Single-owner state is preferred.
|
||||
- Explicit messages/events are preferred over shared mutable dictionaries across threads/processes.
|
||||
|
||||
## Eventual Capacity Strategy
|
||||
|
||||
FreeDMR 2 should support worker-process scaling once state ownership and message boundaries are explicit and tested.
|
||||
|
||||
The purpose of worker processes is not merely performance. It is also:
|
||||
|
||||
- Clearer state ownership.
|
||||
- Failure isolation.
|
||||
- Safer concurrency.
|
||||
- Testable boundaries.
|
||||
- Future capacity scaling.
|
||||
|
||||
Prefer process/actor ownership over shared-memory no-GIL threading for authoritative routing state. No-GIL Python does not remove the need for clear ownership of mutable packet-plane state.
|
||||
|
||||
Offload non-packet-path work first, including reporting, MQTT publishing, global export, SQL writes, dashboard aggregation, alias refresh, analytics, and lab/codec work.
|
||||
|
||||
Routing-core workers are a later stage. Multi-worker sharding should only be considered after single-worker message-boundary behaviour is proven.
|
||||
|
||||
Twisted can remain the transport shell while reporting/export/control workers move out-of-process.
|
||||
|
||||
## Ownership Split
|
||||
|
||||
Twisted parent/transport process may own:
|
||||
|
||||
- UDP sockets.
|
||||
- HBP/FBP packet receive/send.
|
||||
- Timers.
|
||||
- Process supervision.
|
||||
|
||||
Routing core should eventually own:
|
||||
|
||||
- Stream state.
|
||||
- Subscription state.
|
||||
- Dial-a-TG state.
|
||||
- Loop-control state.
|
||||
- Duplicate suppression.
|
||||
- Routing decisions.
|
||||
|
||||
Migration must be staged and covered by tests. FreeDMR should remain deployable on ordinary low-cost systems such as cheap VPS instances and Raspberry Pi-class hardware.
|
||||
|
||||
See `13-worker-process-scaling.md` for the eventual worker-process capacity model.
|
||||
|
||||
## External Databases
|
||||
|
||||
External stores can be useful for configuration, reporting snapshots, global lastheard, operator UI, and coordination. They should not sit in the packet hot path.
|
||||
|
||||
Packet-plane state should stay local and in memory unless a future design proves a bounded, non-blocking alternative.
|
||||
@ -1,100 +0,0 @@
|
||||
# Testing and Release Gates
|
||||
|
||||
FreeDMR 2 must preserve behaviour through tests before changing architecture. The existing deterministic harness, UDP black-box harness, codec tests, support tests, and future live RF validation form the release gate structure.
|
||||
|
||||
## Test Commands
|
||||
|
||||
General test run:
|
||||
|
||||
```bash
|
||||
python -m unittest discover -v
|
||||
```
|
||||
|
||||
Focused support/codec tests:
|
||||
|
||||
```bash
|
||||
python -m unittest tests.test_freedmr_dmr_codec tests.test_utils -v
|
||||
```
|
||||
|
||||
UDP black-box tests:
|
||||
|
||||
```bash
|
||||
FREEDMR_RUN_UDP_TESTS=1 python -m unittest tests.test_udp_blackbox_harness -v
|
||||
```
|
||||
|
||||
UDP black-box tests with venv bootstrap:
|
||||
|
||||
```bash
|
||||
FREEDMR_RUN_UDP_TESTS=1 \
|
||||
FREEDMR_UDP_BOOTSTRAP_VENV=1 \
|
||||
FREEDMR_UDP_VENV_DIR=/tmp/freedmr-blackbox-venv3 \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
python -m unittest tests.test_udp_blackbox_harness -v
|
||||
```
|
||||
|
||||
## Release Gates
|
||||
|
||||
Level 0: Codec/unit/config/support tests. Must pass for every commit.
|
||||
|
||||
Level 1: Deterministic packet/state harness. Must pass before merge to main.
|
||||
|
||||
Level 2: Black-box UDP harness. Must pass before release candidate.
|
||||
|
||||
Level 3: Live RF / real repeater / real peer validation. Required before changing protocol-visible behaviour.
|
||||
|
||||
## What Each Layer Proves
|
||||
|
||||
Deterministic harness:
|
||||
|
||||
- Good for packet parsing seams, routing state, dial-a-TG state, fake-clock expiry, rewrite boundaries, LC/embedded LC tests, data-vs-voice classification, and reporting event generation.
|
||||
- Bypasses real UDP, socket binding, subprocess startup, and Twisted timing.
|
||||
|
||||
UDP black-box harness:
|
||||
|
||||
- Good for subprocess startup, HBP login, UDP parsing, FBP signing, bridge-control, malformed/hostile packets, cadence, packet ordering, and link impairment.
|
||||
- Cannot prove RF-side modem/radio behaviour.
|
||||
|
||||
Live RF validation:
|
||||
|
||||
- Required for protocol-visible changes, prompt/ident behaviour, late entry, OVCM/LC options, repeater/radio compatibility, and real terminal quirks.
|
||||
|
||||
## Required Assertions
|
||||
|
||||
Tests should assert:
|
||||
|
||||
- Route recipients and non-recipients.
|
||||
- Packet byte preservation outside allowed rewrite regions.
|
||||
- Explicit rewrite ranges.
|
||||
- Stream lifecycle.
|
||||
- Subscription state.
|
||||
- Reporting events.
|
||||
- Source quench/STUN behaviour.
|
||||
- Absence of unintended traffic.
|
||||
|
||||
## Worker/process-boundary Release Gates
|
||||
|
||||
Before moving packet-plane behaviour across a process boundary:
|
||||
|
||||
- Deterministic in-process behaviour must already be covered.
|
||||
- The same scenario must be covered through message-boundary tests.
|
||||
- The same scenario must be covered through UDP black-box tests where observable.
|
||||
- Packet bytes must be compared before and after crossing the process boundary.
|
||||
- Route recipient/non-recipient sets must match.
|
||||
- Allowed rewrite regions must match.
|
||||
- Source quench/STUN/loop-control behaviour must match.
|
||||
- Failure injection must prove worker crash/restart does not replay stale packets.
|
||||
- Reporting/control worker backpressure must not block packet routing.
|
||||
- Live RF validation is required for protocol-visible behaviour.
|
||||
|
||||
Suggested future test categories:
|
||||
|
||||
- Reporting worker crash during active call.
|
||||
- Global exporter outage.
|
||||
- API worker unavailable during normal traffic.
|
||||
- Routing worker restart while stream active.
|
||||
- Routing worker backpressure.
|
||||
- Queue overflow from packet process to reporting worker.
|
||||
- Stale `PacketReceived` replay prevention.
|
||||
- Duplicate packet prevention after worker restart.
|
||||
- Stream ownership handoff/drain test.
|
||||
- Coordinator restart test, if a coordinator is introduced.
|
||||
@ -1,56 +0,0 @@
|
||||
# Migration Plan
|
||||
|
||||
Constraints:
|
||||
|
||||
- No big-bang rewrite of packet semantics.
|
||||
- Current FreeDMR remains live until FreeDMR 2 is tested.
|
||||
- Build FreeDMR 2 core beside current code where practical.
|
||||
- Preserve behaviours behind tests.
|
||||
- Compatibility adapters are allowed, but old internal shape should not define the new core.
|
||||
- Worker-process scaling is a design direction, not a reason for a big-bang rewrite.
|
||||
|
||||
## Stages
|
||||
|
||||
Stage 0: Stabilise current code, harness, docs, codec, and known behaviour.
|
||||
|
||||
Stage 1: Distil architecture, glossary, state model, and reporting event contract.
|
||||
|
||||
Stage 2: Extract packet/codec helpers and deterministic routing/subscription seams.
|
||||
|
||||
Stage 3: Introduce explicit internal message/event objects in-process.
|
||||
|
||||
Stage 4: Implement new subscription store in parallel with compatibility export if needed.
|
||||
|
||||
Stage 5: Move reporting/MQTT publisher to an independent worker/sidecar.
|
||||
|
||||
Stage 6: Move global lastheard exporter, SQL writes, dashboard aggregation, and non-critical analytics to workers.
|
||||
|
||||
Stage 7: Define v2 API/control-plane operations over sessions, subscriptions, mesh state, and reporting health, and express them as owner-handled `ControlCommand` messages.
|
||||
|
||||
Stage 8: Introduce listener/client session model supporting multiple clients per listener.
|
||||
|
||||
Stage 9: Introduce mesh auth/BCXX identity admission into the control plane.
|
||||
|
||||
Stage 10: Experiment with a single routing-core worker behind the already-tested message interface.
|
||||
|
||||
Stage 11: Evaluate multi-routing-worker sharding only after single-worker routing is stable and covered.
|
||||
|
||||
Stage 12: Cut over only after deterministic, UDP, and live RF validation.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Do not add general user-to-user private voice routing.
|
||||
- Do not make FreeDMR core a GPS/SMS application processor.
|
||||
- Do not make reporting/dashboard consumers part of packet routing.
|
||||
- Do not encrypt amateur-radio traffic by default.
|
||||
- Do not replace Twisted before extracting and test-covering the core.
|
||||
- Do not preserve the old dashboard protocol inside the FreeDMR 2 packet core.
|
||||
- Do not make worker-process scaling an immediate rewrite requirement.
|
||||
- Do not use Redis, Postgres, MQTT, dashboards, or APIs for live per-packet routing decisions.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Exact cut-over mechanism from current `BRIDGES` state to subscription state.
|
||||
- Whether old dashboard compatibility is shipped as part of FreeDMR 2 or maintained with the dashboard.
|
||||
- Final mesh authentication wire format and key distribution policy.
|
||||
- Live RF validation matrix for repeaters, hotspots, terminals, and analogue/digital bridges.
|
||||
@ -1,289 +0,0 @@
|
||||
# Worker Process Scaling
|
||||
|
||||
## Decision
|
||||
|
||||
FreeDMR 2 should be designed so that, after the protocol/routing/subscription core has been extracted and tested, selected parts of the system can be moved into separate worker processes to improve capacity, isolate failures, and avoid the practical single-thread/GIL limits of one Python process.
|
||||
|
||||
This is not a first-stage rewrite requirement.
|
||||
|
||||
The first stage remains:
|
||||
|
||||
- Keep Twisted initially.
|
||||
- Extract the deterministic core.
|
||||
- Make state ownership explicit.
|
||||
- Preserve packet behaviour through tests.
|
||||
|
||||
But the FreeDMR 2 state model must not block a later move to worker processes.
|
||||
|
||||
## Why Worker Processes
|
||||
|
||||
CPython single-process execution has practical limits for CPU-bound Python code.
|
||||
|
||||
Twisted's single reactor is useful as an initial safety boundary, but one reactor process should not be assumed to be the final capacity architecture.
|
||||
|
||||
Worker processes provide stronger ownership and failure boundaries than shared-memory threads. Process/message boundaries are safer for FreeDMR routing state than no-GIL shared mutable dictionaries.
|
||||
|
||||
Worker processes fit FreeDMR's need for explicit ownership of routing, stream, subscription, loop-control, and reporting state.
|
||||
|
||||
Worker-process scaling must not make ordinary small FreeDMR deployments heavyweight or hard to run. A single-server deployment on a cheap VPS or Raspberry Pi-class system must remain supported.
|
||||
|
||||
## What Should Be Offloaded First
|
||||
|
||||
Low-risk early offload candidates:
|
||||
|
||||
- MQTT/reporting publisher.
|
||||
- Global lastheard exporter.
|
||||
- Dashboard aggregation.
|
||||
- SQL/database writing.
|
||||
- Historical analytics.
|
||||
- Alias download/refresh.
|
||||
- Expensive codec experiments.
|
||||
- Packet capture/replay analysis.
|
||||
- Non-critical maintenance jobs.
|
||||
- Future transcoding/bridge adjuncts.
|
||||
- Future network-analysis or observability tools.
|
||||
|
||||
These workers must be asynchronous from the packet path.
|
||||
|
||||
If they are slow, blocked, crashed, overloaded, or absent, packet routing must continue.
|
||||
|
||||
Reporting completeness is secondary to voice stability.
|
||||
|
||||
## What Should Not Be Offloaded Early
|
||||
|
||||
Do not initially offload hot-path routing decisions if doing so would add IPC, network, database, lock, queue, or back-pressure waits to every DMR packet.
|
||||
|
||||
Specifically keep local and deterministic until the model is proven:
|
||||
|
||||
- Stream admission.
|
||||
- Duplicate suppression.
|
||||
- Loop control.
|
||||
- Source quench checks.
|
||||
- Dial-a-TG state mutation.
|
||||
- Subscription lookup.
|
||||
- Slot/TG rewrite decisions.
|
||||
- Voice/data classification.
|
||||
- Packet mutation/rewrite.
|
||||
- HBP RF-facing tolerance logic.
|
||||
- Protocol-version-sensitive FBP/OBP metadata handling.
|
||||
|
||||
The packet plane must continue to use local in-memory state and must not depend on external databases, MQTT, dashboards, APIs, or reporting consumers.
|
||||
|
||||
## Possible Long-Term Worker Architecture
|
||||
|
||||
### Transport/listener Process
|
||||
|
||||
Owns:
|
||||
|
||||
- UDP sockets.
|
||||
- HBP receive/send.
|
||||
- FBP/OBP receive/send.
|
||||
- Raw packet admission.
|
||||
- Socket identity.
|
||||
- Keepalive.
|
||||
- Low-level protocol parsing.
|
||||
- Forwarding packet events to the owning routing component.
|
||||
|
||||
### Routing Core Worker
|
||||
|
||||
Owns:
|
||||
|
||||
- Subscription state.
|
||||
- Stream state.
|
||||
- Dial-a-TG state.
|
||||
- Loop-control state.
|
||||
- Source-quench state.
|
||||
- Duplicate suppression.
|
||||
- Packet routing decisions.
|
||||
- Explicit packet rewrite decisions.
|
||||
- Authoritative packet-plane state for its assigned clients/streams/TGs.
|
||||
|
||||
### Reporting Worker
|
||||
|
||||
Owns:
|
||||
|
||||
- MQTT publishing.
|
||||
- Retained state refresh.
|
||||
- Reporting event queue.
|
||||
- Dashboard event fanout.
|
||||
- Reporting health events.
|
||||
- Drop/coalesce policy under pressure.
|
||||
|
||||
### Global Exporter Worker
|
||||
|
||||
Owns:
|
||||
|
||||
- Subscribing to local reporting feed.
|
||||
- Filtering and summarising local events.
|
||||
- Publishing curated summaries to global lastheard/network collector.
|
||||
- Retry/spool policy for central outage.
|
||||
|
||||
### Control/API Worker or Control-Plane Adapter
|
||||
|
||||
Owns:
|
||||
|
||||
- Sysop/API requests.
|
||||
- Validating control-plane credentials.
|
||||
- Converting API requests into explicit state-change commands.
|
||||
- Receiving `ControlResult` messages.
|
||||
- Never directly mutating packet-plane state it does not own.
|
||||
|
||||
### Optional Future Codec/Transcode/Analysis Workers
|
||||
|
||||
Own:
|
||||
|
||||
- Expensive or experimental codec work.
|
||||
- Transcoding adjuncts.
|
||||
- Packet replay analysis.
|
||||
- Offline diagnostics.
|
||||
- Future lab features.
|
||||
|
||||
These must remain outside the live packet hot path unless explicitly proven safe.
|
||||
|
||||
## State Ownership Rules
|
||||
|
||||
- Every mutable authoritative state object must have exactly one owner.
|
||||
- Other processes may hold snapshots or caches, but only the owner mutates authoritative state.
|
||||
- Do not use `multiprocessing.Manager().dict()` or shared mutable proxy objects as the main architecture.
|
||||
- Do not recreate a cross-process global `BRIDGES`-style mutable structure.
|
||||
- Use explicit messages/events instead of pretending cross-process state is a normal Python dict.
|
||||
- Packet bytes crossing process boundaries should be immutable.
|
||||
- Packet mutation must remain explicit, named, and testable.
|
||||
- A process boundary must not hide unclear ownership.
|
||||
- State ownership must be visible in tests and documentation.
|
||||
|
||||
## Message Boundary
|
||||
|
||||
Likely internal message families:
|
||||
|
||||
| Message | Plane |
|
||||
| --- | --- |
|
||||
| `PacketReceived` | packet-plane |
|
||||
| `PacketAccepted` | packet-plane |
|
||||
| `PacketDropped` | packet-plane |
|
||||
| `RouteDecision` | packet-plane |
|
||||
| `PacketToSend` | packet-plane |
|
||||
| `PacketMutated` | packet-plane |
|
||||
| `StreamStarted` | packet-plane |
|
||||
| `StreamEnded` | packet-plane |
|
||||
| `StreamLost` | packet-plane |
|
||||
| `SubscriptionActivated` | control-plane |
|
||||
| `SubscriptionDeactivated` | control-plane |
|
||||
| `SubscriptionExpired` | packet-plane |
|
||||
| `SourceQuenchReceived` | packet-plane |
|
||||
| `SourceQuenchSendRequested` | packet-plane |
|
||||
| `StunActivated` | control-plane |
|
||||
| `StunCleared` | control-plane |
|
||||
| `ReportingEvent` | reporting-plane |
|
||||
| `ControlCommand` | control-plane |
|
||||
| `ControlResult` | control-plane |
|
||||
| `WorkerStarted` | worker/supervision-plane |
|
||||
| `WorkerStopping` | worker/supervision-plane |
|
||||
| `WorkerHealth` | worker/supervision-plane |
|
||||
| `WorkerBackpressure` | worker/supervision-plane |
|
||||
| `WorkerCrashed` | worker/supervision-plane |
|
||||
| `WorkerRestarted` | worker/supervision-plane |
|
||||
|
||||
Packet-plane messages must be compact, bounded, and safe for high frequency use.
|
||||
|
||||
## Partitioning / Sharding Options
|
||||
|
||||
Possible future sharding models, without choosing one prematurely:
|
||||
|
||||
- By client/repeater DMR ID.
|
||||
- By listener/access socket.
|
||||
- By conference TG.
|
||||
- By source server / mesh peer.
|
||||
- By stream ID.
|
||||
- Hybrid model.
|
||||
|
||||
Constraints:
|
||||
|
||||
- All packets for a given live stream must be processed in order by the same stream owner.
|
||||
- Dial-a-TG state for one client/slot must have one owner.
|
||||
- Subscription state for one client/slot must have one owner.
|
||||
- Loop-control/source-quench state must be consistent for a given TG/stream/source path.
|
||||
- Cross-worker routing must not reintroduce duplicate packets or loops.
|
||||
- Worker assignment must be observable and testable.
|
||||
- Worker assignment must not depend on dashboard/reporting state.
|
||||
- Sharding must preserve the FreeDMR "everything everywhere" mesh principle, subject to existing source quench, STUN, ACL, policy, and authentication rules.
|
||||
|
||||
## Coordinator Model
|
||||
|
||||
FreeDMR 2 may eventually need a lightweight coordinator.
|
||||
|
||||
The coordinator may:
|
||||
|
||||
- Assign clients/sessions/TGs to workers.
|
||||
- Distribute subscription snapshots.
|
||||
- Manage worker health.
|
||||
- Restart workers.
|
||||
- Publish control-plane updates.
|
||||
- Provide routing-worker discovery.
|
||||
- Coordinate graceful drain/restart.
|
||||
|
||||
The coordinator must not:
|
||||
|
||||
- Synchronously participate in every packet routing decision.
|
||||
- Become a single blocking dependency for live voice.
|
||||
- Hide packet-plane state in an external database.
|
||||
- Make ordinary small deployments require clustered infrastructure.
|
||||
|
||||
Single-process/small-server deployment must remain supported. A coordinator should be optional or internal for simple deployments.
|
||||
|
||||
## Failure Behaviour
|
||||
|
||||
- Reporting worker failure: packet routing continues.
|
||||
- Global exporter failure: local service continues.
|
||||
- Dashboard aggregation failure: packet routing continues.
|
||||
- API/control worker failure: existing packet routing continues, but new control actions may fail.
|
||||
- Alias refresh worker failure: current aliases remain in use.
|
||||
- Analytics worker failure: packet routing continues.
|
||||
- Routing worker failure: affected sessions/streams are dropped or restarted according to explicit policy.
|
||||
- Transport/listener failure: affected sockets/sessions are lost until restart.
|
||||
- Worker restart must not replay stale DMR packets.
|
||||
- Retained/reporting state may be refreshed after recovery.
|
||||
- In-flight voice may be lost during worker crash, but failure must not poison the mesh or produce loops.
|
||||
- Backpressure from non-packet workers must not propagate into the packet path.
|
||||
|
||||
## Tests Required Before Worker Split
|
||||
|
||||
Before moving any packet-plane component into a separate process, require:
|
||||
|
||||
- Deterministic harness coverage of the state machine.
|
||||
- UDP black-box coverage of the same behaviour.
|
||||
- Message-boundary tests proving packet bytes and route decisions are preserved.
|
||||
- Failure-injection tests for worker timeout/crash/restart.
|
||||
- Queue-backpressure tests proving reporting/control workers cannot block packets.
|
||||
- Tests proving no stale packet replay after worker restart.
|
||||
- Tests proving source quench, STUN, loop-control, and duplicate suppression are preserved across the boundary.
|
||||
- Live RF validation for protocol-visible behaviour.
|
||||
|
||||
Worker split must not be considered complete until deterministic, UDP, and live RF validation agree for protocol-visible paths.
|
||||
|
||||
## Migration Path
|
||||
|
||||
Stage A: Extract pure/deterministic routing/subscription state behind explicit interfaces.
|
||||
|
||||
Stage B: Introduce internal message/event objects in-process.
|
||||
|
||||
Stage C: Move reporting/MQTT/global export to separate processes.
|
||||
|
||||
Stage D: Move slow maintenance, alias refresh, SQL/global lastheard, and analytics work to workers.
|
||||
|
||||
Stage E: Experiment with routing core as a child process behind the same message interface.
|
||||
|
||||
Stage F: Evaluate multi-routing-worker sharding only after the single routing-worker process model is stable and fully tested.
|
||||
|
||||
Stage G: Only after the above, consider whether transport/listener processes should be split by listener, client set, protocol, or deployment role.
|
||||
|
||||
## Explicit Non-Goals
|
||||
|
||||
- Do not introduce shared mutable cross-process `BRIDGES`-like state.
|
||||
- Do not depend on Redis/Postgres/MQTT for per-packet routing decisions.
|
||||
- Do not require heavyweight infrastructure for ordinary single-server deployments.
|
||||
- Do not use worker processes to hide unclear state ownership.
|
||||
- Do not move protocol-sensitive packet mutation across process boundaries until byte-preservation and rewrite tests prove equivalence.
|
||||
- Do not assume no-GIL Python solves FreeDMR's state ownership problem.
|
||||
- Do not replace Twisted and introduce worker sharding in the same step.
|
||||
- Do not make reporting, dashboard, SQL, global lastheard, or API availability part of the packet routing path.
|
||||
@ -1,28 +0,0 @@
|
||||
# FreeDMR 2 Architecture Notes
|
||||
|
||||
This directory contains the distilled FreeDMR 2 design notes. The older notes in `docs/codex-notes.md`, `docs/freedmr-2-architecture-decisions.md`, and the test/API docs remain source material and engineering history.
|
||||
|
||||
FreeDMR 2 is not a blind rewrite of packet behaviour. The protected asset is the FreeDMR operating model: DMR packet semantics, dial-a-TG, TG/DMR-ID-centric routing, loop control, source quench, mesh behaviour, RF-side tolerance, data forwarding, and practical amateur-radio interoperability.
|
||||
|
||||
Recommended reading order:
|
||||
|
||||
1. `00-glossary.md`
|
||||
2. `01-system-model.md`
|
||||
3. `02-state-model.md`
|
||||
4. `03-subscription-model.md`
|
||||
5. `04-packet-and-stream-model.md`
|
||||
6. `05-data-packet-policy.md`
|
||||
7. `06-mesh-model.md`
|
||||
8. `07-reporting-model.md`
|
||||
9. `08-api-control-model.md`
|
||||
10. `09-security-model.md`
|
||||
11. `10-runtime-and-concurrency.md`
|
||||
12. `13-worker-process-scaling.md`
|
||||
13. `11-testing-and-release-gates.md`
|
||||
14. `12-migration-plan.md`
|
||||
|
||||
Architecture Decision Records live in `adr/`. They record proposed FreeDMR 2 decisions separately from implementation work.
|
||||
|
||||
Current FreeDMR 1.x remains live until FreeDMR 2 is tested. Compatibility adapters are allowed where useful, but old HBLink object layout, dashboard socket assumptions, and legacy `BRIDGES` structure should not define the FreeDMR 2 core.
|
||||
|
||||
FreeDMR 2 keeps Twisted initially but is designed for eventual worker-process scaling. Non-packet-path workers such as reporting/global export move first; packet-plane routing workers are a later stage after state ownership and message-boundary tests are proven.
|
||||
@ -1,24 +0,0 @@
|
||||
# ADR 0001: Protected Model, Not HBLink Structure
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
FreeDMR current code is HBLink-derived and centred around `bridge_master.py`, `hblink.py`, configured `MASTER`/`SYSTEM` stanzas, global `BRIDGES`, Twisted, and the current dashboard/report model.
|
||||
|
||||
The valuable part is the operating model learned from real deployments: packet semantics, dial-a-TG, TG/DMR-ID-centric routing, loop control, source quench, mesh behaviour, RF-side tolerance, data forwarding, and practical amateur-radio interoperability.
|
||||
|
||||
## Decision
|
||||
The protected asset is FreeDMR behaviour/model, not the HBLink-derived object layout.
|
||||
|
||||
## Rationale
|
||||
HBLink-era structure blocks clarity, scaling, multi-client listeners, and testability. FreeDMR 2 should preserve validated packet behaviour while allowing cleaner internal models.
|
||||
|
||||
## Consequences
|
||||
FreeDMR 2 may replace legacy internal structures. Behaviour changes still require tests and live validation where protocol-visible.
|
||||
|
||||
## Compatibility
|
||||
Compatibility views may expose old shapes such as `BRIDGES`, `MASTER`, `SYSTEM`, or `#` reflector names, but they are adapters, not authoritative core state.
|
||||
|
||||
## Testing Requirements
|
||||
Regression tests must cover routing, dial-a-TG, loop control, source quench, data forwarding, and packet rewrite behaviour before internal models are replaced.
|
||||
@ -1,22 +0,0 @@
|
||||
# ADR 0002: Keep Twisted Initially
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
Twisted currently provides UDP transport, timers, and a single-threaded reactor boundary. Packet behaviour is subtle and production-proven.
|
||||
|
||||
## Decision
|
||||
Keep Twisted initially as a transport safety boundary; consider replacement only after it is a thin shell.
|
||||
|
||||
## Rationale
|
||||
Replacing the event loop and state model at the same time creates avoidable risk. Extracting the routing/subscription core first gives deterministic test coverage and clearer future migration options.
|
||||
|
||||
## Consequences
|
||||
FreeDMR 2 starts with evolutionary architecture work, not an event-loop rewrite. Twisted callbacks must remain non-blocking.
|
||||
|
||||
## Compatibility
|
||||
Current deployment and transport behaviour can remain familiar while the core model is extracted.
|
||||
|
||||
## Testing Requirements
|
||||
Deterministic core tests and UDP black-box tests must cover behaviour before any later Twisted replacement is considered.
|
||||
@ -1,22 +0,0 @@
|
||||
# ADR 0003: Subscription Model Replaces BRIDGES
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
The legacy `BRIDGES` dict mixes configuration, runtime state, reflector naming, routing, and export concerns. FreeDMR 2 models each TG as a conference bridge to which clients subscribe.
|
||||
|
||||
## Decision
|
||||
Use subscription-oriented internal state instead of legacy `BRIDGES` as the authoritative FreeDMR 2 model.
|
||||
|
||||
## Rationale
|
||||
Subscription state directly represents the FreeDMR user model: clients subscribe to TGs they want to hear. It supports direct TGs, dial-a-TG, default reflectors, API control, and future aliases without hard-coding routing modes.
|
||||
|
||||
## Consequences
|
||||
The packet path can use indexed subscription lookups. Existing dashboard/config expectations need compatibility export or migration.
|
||||
|
||||
## Compatibility
|
||||
`BRIDGES` and `#` reflector names may be generated as compatibility/export state where required, but routing should not depend on them.
|
||||
|
||||
## Testing Requirements
|
||||
Tests must assert subscription activation, expiry, direct TG routing, dial-a-TG mapping, default reflector behaviour, and absence of unintended recipients.
|
||||
@ -1,22 +0,0 @@
|
||||
# ADR 0004: Reporting v2 Replaces Legacy Dashboard Protocol
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
The old dashboard/report socket has shaped parts of the current implementation and has caused operational friction. FreeDMR 1.x remains live while FreeDMR 2 is developed.
|
||||
|
||||
## Decision
|
||||
FreeDMR 2 reporting is a new structured event contract. The old dashboard/report socket is not a compatibility constraint inside the core.
|
||||
|
||||
## Rationale
|
||||
Reporting must be observational only. A clean event schema avoids leaking legacy `BRIDGES`/`SYSTEM` state into the new packet core.
|
||||
|
||||
## Consequences
|
||||
The dashboard must be updated or served by an adapter. The core can emit stable v2 events without preserving legacy report names.
|
||||
|
||||
## Compatibility
|
||||
Old dashboard support belongs in a sidecar or adapter, not in packet routing.
|
||||
|
||||
## Testing Requirements
|
||||
Tests must assert expected v2 events for server, client, subscription, call, mesh, loop, and reporting-health changes.
|
||||
@ -1,22 +0,0 @@
|
||||
# ADR 0005: MQTT Reporting Transport
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
FreeDMR needs live local dashboard state and non-real-time global lastheard feeds without blocking packet handling.
|
||||
|
||||
## Decision
|
||||
MQTT is the preferred external live reporting transport, fed through a non-blocking bounded queue and independent publisher.
|
||||
|
||||
## Rationale
|
||||
MQTT is lightweight, familiar in radio/network operations, supports topics, retained state, last-will messages, and network fanout. A local broker lets extra consumers attach without adding work to the packet process.
|
||||
|
||||
## Consequences
|
||||
FreeDMR gains a broker dependency or optional integration. Reporting completeness is best-effort under pressure.
|
||||
|
||||
## Compatibility
|
||||
Legacy dashboard consumers need an adapter or dashboard update. Packet routing must continue if MQTT or consumers are unavailable.
|
||||
|
||||
## Testing Requirements
|
||||
Tests must cover enqueue, overflow/drop policy, publisher disconnect/reconnect events, retained state refresh, and packet-path non-blocking behaviour.
|
||||
@ -1,22 +0,0 @@
|
||||
# ADR 0006: Local Dashboard and Global Lastheard Are Separate
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
Each server has its own live dashboard. The global lastheard service is centrally hosted and non-real-time.
|
||||
|
||||
## Decision
|
||||
Local dashboard consumes local live feed; global lastheard consumes curated summaries via exporter/collector.
|
||||
|
||||
## Rationale
|
||||
Local live visibility must survive central outages. Global aggregation should not add packet-process load or require the core to do database/export work.
|
||||
|
||||
## Consequences
|
||||
A separate exporter process may be needed for global feeds. The broker handles fanout.
|
||||
|
||||
## Compatibility
|
||||
Existing global lastheard behaviour should be migrated to consume summaries rather than packet-plane events.
|
||||
|
||||
## Testing Requirements
|
||||
Tests should confirm local reporting works without global exporter connectivity and that exporter failure does not affect packet handling.
|
||||
@ -1,22 +0,0 @@
|
||||
# ADR 0007: Synthetic LC Service Options
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
Legacy HBLink used `0x20` in synthetic LC service options. Later MMDVMHost evidence indicates `0x20` was an early OVCM bit-position mistake; standards-clean OVCM is `0x04`.
|
||||
|
||||
## Decision
|
||||
Synthetic group voice LC uses normal service options `0x00` by default. OVCM is `0x04` if explicitly selected. HBLink `0x20` is legacy compatibility only. Real inbound LC is preserved.
|
||||
|
||||
## Rationale
|
||||
FreeDMR should not set reserved service-option bits in newly generated LC. Real inbound LC should not be rewritten without a deliberate reason.
|
||||
|
||||
## Consequences
|
||||
Generated fallback LC becomes cleaner. Some legacy interop assumptions may need live RF testing.
|
||||
|
||||
## Compatibility
|
||||
`0x20` may remain as a named legacy compatibility option, not as the default or as a traffic marker.
|
||||
|
||||
## Testing Requirements
|
||||
Codec/unit tests must assert synthetic LC defaults to `0x00`, OVCM uses `0x04`, real inbound LC is preserved, and `0x20` is only used when explicitly configured for compatibility.
|
||||
@ -1,22 +0,0 @@
|
||||
# ADR 0008: Data Packet Forwarding Policy
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
FreeDMR currently supports DMR data forwarding, including data gateway concepts and last-known-location routing. GPS/SMS application processing is better handled outside the core.
|
||||
|
||||
## Decision
|
||||
FreeDMR continues forwarding DMR data packets but does not become a general GPS/SMS application processor.
|
||||
|
||||
## Rationale
|
||||
Data support is part of network interoperability. Application processing would add complexity and CPU cost to the packet process and is better done by FBP-connected systems or sidecars.
|
||||
|
||||
## Consequences
|
||||
The core must preserve data packet routing semantics while keeping application parsing out of the hot path.
|
||||
|
||||
## Compatibility
|
||||
Existing `SUB_MAP` / last-known-location behaviour remains intentional. `DATA-GATEWAY` remains a supported concept where useful.
|
||||
|
||||
## Testing Requirements
|
||||
Tests must cover group-addressed data, unit-addressed data, last-known-location routing, data-vs-voice reporting, and preservation of data forwarding over FBP.
|
||||
@ -1,22 +0,0 @@
|
||||
# ADR 0009: Mesh Authentication Without Default Encryption
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
FreeDMR is an amateur-radio network. In many jurisdictions amateur-radio traffic must not be encrypted, and IP backhaul may itself use amateur-radio links.
|
||||
|
||||
## Decision
|
||||
Use authenticity, integrity, membership validation, and local policy; do not encrypt amateur-radio mesh traffic by default.
|
||||
|
||||
## Rationale
|
||||
Signing and authentication protect the mesh from impersonation and unauthorized traffic while preserving FreeDMR's open, inspectable, amateur-radio character.
|
||||
|
||||
## Consequences
|
||||
Traffic remains visible. Security focuses on who is allowed to inject or carry traffic, not secrecy.
|
||||
|
||||
## Compatibility
|
||||
Existing cleartext FBP/OBP interop remains possible. New authenticated admission can be introduced through bridge-control mechanisms and cached session state.
|
||||
|
||||
## Testing Requirements
|
||||
Tests must cover valid identity, invalid signature, revocation, endpoint change requiring re-authentication, grace expiry, and local policy overriding signed membership.
|
||||
@ -1,22 +0,0 @@
|
||||
# ADR 0010: API Is Control Plane Only
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
The API is useful for local administration, automation, and dashboard control. FreeDMR is a live voice stream program, so API work must not delay packet processing.
|
||||
|
||||
## Decision
|
||||
API operations are bounded control-plane actions and must not perform heavy serialization or block packet routing.
|
||||
|
||||
## Rationale
|
||||
Control operations should act on sessions, subscriptions, mesh peers, and reporting state. Heavy views should come from snapshots or reporting feeds.
|
||||
|
||||
## Consequences
|
||||
The API remains small and predictable. Complex dashboard state should not be assembled synchronously from hot packet state.
|
||||
|
||||
## Compatibility
|
||||
Old API endpoints may be adapted if needed, but the v2 API should not expose raw legacy internals as its primary contract.
|
||||
|
||||
## Testing Requirements
|
||||
Tests must cover auth, bounded request bodies, destructive-action gating, audit logging, and packet-path independence under API load.
|
||||
@ -1,22 +0,0 @@
|
||||
# ADR 0011: Process/Actor Model Over No-GIL Threading
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
FreeDMR may need more concurrency for reporting, export, analysis, or future scaling. Shared mutable routing state is risky.
|
||||
|
||||
## Decision
|
||||
If FreeDMR 2 needs concurrency beyond the reactor, prefer explicit parent/child or actor-style ownership boundaries over shared-memory no-GIL threading for routing state.
|
||||
|
||||
## Rationale
|
||||
Single-owner state and explicit messages are easier for sysops and contributors to reason about. They reduce race risks in delay-sensitive packet handling.
|
||||
|
||||
## Consequences
|
||||
Some features may require serialization and message protocols between processes. This is clearer than shared locks around routing dictionaries.
|
||||
|
||||
## Compatibility
|
||||
Twisted can remain the initial transport shell while workers handle reporting/export or expensive tasks.
|
||||
|
||||
## Testing Requirements
|
||||
Tests must cover worker failure, queue overflow, restart behaviour, message ordering where required, and packet handling continuing when a non-critical worker fails.
|
||||
@ -1,22 +0,0 @@
|
||||
# ADR 0012: Testing Gates for Protocol-Visible Change
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
FreeDMR behaviour was validated through real global servers, RF links, and community use. Protocol-visible changes can affect repeaters, terminals, dashboards, and peer servers.
|
||||
|
||||
## Decision
|
||||
Protocol-visible changes require deterministic, UDP, and live RF validation according to release gates.
|
||||
|
||||
## Rationale
|
||||
Deterministic tests catch state and rewrite errors. UDP black-box tests catch transport/subprocess/protocol integration issues. Live RF catches terminal/repeater quirks that harnesses cannot prove.
|
||||
|
||||
## Consequences
|
||||
Some changes take longer to release. The risk of breaking real deployments is reduced.
|
||||
|
||||
## Compatibility
|
||||
FreeDMR 1.x remains live while FreeDMR 2 behaviour is validated. Changes can be staged behind compatibility adapters.
|
||||
|
||||
## Testing Requirements
|
||||
Level 0 unit/support tests, Level 1 deterministic harness, Level 2 UDP black-box harness, and Level 3 live RF validation are required according to the risk and protocol visibility of the change.
|
||||
@ -1,87 +0,0 @@
|
||||
# ADR 0013: Worker Process Capacity Scaling
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
|
||||
FreeDMR currently relies heavily on a single Python process, Twisted reactor callbacks, and in-memory mutable state inherited from the HBLink-era architecture.
|
||||
|
||||
This is useful as an initial safety boundary because it avoids many shared-memory races, but it also creates practical capacity limits and makes some kinds of expensive work unsafe in the packet path.
|
||||
|
||||
CPython's GIL and single-reactor execution mean that CPU-bound work, reporting fanout, SQL/global export, analytics, and future codec/transcoding work should not be assumed to scale inside one process.
|
||||
|
||||
At the same time, FreeDMR's packet/routing state is subtle and protocol-sensitive. Moving it prematurely across process boundaries could introduce latency, ordering bugs, stale packet replay, duplicate packets, routing loops, broken source quench, or incorrect packet mutation.
|
||||
|
||||
## Decision
|
||||
|
||||
FreeDMR 2 will be designed for eventual worker-process scaling, but the first migration stage will keep Twisted as the transport shell and extract/test the routing/subscription core in-process.
|
||||
|
||||
The first worker-process targets are non-packet-path or low-risk side effects:
|
||||
|
||||
- Reporting/MQTT publisher.
|
||||
- Global lastheard exporter.
|
||||
- SQL/database writes.
|
||||
- Dashboard aggregation.
|
||||
- Alias refresh.
|
||||
- Analytics.
|
||||
- Packet replay/diagnostics.
|
||||
- Future codec/transcoding adjuncts.
|
||||
|
||||
Packet-plane routing may move behind a process/message boundary later, but only after state ownership, subscription lookup, stream lifecycle, loop control, source quench, and packet mutation semantics are covered by deterministic, UDP, and live RF tests.
|
||||
|
||||
FreeDMR 2 prefers explicit process/actor ownership boundaries over shared-memory threading or no-GIL Python for authoritative routing state.
|
||||
|
||||
## Rationale
|
||||
|
||||
- Worker processes provide clearer ownership and failure boundaries.
|
||||
- Explicit messages are easier to test than shared mutable dictionaries.
|
||||
- Reporting/export failures must not affect packet routing.
|
||||
- FreeDMR should be able to scale beyond one reactor process without making ordinary small deployments complex.
|
||||
- No-GIL Python does not remove the need for state ownership discipline.
|
||||
- A process model better matches FreeDMR's distributed-system nature: packet events, routing decisions, control messages, and reporting events are already conceptually separate.
|
||||
|
||||
## Consequences
|
||||
|
||||
Positive:
|
||||
|
||||
- Clearer state ownership.
|
||||
- Improved future capacity.
|
||||
- Safer isolation of reporting/export/database work.
|
||||
- Better failure containment.
|
||||
- Better testability of message boundaries.
|
||||
- Easier future sharding by client, TG, stream, listener, or mesh peer.
|
||||
|
||||
Negative:
|
||||
|
||||
- More implementation complexity.
|
||||
- Message schemas must be designed and versioned.
|
||||
- IPC adds latency and failure modes.
|
||||
- Routing-worker split requires strong tests.
|
||||
- Worker supervision and restart policy become part of the system design.
|
||||
|
||||
## Compatibility
|
||||
|
||||
FreeDMR 1.x/current code remains live until FreeDMR 2 is ready.
|
||||
|
||||
Initial FreeDMR 2 worker work should not change packet semantics.
|
||||
|
||||
Legacy dashboard/reporting compatibility, if required, belongs in reporting/export adapters, not in the packet core.
|
||||
|
||||
A single-process deployment must remain supported for small servers.
|
||||
|
||||
## Testing Requirements
|
||||
|
||||
Before any packet-plane worker split:
|
||||
|
||||
- Deterministic harness coverage of the state machine.
|
||||
- Message-boundary equivalence tests.
|
||||
- UDP black-box equivalence tests.
|
||||
- Packet byte preservation tests.
|
||||
- Allowed rewrite-region tests.
|
||||
- Source quench/STUN/loop-control tests.
|
||||
- Duplicate/out-of-order tests.
|
||||
- Worker crash/restart tests.
|
||||
- Stale packet replay prevention tests.
|
||||
- Queue backpressure tests.
|
||||
- Live RF validation for protocol-visible behaviour.
|
||||
Loading…
Reference in new issue