FreeDMR/docs/freedmr-2/adr/0013-worker-process-capacity-scaling.md

ADR 0013: Worker Process Capacity Scaling

Status

Proposed

Context

FreeDMR currently relies heavily on a single Python process, Twisted reactor callbacks, and in-memory mutable state inherited from the HBLink-era architecture.

This is useful as an initial safety boundary because it avoids many shared-memory races, but it also creates practical capacity limits and makes some kinds of expensive work unsafe in the packet path.

CPython's GIL and single-reactor execution mean that CPU-bound work, reporting fanout, SQL/global export, analytics, and future codec/transcoding work should not be assumed to scale inside one process.

At the same time, FreeDMR's packet/routing state is subtle and protocol-sensitive. Moving it prematurely across process boundaries could introduce latency, ordering bugs, stale packet replay, duplicate packets, routing loops, broken source quench, or incorrect packet mutation.

Decision

FreeDMR 2 will be designed for eventual worker-process scaling, but the first migration stage will keep Twisted as the transport shell and extract/test the routing/subscription core in-process.

The first worker-process targets are non-packet-path or low-risk side effects:

• Reporting/MQTT publisher.
• Global lastheard exporter.
• SQL/database writes.
• Dashboard aggregation.
• Alias refresh.
• Analytics.
• Packet replay/diagnostics.
• Future codec/transcoding adjuncts.

Packet-plane routing may move behind a process/message boundary later, but only after state ownership, subscription lookup, stream lifecycle, loop control, source quench, and packet mutation semantics are covered by deterministic, UDP, and live RF tests.

FreeDMR 2 prefers explicit process/actor ownership boundaries over shared-memory threading or no-GIL Python for authoritative routing state.

Rationale

• Worker processes provide clearer ownership and failure boundaries.
• Explicit messages are easier to test than shared mutable dictionaries.
• Reporting/export failures must not affect packet routing.
• FreeDMR should be able to scale beyond one reactor process without making ordinary small deployments complex.
• No-GIL Python does not remove the need for state ownership discipline.
• A process model better matches FreeDMR's distributed-system nature: packet events, routing decisions, control messages, and reporting events are already conceptually separate.

Consequences

Positive:

• Clearer state ownership.
• Improved future capacity.
• Safer isolation of reporting/export/database work.
• Better failure containment.
• Better testability of message boundaries.
• Easier future sharding by client, TG, stream, listener, or mesh peer.

Negative:

• More implementation complexity.
• Message schemas must be designed and versioned.
• IPC adds latency and failure modes.
• Routing-worker split requires strong tests.
• Worker supervision and restart policy become part of the system design.

Compatibility

FreeDMR 1.x/current code remains live until FreeDMR 2 is ready.

Initial FreeDMR 2 worker work should not change packet semantics.

Legacy dashboard/reporting compatibility, if required, belongs in reporting/export adapters, not in the packet core.

A single-process deployment must remain supported for small servers.

Testing Requirements

Before any packet-plane worker split:

• Deterministic harness coverage of the state machine.
• Message-boundary equivalence tests.
• UDP black-box equivalence tests.
• Packet byte preservation tests.
• Allowed rewrite-region tests.
• Source quench/STUN/loop-control tests.
• Duplicate/out-of-order tests.
• Worker crash/restart tests.
• Stale packet replay prevention tests.
• Queue backpressure tests.
• Live RF validation for protocol-visible behaviour.