FreeDMR/docs/hdstack.md

# HDStack multi-process deployment

HDStack raises the aggregate hotspot capacity of one FreeDMR container by
running up to five independent FreeDMR HBP workers. It does not raise the
connection limit of an individual worker. The default worker count is the
available CPU count minus one, clamped to the supported range of one through
five.

Omitting `HDSTACK` selects that CPU-derived default. Setting `HDSTACK=1`
explicitly preserves the existing standard single-FreeDMR process layout.
Setting `HDSTACK` to a value from 2 through 5 selects the generated
multi-process layout:

```text
hotspots -> proxy -> HBP workers -> no-HBP aggregator -> external FBP/OBP
                       |              |
                       +-> Loro       +-> optional bridge.py over FBP v5/FBCP
                       +-> Data Gateway over FBP v5/FBCP

worker reports -> reporting MUX -> existing dashboard/reporting consumer
```

The aggregator owns all OpenBridge/FBP links from the main `freedmr.cfg` and
has no HBP `MASTER`, `PEER` or `XLXPEER` system. The optional `bridge.py`
process retains its own configuration and rules. The reporting MUX reads only
worker reports; aggregator and bridge reporting are not included.

## Installer and container configuration

The maintained installer asks for the main server ID, proposes the CPU-derived
worker count, and asks whether the separate bridge process is required. It
writes the resulting values directly into `/etc/freedmr/docker-compose.yml`.

```yaml
environment:
  - HDSTACK=5
  - HDSTACK_BASEID=23400
  - HDSTACK_BRIDGE=0
```

`HDSTACK_BASEID` is the aggregator server ID. Worker IDs are allocated by
adding their one-based worker number. For the example above, the aggregator is
`23400` and worker IDs continue from `23401` through `23405`.

The separate `bridge.py` process is disabled by default. Enable it when the
deployment requires bridge rules or HBP peer/XLX connections:

```yaml
environment:
  - HDSTACK_BRIDGE=1
```

`HDSTACK_BRIDGE` accepts only `0` or `1`. It affects only multi-process mode.
When disabled, no `bridge.py` process is started.

The installer provides `freedmr-bridge.cfg` and `rules-bridge.py` skeletons.
The latter contains the required `BRIDGES = {}` structure, so the optional
process starts idle until routing rules are configured. Enabling the bridge
also requires a separate bridge server ID which must not collide with the
aggregator or worker IDs.

The standard Compose file mounts these inputs:

```yaml
volumes:
  - '/etc/freedmr/freedmr.cfg:/opt/freedmr/freedmr.cfg'
  - '/etc/freedmr/freedmr-bridge.cfg:/opt/freedmr/freedmr-bridge.cfg:ro'
  - '/etc/freedmr/rules-bridge.py:/opt/freedmr/rules.py:ro'
```

The paths can be changed with:

```yaml
environment:
  - HDSTACK_BRIDGE_CONFIG=/opt/freedmr/freedmr-bridge.cfg
  - HDSTACK_BRIDGE_RULES=/opt/freedmr/rules.py
```

The main `freedmr.cfg` remains the sysop-facing aggregator and worker-template
configuration. It must contain one enabled `[SYSTEM]` stanza in `MASTER` mode.
All enabled and disabled `OPENBRIDGE` sections except `[DATA-GATEWAY]` are
copied to the aggregator; this includes external FBP/OBP links. Other protocol
systems, including `[SYSTEM]` and `[ECHO]`, are not copied to the aggregator.

Each worker receives the same `[SYSTEM]` policy. Only its base HBP port is
derived. When the native Data Gateway is enabled, each worker also receives a
direct `[DATA-GATEWAY]` FBP v5/FBCP relationship. Worker and aggregator
configuration files are generated privately in `/dev/shm/freedmr-hdstack` when
the container starts. The mounted source configuration and bridge files are
not changed.

When `[ECHO]` is enabled, each worker receives its own generated ECHO peer and
Loro playback process on a private loopback port pair. Calls cannot leak
between worker playback state, and each Loro accepts only its corresponding
worker.

When enabled, the bridge configuration supplies its distinct server ID and any
`PEER` or `XLXPEER` systems used by `bridge.py`. HDStack generates the
reciprocal private FBP v5/FBCP connection to the aggregator. Its `rules.py` remains
the authority for bridge routing. Bridge reporting must either be disabled or
use a port which does not collide with the worker reports or MUX.

## Worker ports

The `[SYSTEM]` `PORT` and `GENERATOR` values define each worker range. With the
standard values `PORT=54000` and `GENERATOR=100`:

```text
worker 1 -> 54000..54099
worker 2 -> 54100..54199
worker 3 -> 54200..54299
worker 4 -> 54300..54399
worker 5 -> 54400..54499
```

The generated ranges must fit within the UDP port range and must not collide
with configured bridge, external OpenBridge or reporting ports.

The internal worker path consumes one additional FBP hop. FreeDMR accepts a
maximum of 20 FBP hops before discarding traffic; this ceiling applies to all
FBP relationships rather than being a special HDStack exception.

Workers connect to the aggregator over generated loopback FBP v5 links with
FBCP enabled. FBP v5 preserves origin metadata and carries unit data as well as
group traffic; FBCP provides source quench and the existing link-control
behaviour. Strict reciprocal endpoint checks remain enabled because both ends
use fixed loopback addresses and ports. Worker ports are `7001` through
`7005`; reciprocal aggregator ports are `7101` through `7105`. These links
are private to the container.

The generated section names retain the historical `OBP-HDSTACK-*` prefix even
though their configured wire version is FBP v5. Existing
`bridge_master.py` dynamic bridge construction recognises that prefix.

When the optional bridge process is enabled, it connects to the aggregator over
the same strict private FBP v5/FBCP profile. The aggregator binds UDP port
`7200` and the bridge binds UDP port `7201`. The generated peer identity on
each side is the other process's configured server ID. External FBP and OBP
relationships remain on the aggregator.

Each generated Loro pair uses successive private ports beginning with the
existing `[ECHO]` peer and master ports, normally `54916` and `54915`.

## Data Gateway

New installations include a separate FreeDMR Data Gateway container for
D-APRS and future packet-data services. The installer asks for the APRS-IS
login callsign and passcode, generates a private FBP key, enables the
`[DATA-GATEWAY]` relationship, and writes matching relationships into the
Compose service. Existing configurations keep `DATA_GATEWAY: False` and the
disabled section by default, so replacing only the FreeDMR image does not
remove the legacy D-APRS path.

Each worker connects directly to the gateway. The aggregator and optional
`bridge.py` process do not. The default ports are:

```text
worker 1 62041 -> gateway 62031
worker 2 62042 -> gateway 62032
worker 3 62043 -> gateway 62033
worker 4 62044 -> gateway 62034
worker 5 62045 -> gateway 62035
```

FreeDMR sends every admitted, locally originated HBP DMR packet to this link,
including unit data, group data and voice packets. Peer or mesh ingress is not
copied back to the gateway. The complete 33-byte DMR burst is preserved; the
gateway owns decoding, APRS consent and application policy. Enabling the native
gateway disables legacy destination-`900999` D-APRS delivery to avoid duplicate
processing. It is not an automatic fallback if the native link is unavailable.

The default installation runs the gateway locally, but this is not a required
one-gateway-per-server architecture. A sysop may point each worker relationship
at a shared remote gateway by configuring matching contiguous target ports,
peer IDs and keys, then removing or disabling the local Compose service. The
gateway endpoint must be reachable from the FreeDMR container; `localhost`
would address the FreeDMR container itself, not a separate gateway container.

### Migrating the legacy D-APRS container

`upgrade-daprs-to-data-gateway.sh` migrates the documented legacy `D-APRS`
Compose service to the native Data Gateway without replacing unrelated
FreeDMR configuration or Compose services. Download it, inspect it, and run a
dry run first:

```sh
curl -fsSL \
  https://gitlab.hacknix.net/hacknix/FreeDMR/-/raw/master/docker-configs/upgrade-daprs-to-data-gateway.sh \
  -o /tmp/upgrade-daprs-to-data-gateway.sh
chmod 755 /tmp/upgrade-daprs-to-data-gateway.sh
sudo /tmp/upgrade-daprs-to-data-gateway.sh --dry-run
sudo /tmp/upgrade-daprs-to-data-gateway.sh
```

The script reads the server ID and any existing Data Gateway ports and
non-placeholder passphrase from `freedmr.cfg`. It reads the HDStack settings
and legacy `APRS_CALL` and `APRS_PASSCODE` from `docker-compose.yml`. If
`HDSTACK` is
omitted, it applies the container's CPU-count-minus-one default, clamped to one
through five. A private FBP key is generated only when no usable one exists.
Missing or invalid existing values cause the migration to stop rather than
prompt for replacement configuration.

Before changing either file, the script validates the proposed Compose
configuration and asks for confirmation. It creates timestamped backups beside
`/etc/freedmr/freedmr.cfg` and `/etc/freedmr/docker-compose.yml`. It refuses an
unrecognised layout or an existing native `data-gateway` service rather than
guessing. The script does not pull images or restart containers; follow the
commands it prints after reviewing the generated files.

## Session assignment

The proxy rotates new DMR IDs through the configured worker ranges. With two
workers the assignments are:

```text
new session 1 -> worker 1
new session 2 -> worker 2
new session 3 -> worker 1
new session 4 -> worker 2
```

Within the selected worker range, the proxy retains its existing random choice
of an available generated system port.
Subscriber location remains best-effort state. A worker records the originating
HBP repeater or hotspot ID, slot and observation time rather than a worker-local
generated system name. Authenticated FBP v5 traffic received through the
private aggregator relationship also updates this map when its source server
and hop path identify another worker in the same HDStack deployment.

Each worker resolves a unit-data destination against its currently connected
HBP peers before transmitting. This allows a hotspot or repeater to reconnect
on another worker without requiring the subscriber to transmit again first,
provided the subscriber-to-access-peer observation has already propagated.
Workers retain separate persistence files; they do not concurrently write one
shared pickle file.


A DMR ID remains pinned to that exact generated port for the lifetime of its
active proxy session. Moving it would start a new HBP login and reset its
BRIDGES and OPTIONS state. The proxy does not rebalance or fail over an active
session. Once the session expires, a later login by the same DMR ID is a new
assignment. Affinity is not persisted across proxy restarts.

Existing explicit proxy configurations using `DestportStart` and `DestportEnd`
continue to describe one range. The equivalent direct multi-range setting is:

```ini
[PROXY]
BackendPortRanges: [[54000, 54099], [54100, 54199]]
```

or:

```sh
FDPROXY_BACKEND_PORT_RANGES='[[54000,54099],[54100,54199]]'
```

The standard HDStack entrypoint supplies this setting automatically.

## Proxy diagnostics

In multi-worker mode the proxy logs each backend number and port range during
startup. With the standard two-worker layout this includes:

```text
(PROXY)(HDSTACK) Backend:1 ports:54000-54099.
(PROXY)(HDSTACK) Backend:2 ports:54100-54199.
```

When `FDPROXY_CLIENTINFO=1`, as in the shipped Compose configuration, client
assignment and removal messages include both the backend number and exact
generated port. This makes round-robin assignment and retained session
affinity visible without enabling packet-level debug logging. Single-backend
startup output remains unchanged.

## Reporting MUX

When reporting is enabled in the main configuration, worker report ports are
derived from its `REPORT_PORT`. With the default output port `4321`:

```text
MUX output -> 4321
worker 1   -> 4322
worker 2   -> 4323
worker 3   -> 4324
worker 4   -> 4325
worker 5   -> 4326
```

The MUX preserves the existing netstring reporting protocol and the main
configuration's `REPORT_CLIENTS` allow-list. It merges worker configuration
and bridge snapshots and relays worker bridge events. System names are
prefixed with the stable worker number to prevent collisions:

```text
1:SYSTEM-001
2:SYSTEM-001
```

Only the system-name field is changed. The reporting protocol contains trusted
Python pickle data and must remain on a trusted network.

A failed or malformed worker reporting source is isolated from healthy
sources. Reporting failure does not affect the proxy or DMR traffic. If main
reporting is disabled, worker reporting and the MUX are both disabled.

## Limitations

HDStack deliberately provides no least-connections balancing, active backend
failover, live migration, persistent affinity or automatic reconfiguration.
A failed worker does not cause its active sessions to move to another worker.
The separate bridge config and rules remain sysop-managed. The runtime
generator creates the private transport link but does not create routing
policy.