Merge pull request #31 from ce5rpy/develop

Release 2.2.2

docs/en/monitor/self-service.md

@@ -65,6 +65,14 @@ sequenceDiagram
 
 Important: the proxy sends **RPTO to the master only**, not to the hotspot directly. If the proxy is not in the path, you must ensure another mechanism applies **OPTIONS** or you run the server without this proxy path.
 
+> **Prerequisite:** the hotspot must send `PASS=` in its OPTIONS line for
+> password login and bidirectional DB sync. If the hotspot sends explicit
+> content (TGs, SINGLE, etc.) **without** `PASS=`, the server takes options
+> directly from that line and the DB row is ignored. If the hotspot sends no
+> RPTO at all (timer expires) or an empty OPTIONS, the server falls back to
+> the database.
+> See [Hotspot proxy — OPTIONS line behaviour](../server/user-guide/hotspot-proxy.md#options-line-behaviour).
+
 ---
 
 ## Password hashing

docs/en/server/user-guide/hotspot-proxy.md

@@ -96,6 +96,37 @@ Details of the dashboard flow: [Self-service](../../monitor/self-service.md).
 
 ---
 
+## OPTIONS line behaviour
+
+After login (RPTL → RPTK → RPTC), the proxy starts a **10 s timer** waiting for
+the hotspot's **RPTO** packet with its **OPTIONS** line. What the hotspot sends
+(or does not send) determines **who is the source of truth** for the peer's
+static talkgroups:
+
+| Hotspot sends in the RPTO | Who defines the TGs | Behaviour |
+|---|---|---|
+| `OPTIONS=PASS=xxxxxx;` | **Self-service** (dashboard) | The proxy processes the `PASS=`, verifies the individual password (PBKDF2 against `Clients.psswd`), marks the peer as authenticated, cancels the 10 s timer, and pushes the DB-configured TGs to the master. The user **can** log in by password and auto-login by IP on the dashboard. |
+| `OPTIONS=` (empty) | **Self-service** (dashboard) | The proxy reads the TGs from the DB and injects them to the master. The user can **only** use auto-login by IP on the dashboard (no password). |
+| **No RPTO at all** (10 s timer expires) | **Self-service** (dashboard) | The proxy assumes the hotspot has no OPTIONS of its own and falls back to the DB. Same effect as `OPTIONS=` empty. |
+| `OPTIONS=TS2=730444;SINGLE=0;` (content without `PASS=`) | **The hotspot itself** | The master takes the TGs **directly from the OPTIONS line**. The DB is ignored. The user can **only** use auto-login by IP on the dashboard (no password). |
+
+**Key rule:** self-service is the source of truth **unless** the hotspot sends
+explicit content (TGs, SINGLE, TIMER, etc.) in its OPTIONS line. In that case,
+what the hotspot says **takes precedence** and self-service is ignored.
+
+### Individual password and dashboard login
+
+- If the hotspot **never** sends `PASS=` in its RPTO, the user **cannot** log in
+  to the dashboard with a password. They can only use **auto-login by IP**
+  (if their IP matches `Clients.host`).
+- To enable password login on the dashboard, the hotspot must send
+  `OPTIONS=PASS=your_password;` in its configuration (Pi-Star / WPSD / MMDVM
+  `optsfile`). The password must match the PBKDF2 hash stored in `Clients.psswd`.
+- The `PASS=` flow is what activates bidirectional sync: the proxy stores the
+  hash, notes the `modified` flag, and pushes the DB-configured TGs to the master.
+
+---
+
 ## Hot reload (`SIGHUP`)
 
 **Applied without restart** (active proxy sessions stay up):

docs/en/server/user-guide/special-numbers.md

@@ -98,7 +98,7 @@ The **audio** is sent with **source ID 5000** and **destination TG 9** in the ge
 
 **Purpose:** Bridge rows for **echo** often use **9990** with the **ECHO** system (see `BRIDGES` and options in your YAML).
 
-**`SINGLE=1`:** Keying **9990** does **not** create an exclusive listen session (same as **4000**). Downlink echo always returns to the calling hotspot even when another TG holds the SINGLE lock. See [Hotspot proxy](hotspot-proxy.md#behaviour-with-multiple-hotspots) and [Voice routing and contention — SINGLE exceptions](../development/routing-and-contention.md#single-exceptions).
+**`SINGLE=1`:** Keying **9990** does **not** create an exclusive listen session (same as **4000**). Downlink echo always returns to the calling hotspot even when another TG holds the SINGLE lock. See [Hotspot proxy](hotspot-proxy.md#multi-hotspot-behaviour) and [Voice routing and contention — SINGLE exceptions](../development/routing-and-contention.md#single-exceptions).
 
 **Note:** A **standalone echo** is also available as a separate process — [Echo](echo.md).
 

docs/es/monitor/self-service.md

@@ -65,6 +65,14 @@ sequenceDiagram
 
 Importante: el proxy envía **RPTO solo al master**, no al hotspot directamente. Si el proxy no está en el camino, necesitas otro mecanismo que aplique **OPTIONS** o ejecutas el servidor sin este camino proxy.
 
+> **Prerrequisito:** el hotspot debe enviar `PASS=` en su línea OPTIONS para
+> login por contraseña y sincronización bidireccional con la BD. Si el hotspot
+> envía contenido explícito (TGs, SINGLE, etc.) **sin** `PASS=`, el servidor
+> toma las opciones directamente de esa línea y la fila de la BD se ignora. Si
+> el hotspot no envía RPTO (el timer expira) o envía OPTIONS vacío, el servidor
+> hace fallback a la base de datos.
+> Ver [Proxy hotspot — comportamiento de la línea OPTIONS](../server/user-guide/hotspot-proxy.md#comportamiento-de-la-linea-options).
+
 ---
 
 ## Hash de contraseñas

docs/es/server/user-guide/hotspot-proxy.md

@@ -96,6 +96,38 @@ Detalle del flujo en el panel: [Self-service](../../monitor/self-service.md).
 
 ---
 
+## Comportamiento de la línea OPTIONS
+
+Tras el login (RPTL → RPTK → RPTC), el proxy arranca un **timer de 10 s**
+esperando el paquete **RPTO** del hotspot con su línea **OPTIONS**. Lo que el
+hotspot envíe (o no envíe) en ese RPTO determina **quién es la fuente de
+verdad** de los TG estáticos del peer:
+
+| El hotspot envía en el RPTO | Quién define los TG | Comportamiento |
+|---|---|---|
+| `OPTIONS=PASS=xxxxxx;` | **Auto-servicio** (panel web) | El proxy procesa el `PASS=`, verifica la contraseña individual (PBKDF2 contra `Clients.psswd`), marca al peer como autenticado, cancela el timer de 10 s y empuja los TG de la BD al master. El usuario **puede** hacer login por contraseña y auto-login por IP en el dashboard. |
+| `OPTIONS=` (vacío) | **Auto-servicio** (panel web) | El proxy lee los TG de la BD y los inyecta al master. El usuario **solo** puede usar auto-login por IP en el dashboard (sin contraseña). |
+| **No envía RPTO** (el timer de 10 s expira) | **Auto-servicio** (panel web) | El proxy asume que el hotspot no tiene OPTIONS propias y hace fallback a la BD. Mismo efecto que `OPTIONS=` vacío. |
+| `OPTIONS=TS2=730444;SINGLE=0;` (contenido sin `PASS=`) | **El propio hotspot** | El master toma los TG **directamente de la línea OPTIONS**. Se ignora la BD. El usuario **solo** puede usar auto-login por IP en el dashboard (sin contraseña). |
+
+**Regla clave:** el auto-servicio es la fuente de verdad **excepto** cuando el
+hotspot envía contenido explícito (TGs, SINGLE, TIMER, etc.) en su línea
+OPTIONS. En ese caso, lo que dice el hotspot **priman** y el auto-servicio se ignora.
+
+### Contraseña individual y login por dashboard
+
+- Si el hotspot **nunca** envía `PASS=` en su RPTO, el usuario **no** podrá
+  entrar al dashboard con contraseña. Solo podrá usar **auto-login por IP**
+  (si su IP coincide con `Clients.host`).
+- Para habilitar el login por contraseña en el dashboard, el hotspot debe enviar
+  `OPTIONS=PASS=tu_contraseña;` en su configuración (Pi-Star / WPSD / MMDVM
+  `optsfile`). La contraseña debe coincidir con el hash PBKDF2 almacenado en
+  `Clients.psswd`.
+- El flujo `PASS=` es lo que activa la sincronización bidireccional: el proxy
+  almacena el hash, registra el flag `modified` y empuja los TG de la BD al master.
+
+---
+
 ## Recarga en caliente (`SIGHUP`)
 
 **Se aplica sin reiniciar** (las sesiones activas del proxy se mantienen):

src/adn_server/application/ports.py

@@ -363,8 +363,8 @@ class ProxySelfServiceStore(ABC):
         ...
 
     @abstractmethod
-    def clean_tbl(self) -> Any:
-        """Returns Deferred."""
+    def reconcile_logged_in(self, connected_peer_ids: list[bytes]) -> Any:
+        """logged_in=1 for connected peers, 0 for the rest. Returns Deferred."""
 
 
 class DynamicTgStore(ABC):

src/adn_server/infrastructure/proxy/null_self_service.py

@@ -67,7 +67,7 @@ class NullProxySelfServiceStore(ProxySelfServiceStore):
     def updt_lstseen(self, dmrid_list: list[tuple[bytes, ...]]) -> None:
         pass
 
-    def clean_tbl(self) -> Any:
+    def reconcile_logged_in(self, connected_peer_ids: list[bytes]) -> Any:
         from twisted.internet.defer import succeed
 
         return succeed(None)

src/adn_server/infrastructure/proxy/persistence/proxy_repository.py

@@ -141,8 +141,28 @@ class ProxySelfServiceRepository(ProxySelfServiceStore):
                 )
             )
 
-    @inlineCallbacks
-    def clean_tbl(self) -> Any:
-        yield self._pool.runOperation(
-            "UPDATE Clients SET logged_in=0 WHERE logged_in=1 AND last_seen < UNIX_TIMESTAMP() - 86400"
-        ).addErrback(lambda f: logger.error("(SELF_SERVICE) clean_tbl: %s", f.getTraceback()))
+    def reconcile_logged_in(self, connected_peer_ids: list[bytes]) -> Any:
+        if connected_peer_ids:
+            ph = ",".join(["%s"] * len(connected_peer_ids))
+            self._pool.runOperation(
+                f"UPDATE Clients SET logged_in=1 WHERE dmr_id IN ({ph})",
+                tuple(connected_peer_ids),
+            ).addErrback(
+                lambda f: logger.error(
+                    "(SELF_SERVICE) reconcile set 1: %s", f.getTraceback()
+                )
+            )
+            self._pool.runOperation(
+                f"UPDATE Clients SET logged_in=0 WHERE dmr_id NOT IN ({ph})",
+                tuple(connected_peer_ids),
+            ).addErrback(
+                lambda f: logger.error(
+                    "(SELF_SERVICE) reconcile set 0: %s", f.getTraceback()
+                )
+            )
+        else:
+            self._pool.runOperation("UPDATE Clients SET logged_in=0").addErrback(
+                lambda f: logger.error(
+                    "(SELF_SERVICE) reconcile clear: %s", f.getTraceback()
+                )
+            )

src/adn_server/infrastructure/proxy/self_service_bridge.py

@@ -81,18 +81,17 @@ class ProxySelfServiceBridge:
         self._mysql_option_peers: set[bytes] = set()
 
     def start_loops(self) -> None:
-        """Legacy timers: send_opts 10s, lst_seen 120s, clean_tbl 3600s."""
-        for interval, fn in (
-            (10.0, self.send_opts),
-            (120.0, self.lst_seen),
-            (3600.0, self._clean_tbl),
+        """Timers: send_opts 10s, lst_seen+reconcile 120s (now=True for startup clean slate)."""
+        for interval, fn, now in (
+            (10.0, self.send_opts, False),
+            (120.0, self.lst_seen, True),
         ):
             call = LoopingCall(fn)
-            call.start(interval, now=False)
+            call.start(interval, now=now)
             self._loop_calls.append(call)
         self._log.info(
             "(SELF_SERVICE) DB options on PASS= (immediate), send_opts every 10s, "
-            "clean_tbl every 1h, lst_seen every 2min"
+            "lst_seen + reconcile_logged_in every 2min"
         )
 
     def stop_loops(self) -> None:
@@ -349,9 +348,8 @@ class ProxySelfServiceBridge:
             self._log.warning("(SELF_SERVICE) send_opts error: %s", err)
 
     def lst_seen(self) -> None:
-        dmrid_list = [(slot.peer_id,) for slot in self._use_cases.list_slots()]
+        slots = self._use_cases.list_slots()
+        dmrid_list = [(slot.peer_id,) for slot in slots]
         if dmrid_list:
             self._store.updt_lstseen(dmrid_list)
-
-    def _clean_tbl(self) -> None:
-        self._store.clean_tbl()
+        self._store.reconcile_logged_in([slot.peer_id for slot in slots])

tests/infrastructure/test_proxy_self_service.py

@@ -22,6 +22,8 @@
 
 from __future__ import annotations
 
+from typing import Any
+
 from twisted.internet.defer import Deferred
 
 from adn_server.application.proxy import ProxyUseCases
@@ -56,6 +58,7 @@ class _FakeStore(NullProxySelfServiceStore):
         self.actions: list[tuple[str, bytes]] = []
         self.options_by_peer: dict[bytes, str] = {}
         self.pending_modified: list[tuple[bytes, str]] = []
+        self.reconcile_calls: list[list[bytes]] = []
 
     def ins_conf(
         self,
@@ -89,6 +92,15 @@ class _FakeStore(NullProxySelfServiceStore):
 
         return succeed([(pid, opt) for pid, opt in self.pending_modified])
 
+    def updt_lstseen(self, dmrid_list: list[tuple[bytes, ...]]) -> None:
+        self.actions.append(("updt_lstseen", b""))
+
+    def reconcile_logged_in(self, connected_peer_ids: list[bytes]) -> Any:
+        from twisted.internet.defer import succeed
+
+        self.reconcile_calls.append(list(connected_peer_ids))
+        return succeed(None)
+
 
 def _bridge() -> tuple[ProxySelfServiceBridge, _RecordingSink, _FakeStore, _RecordingSender]:
     store = _FakeStore()
@@ -302,6 +314,33 @@ def test_session_expired_logs_out() -> None:
     assert ("log_out", peer) in store.actions
 
 
+def test_lst_seen_reconciles_connected_peers() -> None:
+    """lst_seen calls reconcile_logged_in with the connected peer IDs."""
+    bridge, _sink, store, _sender = _bridge()
+    peer = bytes_4(7300444)
+    bridge.lst_seen()
+    assert store.reconcile_calls == [[peer]]
+
+
+def test_lst_seen_reconciles_empty_when_no_peers() -> None:
+    """lst_seen with no slots calls reconcile_logged_in([]) — startup clean slate."""
+    from adn_server.application.proxy import ProxyUseCases
+    from adn_server.infrastructure.proxy.rpto_queue import InMemoryPendingRptoQueue
+    from adn_server.infrastructure.proxy.slot_store import InMemoryProxySlotStore
+
+    store = _FakeStore()
+    sink = _RecordingSink()
+    sender = _RecordingSender()
+    use_cases = ProxyUseCases(
+        InMemoryProxySlotStore(), InMemoryPendingRptoQueue(), max_peers=4
+    )
+    bridge = ProxySelfServiceBridge(
+        store, use_cases, sink, sender, pbkdf2_salt="ADN", pbkdf2_iterations=2000
+    )
+    bridge.lst_seen()
+    assert store.reconcile_calls == [[]]
+
+
 def test_yaml_loader_preserves_self_service_block(tmp_path) -> None:
     """SELF_SERVICE from adn-server.yaml must reach runtime (not stripped at load)."""
     from adn_server.infrastructure.config_loader import YamlConfigLoader

tests/infrastructure/test_proxy_self_service_repository.py

@@ -0,0 +1,72 @@
+# ADN DMR Peer Server - tests infrastructure proxy self service repository
+#
+# Copyright (C) 2026  Rodrigo Pérez, CE5RPY <ce5rpy@qmd.cl>
+#
+###############################################################################
+#   This program is free software; you can redistribute it and/or modify
+#   it under the terms of the GNU General Public License as published by
+#   the Free Software Foundation; either version 3 of the License, or
+#   (at your option) any later version.
+#
+#   This program is distributed in the hope that it will be useful,
+#   but WITHOUT ANY WARRANTY; without even the implied warranty of
+#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#   GNU General Public License for more details.
+#
+#   You should have received a copy of the GNU General Public License
+#   along with this program; if not, write to the Free Software Foundation,
+#   Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301  USA
+###############################################################################
+
+"""ProxySelfServiceRepository SQL for reconcile_logged_in (mock pool)."""
+
+from __future__ import annotations
+
+from unittest.mock import MagicMock
+
+from twisted.internet.defer import succeed
+
+from adn_server.infrastructure.proxy.persistence.proxy_repository import (
+    ProxySelfServiceRepository,
+)
+
+
+def _repo() -> ProxySelfServiceRepository:
+    pool = MagicMock()
+    pool.runOperation.return_value = succeed(None)
+    return ProxySelfServiceRepository(pool)
+
+
+def test_reconcile_empty_clears_all_logged_in() -> None:
+    repo = _repo()
+    repo.reconcile_logged_in([])
+    sql = repo._pool.runOperation.call_args[0][0]  # noqa: SLF001
+    assert sql == "UPDATE Clients SET logged_in=0"
+
+
+def test_reconcile_with_peers_sets_in_and_not_in() -> None:
+    repo = _repo()
+    peer_a = b"\x00\x70\x22\x34"
+    peer_b = b"\x00\x70\x22\x35"
+    repo.reconcile_logged_in([peer_a, peer_b])
+    calls = repo._pool.runOperation.call_args_list  # noqa: SLF001
+    assert len(calls) == 2
+    set_1_sql, set_1_args = calls[0][0]
+    set_0_sql, set_0_args = calls[1][0]
+    assert "logged_in=1" in set_1_sql
+    assert "IN (%s,%s)" in set_1_sql
+    assert set_1_args == (peer_a, peer_b)
+    assert "logged_in=0" in set_0_sql
+    assert "NOT IN (%s,%s)" in set_0_sql
+    assert set_0_args == (peer_a, peer_b)
+
+
+def test_reconcile_single_peer_uses_one_placeholder() -> None:
+    repo = _repo()
+    peer = b"\x00\x70\x22\x34"
+    repo.reconcile_logged_in([peer])
+    calls = repo._pool.runOperation.call_args_list  # noqa: SLF001
+    assert len(calls) == 2
+    set_1_sql, set_1_args = calls[0][0]
+    assert "IN (%s)" in set_1_sql
+    assert set_1_args == (peer,)