diff --git a/docs/en/monitor/architecture.md b/docs/en/monitor/architecture.md index 6d3cc33..cafde7c 100644 --- a/docs/en/monitor/architecture.md +++ b/docs/en/monitor/architecture.md @@ -10,6 +10,14 @@ Under `monitor/src/adn_monitor/`: Composition root: `infrastructure/fastapi/composition.py` (`build_monitor_api`). +```mermaid +flowchart TD + INF["Infrastructure
YAML · FastAPI /ws · TCP/MQTT ingest
MySQL repos · pickle/JSON decoders"] + APP["Application
MonitorState · report/dashboard use cases
auth · self-service · aliases"] + DOM["Domain
value objects · errors · opcodes
UserSession · Result"] + INF --> APP --> DOM +``` + ## Unified process (`monitor.py`) Single uvicorn/FastAPI process: @@ -41,15 +49,19 @@ The standalone **`adn-proxy`** process was removed from the **adn-monitor** repo ## Typical deployment topology -```text -[Hotspots] --UDP--> [adn-server PROXY] --UDP--> [peer MASTER] - | - v - MySQL (Clients) +```mermaid +flowchart TD + HS[Hotspots] -->|UDP HBP| PROXY[adn-server PROXY] + PROXY --> MASTER[MASTER inject] + PROXY --> DB[(MySQL Clients)] -[Peer server :REPORT_PORT] <--- TCP or MQTT --- [monitor.py ingest] + MASTER --> REPORT[REPORTS listener] + REPORT <-->|TCP or MQTT| ING[monitor.py ingest] + ING --> APP[FastAPI /api /ws] + APP --> DB -[Browser] --HTTPS--> [Nginx: static frontend + proxy /api,/ws --> MONITOR_APP.LISTEN_PORT] + BR[Browser] -->|HTTPS| FE[Nginx or SERVE_STATIC] + FE --> APP ``` --- diff --git a/docs/en/monitor/self-service.md b/docs/en/monitor/self-service.md index a94ecca..6c6d7cb 100644 --- a/docs/en/monitor/self-service.md +++ b/docs/en/monitor/self-service.md @@ -41,6 +41,23 @@ Session lifetime is extended on activity (**SelfServiceController** uses a long ## End-to-end flow (why options reach the hotspot) +```mermaid +sequenceDiagram + participant UI as React /self-service + participant API as monitor FastAPI + participant DB as MySQL Clients + participant SRV as adn-server PROXY + participant HS as Hotspot + + UI->>API: POST /api/self-service/device/options + API->>DB: UPDATE options, modified = 1 + Note over SRV: send_opts loop ~every 10 s + SRV->>DB: read rows with modified = 1 + SRV->>SRV: RPTO on MASTER leg + SRV->>HS: OPTIONS via HBP path + SRV->>DB: clear modified +``` + 1. User saves options in the web UI → **monitor API** writes **`Clients.options`** and **`modified = 1`**. 2. The **hotspot proxy** runs **`send_opts`** on a loop (~every **10 s**). For rows with **`modified = 1`**, it reads options from the DB, sends **RPTO** (options) **to the peer server** at **`(MASTER, assigned_dest_port)`**, then clears **`modified`** in the DB. 3. The **ADN DMR Peer Server** receives **RPTO** on the MASTER leg and updates its **OPTIONS** / bridge state (same path as a normal hotspot registration refresh). diff --git a/docs/en/server/development/architecture.md b/docs/en/server/development/architecture.md index e61068e..087af46 100644 --- a/docs/en/server/development/architecture.md +++ b/docs/en/server/development/architecture.md @@ -8,6 +8,14 @@ **Dependency rule:** infrastructure → application → domain (inward only). +```mermaid +flowchart TD + INF["Infrastructure
Twisted UDP/TCP · YAML · voice
report_server · persistence · security"] + APP["Application
RoutingUseCases · VoiceUseCases
ports · SubscriptionStore · router"] + DOM["Domain
entities · value objects · errors · Result"] + INF --> APP --> DOM +``` + ## Entrypoint `main.py` wires configuration, **LoopingCall** timers, factories for **HBPProtocol**, report client, and injects use cases. @@ -16,6 +24,10 @@ Voice routing is driven by **`SubscriptionStore`** (domain subscriptions). `RoutingUseCases` orchestrates `dmrd_received` and delegates forward resolution to **`SubscriptionRouter`**. +Conceptual comparison with legacy **`BRIDGES`**: [BRIDGES vs Subscriptions](bridges-vs-subscriptions.md). + +Performance changes in 2.x (indexes, reporting, integrated proxy): [Performance (2.x)](performance.md). + - **`InMemoryAclRouter`** (`AclRouter` port) — ACL range checks only (`acl_check`). - **`routing_table_for_report()`** — export shim for monitor/report (legacy BRIDGE_SND shape); not used for runtime forwards. diff --git a/docs/en/server/development/bridges-vs-subscriptions.md b/docs/en/server/development/bridges-vs-subscriptions.md new file mode 100644 index 0000000..8444a26 --- /dev/null +++ b/docs/en/server/development/bridges-vs-subscriptions.md @@ -0,0 +1,208 @@ +# BRIDGES (legacy) vs Subscriptions (new server) + +**adn-dmr-server** and **adn-server 2.x** forward group voice the same way at the wire: a talkgroup “bridge table” decides which **systems** receive a copy of the stream, with **LC rewrite** per leg. What changed in 2.x is **how that table is represented in code** — not the operator-visible rules. + +## For operators + +Runtime bridge behaviour is unchanged: TGs, slots, OPTIONS, UA, TG 4000, OpenBridge. **Subscriptions** is the internal name for the 2.x routing engine — not a separate operating mode and not something you configure on its own. + +**No operational change** + +- Configuration: **`SYSTEMS`**, hotspot OPTIONS, **`SELF_SERVICE`** / MariaDB — same as **adn-dmr-server**. Neither stack loads a `BRIDGES` block from YAML. +- Rule parity: ACTIVE source row, `#…` tables, UA timers, `GEN_STAT_BRIDGES`, etc. + +**Concrete gains in 2.x** + +| Area | Effect | +|------|--------| +| **Routing stability** | Bridge state lives in a dedicated store; reports and timers no longer share the same mutable structure as voice forwarding. Fewer mismatches between what the server forwards and what the dashboard shows under load. | +| **Monitor** | **Report v2** (`routing_table`, `topology`) to **adn-monitor 2.x** replaces pickle/CSV; BTABLE tracks peer state more faithfully. | +| **Dynamic TGs** | With **`DATABASE`**, per-peer dynamics are **persisted** and restored on reconnect (≥ 2.0.0-rc.3). | +| **Maintenance** | Timer, OpenBridge, ACL, and self-service fixes do not go through one process-wide dict shared with every subsystem. | + +The word **subscription** only matters when reading code or this guide; on the dashboard and on the air you still work with **bridges** and **talkgroups**. + +--- + +## At a glance + +| | **Legacy (`adn-dmr-server`)** | **New (`adn-server` 2.x)** | +|---|------------------------------|----------------------------| +| **Runtime authority** | Global `BRIDGES` dict (`bridge_master.py`) | **`SubscriptionStore`** (domain `Subscription` objects) | +| **Structure** | `bridge_key → [ row, row, … ]` | One **subscription** per system leg on a channel | +| **Forward resolution** | Scan rows, call `to_target` | **`SubscriptionRouter.resolve()`** → `ForwardLeg` | +| **Monitor / report wire** | Pickle `BRIDGE_SND` = `BRIDGES` | JSON `routing_table` (v2) or exported `BRIDGES` shim (v1 compat) | +| **YAML `BRIDGES:` block** | Not loaded from config in either stack; rows are built at runtime | Same — rows come from OPTIONS, UA, STAT, OpenBridge, echo bootstrap | + +Observable behaviour (source-row guard, dynamic UA, static TG, reflector keys `#…`, OpenBridge TS1 match, timers) follows **legacy parity** with `bridge_master.py`. + +## Legacy: `BRIDGES` dict + +In **adn-dmr-server**, routing state is a **process-wide dictionary**: + +```text +BRIDGES["52090"] = [ + { "SYSTEM": "MASTER-A", "TS": 2, "TGID": b'...', "ACTIVE": True, "TO_TYPE": "ON", "TIMER": …, … }, + { "SYSTEM": "MASTER-B", "TS": 2, "TGID": b'...', "ACTIVE": True, "TO_TYPE": "ON", … }, + { "SYSTEM": "OBP-UK", "TS": 1, "TGID": b'...', "ACTIVE": False, … }, +] +BRIDGES["#310"] = [ … ] # reflector / marked TG tables +``` + +Each **row** is a leg. Important fields: + +- **`SYSTEM`** — configured system name (HBP master or OpenBridge leg). +- **`TS`** — timeslot 1 or 2 (OpenBridge sources use **TS 1** in the match path). +- **`TGID`** — destination ID bytes used for **LC rewrite** toward that leg. +- **`ACTIVE`** — leg participates in forwarding when true. +- **`TIMEOUT` / `TIMER` / `TO_TYPE` / `ON` / `OFF` / `RESET`** — UA timers, static/stat, in-band VTERM rules. + +**Voice path (`dmrd_received`):** + +1. Derive **bridge key** from destination TG (and reflector `#…` tables when applicable). +2. Create a dynamic table if missing (UA / STAT / static OPTIONS — same triggers as legacy). +3. Find an **ACTIVE source row** matching current **system + slot + TGID**. +4. For each other **ACTIVE** row in **that same table**, call **`to_target`** (contention, ACL, LC/TA rewrite, OpenBridge loop control). + +```mermaid +flowchart TB + IN[DMRD / DMRE ingress] --> DM[dmrd_received] + DM --> KEY{bridge key exists?} + KEY -->|no| CREATE[ensure dynamic / stat / static row] + CREATE --> BR + KEY -->|yes| BR[(BRIDGES dict)] + BR --> SRC{ACTIVE source row\nSYSTEM + TS + TGID?} + SRC -->|yes| SCAN[other ACTIVE legs\nsame table] + SCAN --> TT[to_target per leg] + TT --> OUT[HBP / OpenBridge egress] + SRC -->|no| DROP[no forward] +``` + +The monitor reads the **same dict** via pickle **`BRIDGE_SND`**. + +## New server: `Subscription` + `SubscriptionStore` + +In **adn-server 2.x**, the **domain model** replaces ad-hoc dict rows: + +- **`AudioChannel`** — logical TG + slot `(tgid, slot)`. +- **`Subscription`** — one system’s participation: **role**, **activation policy**, **state** (phase, timer), **target_tgid** (LC rewrite), optional **`relay_table_key`** (reflector `#…` tables). +- **`SubscriptionStore`** — sole **runtime routing authority** (no parallel `BRIDGES` mutation). + +**Voice path** (same semantics, different types): + +1. `RoutingUseCases.dmrd_received` updates the store (create relay table, static TG, UA timeout — legacy hooks). +2. **`SubscriptionRouter.relay_tables_with_active_source`** — tables where the ingress system has an **ACTIVE** subscription matching slot/TG. +3. **`SubscriptionRouter.resolve`** — returns **`ForwardLeg`** targets (system, slot, tgid) for all other **ACTIVE** subscriptions in those tables. +4. Forward mixins send packets (`to_target` parity). + +```mermaid +flowchart TB + IN[DMRD / DMRE ingress] --> RU[RoutingUseCases.dmrd_received] + RU --> SS[(SubscriptionStore)] + RU --> SR[SubscriptionRouter] + SS --> SR + SR --> LEGS[ForwardLeg list] + LEGS --> FWD[HBP / OBP forward mixins] + SS --> EXP[RoutingTableLegacyView\nexport shim only] + EXP --> MON[BRIDGE_SND pickle\nor routing_table JSON] +``` + +**Important:** `routing_table_for_report()` / **`BRIDGE_SND`** is a **one-way export** for dashboards (`RoutingTableLegacyView`). It is **not** used to decide forwards. That avoids the legacy pattern of mutating a global dict shared with reporting. + +## Row → subscription mapping + +| Legacy `BRIDGES` row | Domain `Subscription` | +|----------------------|-------------------------| +| Table key (`"52090"`, `"#310"`) | `relay_table_key` + channel TG | +| `SYSTEM` | `system` (`SystemId`) | +| `TS` + table TG context | `channel.slot` / `channel.tgid` | +| `TGID` (bytes) | `target_tgid` (LC rewrite) | +| `ACTIVE` | `state.phase` (`ACTIVE` / `IDLE`) | +| `TIMER` | `state.timer_expires_at` | +| `TIMEOUT` | `timeout_seconds` | +| `TO_TYPE` (`ON`, `OFF`, `STAT`, `NONE`) | `role` + `policy` (`ActivationPolicy`, `SubscriptionRole`) | +| `ON` / `OFF` / `RESET` | `triggers` (`InbandTriggers`) | + +Import/export helpers: `routing_table_import.py`, `routing_table_export.py` (mirror of legacy `bridges_export`). + +## End-to-end comparison (one voice frame) + +```mermaid +sequenceDiagram + participant Radio + participant Server + participant Peer as Other system + + Radio->>Server: Group voice TG 52090 TS2 + + rect rgb(40,40,50) + note right of Server: Legacy + Server->>Server: BRIDGES["52090"] source row ACTIVE? + Server->>Server: foreach ACTIVE leg in table + Server->>Peer: to_target (rewrite LC) + end + + rect rgb(30,50,40) + note right of Server: New server (same rules) + Server->>Server: SubscriptionRouter.resolve() + Server->>Peer: forward ForwardLeg(s) + end +``` + +## What did **not** change + +- Bridge **keys** (`52090`, `#reflector`, …) and **multi-leg tables**. +- **Source-row guard** — forward only from a table where **this** system is an ACTIVE source for that TG/slot context. +- **Dynamic UA**, **static OPTIONS**, **`GEN_STAT_BRIDGES`**, **TG 4000** clearing, **echo 9990** bootstrap. +- Timer passes (`rule_timer`, `bridgeDebug`, …) — still driven off the same logical table, implemented on the store in 2.x. + +## Concrete example + +**Bridge** (network concept): “TG 52090 connects these systems and forwards voice between them”. +**Subscription** (2.x code only): **one leg** of that table — e.g. “MASTER-A on TG 52090, slot 2, ACTIVE, with its LC and timer”. + +It is not bridge *or* subscription: in 2.x a bridge **is** a set of subscriptions on the same channel (TG + slot). + +### Scenario + +Someone keys **TG 52090** and **MASTER-A**, **MASTER-B**, and an **OpenBridge** leg should hear it. On the dashboard and on the air that is a **bridge** (the TG 52090 table). + +**Legacy (`adn-dmr-server`)** — everything in one global dict: + +```text +BRIDGES["52090"] = [ + { SYSTEM: "MASTER-A", TS: 2, ACTIVE: True, TGID: …, TIMER: … }, + { SYSTEM: "MASTER-B", TS: 2, ACTIVE: True, TGID: … }, + { SYSTEM: "OBP-UK", TS: 1, ACTIVE: False, TGID: … }, +] +``` + +When a voice frame arrives: + +1. Find the row where **this** system is the **ACTIVE** source (same TG/slot). +2. Walk the **other ACTIVE rows** in the same table. +3. For each, call **`to_target`** → forward with rewritten LC. + +The monitor reads **the same dict** (pickle **`BRIDGE_SND`**). + +**New server (`adn-server` 2.x)** — same table, different internal shape: + +```text +SubscriptionStore — TG 52090 / slot 2: + - subscription MASTER-A (ACTIVE, target_tgid, timer…) + - subscription MASTER-B (ACTIVE, …) + - subscription OBP-UK (IDLE, …) +``` + +On voice, **`SubscriptionRouter.resolve()`** applies the **same rules** (ACTIVE source, other ACTIVE legs) and returns **`ForwardLeg`** entries to forward. The monitor gets an **exported view** (`BRIDGE_SND` or JSON **`routing_table`**); that export does **not** drive forwarding. + +## Where to read code + +| Topic | Legacy | New | +|-------|--------|-----| +| Voice ingress | `adn-dmr-server/bridge_master.py` (`routerHBP.dmrd_received`) | `application/routing_use_cases.py` | +| Forward to leg | `to_target` | `application/routing/hbp_forward.py`, `obp_forward.py` | +| Table state | global `BRIDGES` | `application/subscription/` (`store`, `router`, ops) | +| Monitor export | `send_routing_table` / pickle | `routing_table_legacy_view.py`, report v2 `routing_table` | + +See also: [Bridges and talkgroups](../user-guide/bridges-and-talkgroups.md), [Architecture](architecture.md), [Performance (2.x)](performance.md), [Report protocol v2](../protocols/report-v2.md#routing_table). diff --git a/docs/en/server/development/performance.md b/docs/en/server/development/performance.md new file mode 100644 index 0000000..5dfdc50 --- /dev/null +++ b/docs/en/server/development/performance.md @@ -0,0 +1,99 @@ +# Performance (2.x) + +**adn-server 2.x** and **adn-monitor 2.x** include several changes that reduce CPU work and memory footprint compared with **adn-dmr-server** and the old monitor/proxy stack. This page lists **what** improves and **what causes it**. + +## At a glance + +| Area | Typical effect | Main cause | +|------|----------------|------------| +| **Voice downlink (inject proxy)** | Lower CPU under busy group traffic | **`PeerDownlinkIndex`** — fan-out to peers that match `(slot, TG)` instead of scanning every connected hotspot per packet | +| **Bridge source lookup** | Faster “am I the ACTIVE source?” | **`SubscriptionStore`** indexes (`relay_tables_with_active_source`) — O(1) by `(system, slot, tgid)` vs scanning table rows | +| **Background CPU** | Fewer wakeups | **Event-driven OPTIONS / static TG** — removed legacy **26 s** `options_config_loop` ([Behaviour and timers](behaviour-and-timers.md)) | +| **Mass peer login** | Less redundant CONFIG traffic | **`ConfigPushThrottle`** — adaptive debounce on CONFIG push to the monitor | +| **Reporting vs voice** | Voice path less blocked by reports | **`BoundedReportQueue`** — coalesced snapshots, bounded drain per tick | +| **Server → monitor wire** | Less serialize/send work | **Report v2** JSON (`routing_table`, `topology`, `voice_event`) instead of periodic full pickle of `CONFIG`/`BRIDGES` ([Report protocol v2](../protocols/report-v2.md)) | +| **Process count (RAM)** | One Python process instead of two | **Integrated `PROXY`** in `adn-server.py` — no separate **adn-proxy** process ([Hotspot proxy](../user-guide/hotspot-proxy.md)) | +| **Monitor RAM / WS load** | Smaller in-memory dashboard state | **Slim `dashboard_state` wire**, `clean_sys_dict`, lighter WebSocket fingerprints ([Monitor architecture](../../monitor/architecture.md)) | + +## Server: inject-only downlink index + +The largest **CPU** win on many ADN networks is on the **MASTER inject-only** path (`PROXY` with inject-only mode). + +**Legacy:** `send_peers` walks **every registered peer** for each downlink packet → cost grows as **O(peers × packets/s)**. + +**2.x:** `PeerDownlinkIndex` precomputes candidates from each peer’s **OPTIONS** (static TGs) and **UA session** state. For each group voice frame, only peers that **might** want that `(slot, TG)` are considered; each candidate still passes `peer_should_receive_group_voice`. + +```text +Legacy: every DMRD → try all N peers +2.x: every DMRD → index lookup → try k peers (k ≪ N on busy proxies) +``` + +OPTIONS parsing is **cached per peer** (`_CACHED_OPTIONS_STATIC`): if the OPTIONS blob is unchanged, already-parsed static TGs are reused instead of re-parsing on every packet. + +| Code | Role | +|------|------| +| `application/routing/peer_downlink_index.py` | Index build and `(slot, tgid) → candidates` | +| `infrastructure/twisted_adapters/udp_hbp.py` | `_iter_downlink_peers`, `send_peers` | +| `tests/infrastructure/test_peer_downlink_fanout.py` | Inject-only fan-out tests | + +**When it matters:** proxy with **tens to hundreds** of hotspots and steady group voice. On a small conference with few peers, the difference is minor. + +## Server: routing indexes + +On every group voice frame the server must find relay tables where **this system is the ACTIVE source**. + +**Legacy:** scan rows inside `BRIDGES[table_key]` (and related tables). + +**2.x:** `InMemorySubscriptionStore.relay_tables_with_active_source()` uses a maintained **`_source_tables`** index — lookup by `(system, slot, dst_tgid)` without walking all legs. + +This lives in the subscription store implementation; it is an **algorithmic index**, not a separate feature you configure. + +| Code | Role | +|------|------| +| `infrastructure/subscription_store.py` | `_source_tables`, `_by_table`, `_active_target_counts` | +| `application/subscription/router.py` | `SubscriptionRouter.resolve()` | + +## Server: less periodic and login-storm work + +| Change | What it avoids | +|--------|----------------| +| **No 26 s OPTIONS loop** | Timer firing every 26 s across all systems to refresh static bridges when RPTO/startup/reload already handle it | +| **`ConfigPushThrottle`** | Flooding the monitor with CONFIG snapshots when many peers connect within a few seconds (debounce widens from ~0.3 s to ~2 s during bursts) | +| **`BoundedReportQueue`** | Doing pickle/JSON encode and TCP send synchronously on the voice hot path; coalesces duplicate config/bridge snapshots | + +## Server: reporting and deployment + +- **Report v2** — structured JSON replaces opaque pickle snapshots for bridge/config state on the **2.x monitor** wire. See [Monitoring and reports](../user-guide/monitoring.md) and [Report protocol v2](../protocols/report-v2.md). +- **Integrated proxy** — `PROXY` runs **in-process**; dropping the standalone **adn-proxy** saves baseline **RAM** (one interpreter, shared config) and simplifies ops. + +## Monitor (adn-monitor 2.x) + +Pair **adn-server 2.x** with **adn-monitor 2.x** to get the reporting-side gains: + +| Change | Effect | +|--------|--------| +| **Slim wire / `dashboard_state`** | Monitor ingests compact JSON state instead of holding full duplicated pickle trees from v1 | +| **`clean_sys_dict`** | Periodic eviction of stale in-memory entries (caps runaway growth on long-lived panels) | +| **Last-heard row cache, lighter WS fingerprints** | Less work per dashboard refresh | +| **Unified FastAPI stack** | Removed separate PHP API and standalone monitor **proxy** process | + +Details: [Monitor architecture](../../monitor/architecture.md). + +## When you will notice a difference + +| Deployment | CPU | RAM | +|------------|-----|-----| +| Few masters, no inject proxy, light traffic | Small | Small | +| **Inject-only proxy, many hotspots, busy TG** | **Clear** (downlink index) | Moderate (single server process vs server+proxy) | +| Long-lived monitor + report v2 | Moderate (less serialize on wire) | **Clearer** on monitor (slim state, `clean_sys_dict`) | + +Crypto, AMBE, and OpenBridge MAC work still dominate on OpenBridge-heavy paths — routing-table optimizations do not remove that cost. + +## Related reading + +- [Architecture](architecture.md) — layers and entrypoint +- [BRIDGES vs Subscriptions](bridges-vs-subscriptions.md) — routing model (not a performance feature) +- [Behaviour and timers](behaviour-and-timers.md) — event-driven OPTIONS vs legacy 26 s loop +- [Hotspot proxy](../user-guide/hotspot-proxy.md) — integrated `PROXY` / inject-only +- [Report protocol v2](../protocols/report-v2.md) — JSON wire to monitor +- Release notes: `CHANGELOG.md` at the repository root (`Performance` under **2.0.0-rc.1**). diff --git a/docs/en/server/protocols/report-v2.md b/docs/en/server/protocols/report-v2.md index d78e3be..0200fb5 100644 --- a/docs/en/server/protocols/report-v2.md +++ b/docs/en/server/protocols/report-v2.md @@ -28,6 +28,22 @@ Replace monitor snapshots that today use **pickle** (`CONFIG_SND`, `BRIDGE_SND`) Proposed opcodes `0x10`–`0x13` are reserved in the schema phase; exact values may change before P1-002 ships. +```mermaid +flowchart TB + subgraph v1 [Report v1 — adn-server 1.0.x + monitor 1.0.x] + H1[HELLO protocol 1] --> C1[CONFIG_SND pickle] + C1 --> B1[BRIDGE_SND pickle] + B1 --> V1[BRDG_EVENT CSV] + end + + subgraph v2 [Report v2 — adn-server 2.x + monitor 2.x] + H2[HELLO report_protocol 2] --> T2[TOPOLOGY_SND JSON] + T2 --> R2[ROUTING_TABLE_SND JSON] + R2 --> V2[VOICE_EVENT_SND JSON] + R2 -.-> D2[DELTA_SND optional] + end +``` + ## Handshake (`hello`) On connect the server sends **`HELLO` (`0xFF`)** first (same as today). v2 clients inspect `report_protocol`: diff --git a/docs/en/server/user-guide/bridges-and-talkgroups.md b/docs/en/server/user-guide/bridges-and-talkgroups.md index af8230b..5a05de1 100644 --- a/docs/en/server/user-guide/bridges-and-talkgroups.md +++ b/docs/en/server/user-guide/bridges-and-talkgroups.md @@ -12,6 +12,8 @@ The bridge table maps **talkgroup keys** (strings, e.g. `"26811"`, `"#reflector" The router scans `BRIDGES` for an **ACTIVE** row matching the **current source system**, **slot**, and **destination TG** before forwarding (`dmrd_received` → `to_target`). +In **adn-server 2.x** the same rules live in **`SubscriptionStore`** / **`SubscriptionRouter`**; `BRIDGES` is only an export shape for the monitor. See [BRIDGES vs Subscriptions](../development/bridges-vs-subscriptions.md). + ## Dynamic vs static - **User-activated** bridges are created when a user keys a TG without a pre-built row (subject to `DEFAULT_UA_TIMER` and options). diff --git a/docs/en/server/user-guide/hotspot-proxy.md b/docs/en/server/user-guide/hotspot-proxy.md index 12231e8..3650697 100644 --- a/docs/en/server/user-guide/hotspot-proxy.md +++ b/docs/en/server/user-guide/hotspot-proxy.md @@ -14,6 +14,13 @@ Configuration lives in **`adn-server.yaml`** under **`PROXY`** and optional **`S The integrated proxy uses **fan-in**: hotspots only need **`PROXY.LISTEN_PORT`** (e.g. **62031**). The target **MASTER** is **inject-only** — it does **not** bind its own UDP port for that system (no per-hotspot port range on the server host). +```mermaid +flowchart LR + HS1[Hotspot A] -->|UDP HBP| LP[PROXY LISTEN_PORT] + HS2[Hotspot B] -->|UDP HBP| LP + LP -->|inject| MASTER[TARGET_SYSTEM MASTER] +``` + --- ## Optional dependency (self-service) diff --git a/docs/en/server/user-guide/introduction.md b/docs/en/server/user-guide/introduction.md index efc38bf..7a49696 100644 --- a/docs/en/server/user-guide/introduction.md +++ b/docs/en/server/user-guide/introduction.md @@ -37,4 +37,5 @@ Routing, timers, OpenBridge loop control, and protocol handling are implemented - [Special numbers](special-numbers.md) — TG 4000, information services, echo. - [Hotspot proxy](hotspot-proxy.md) — integrated **`PROXY`** / **`SELF_SERVICE`** in `adn-server.yaml`. - [ADN Monitor](../../monitor/index.md) — dashboard, `adn-monitor.yaml`, self-service UI (separate repo, deployed with the server). +- [Performance (2.x)](../development/performance.md) — CPU/RAM improvements in this release and what causes them. - [Credits & license](attribution.md) — ADN → FreeDMR → hblink3, license. diff --git a/docs/en/server/user-guide/monitoring.md b/docs/en/server/user-guide/monitoring.md index d45a460..696d6bd 100644 --- a/docs/en/server/user-guide/monitoring.md +++ b/docs/en/server/user-guide/monitoring.md @@ -24,6 +24,14 @@ The **monitor** decodes these messages, updates its **CTABLE** / **BTABLE**, and **Full stack:** [ADN Monitor overview](../../monitor/index.md) (FastAPI monitor, WebSocket, self-service). +```mermaid +flowchart LR + SRV[adn-server\nREPORTS.REPORT_PORT] -->|TCP netstring| ING[adn-monitor ingest] + ING --> STATE[CTABLE / BTABLE] + STATE --> WS[WebSocket /ws] + WS --> UI[React dashboard] +``` + ### Report channel log lines (`adn-monitor` logger) Python uses the logger name **`adn-monitor`** (see **`LOGGER.LOG_FILE`** in `adn-monitor.yaml`). Typical **INFO** lines for the TCP report client: diff --git a/docs/en/server/user-guide/special-numbers.md b/docs/en/server/user-guide/special-numbers.md index e6b5725..f195bb9 100644 --- a/docs/en/server/user-guide/special-numbers.md +++ b/docs/en/server/user-guide/special-numbers.md @@ -98,6 +98,8 @@ The **audio** is sent with **source ID 5000** and **destination TG 9** in the ge **Purpose:** Bridge rows for **echo** often use **9990** with the **ECHO** system (see `BRIDGES` and options in your YAML). +**`SINGLE=1`:** Keying **9990** does **not** create an exclusive listen session (same as **4000**). Downlink echo always returns to the calling hotspot even when another TG holds the SINGLE lock. See [Hotspot proxy](hotspot-proxy.md#behaviour-with-multiple-hotspots). + **Note:** A **standalone echo** is also available as a separate process — [Echo](echo.md). ## Private call to ID 4000 diff --git a/docs/es/monitor/architecture.md b/docs/es/monitor/architecture.md index f72758c..dfef025 100644 --- a/docs/es/monitor/architecture.md +++ b/docs/es/monitor/architecture.md @@ -10,6 +10,14 @@ Bajo `monitor/src/adn_monitor/`: El **composition root** está en `infrastructure/fastapi/composition.py` (`build_monitor_api`). +```mermaid +flowchart TD + INF["Infraestructura
YAML · FastAPI /ws · ingest TCP/MQTT
MySQL · decodificadores pickle/JSON"] + APP["Aplicación
MonitorState · informes/dashboard
auth · self-service · alias"] + DOM["Dominio
objetos de valor · errores · opcodes
UserSession · Result"] + INF --> APP --> DOM +``` + ## Proceso unificado (`monitor.py`) Un solo proceso uvicorn/FastAPI: @@ -39,15 +47,19 @@ El proceso **`adn-proxy`** independiente se eliminó del repositorio **adn-monit ## Topología típica -```text -[Hotspots] --UDP--> [adn-server PROXY] --UDP--> [peer MASTER] - | - v - MySQL (Clients) +```mermaid +flowchart TD + HS[Hotspots] -->|UDP HBP| PROXY[adn-server PROXY] + PROXY --> MASTER[MASTER inyectado] + PROXY --> DB[(MySQL Clients)] -[Peer :REPORT_PORT] <--- TCP o MQTT --- [monitor.py ingest] + MASTER --> REPORT[REPORTS listener] + REPORT <-->|TCP o MQTT| ING[monitor.py ingest] + ING --> APP[FastAPI /api /ws] + APP --> DB -[Navegador] --HTTPS--> [Nginx: frontend/dist + proxy /api,/ws --> :8080] + BR[Navegador] -->|HTTPS| FE[Nginx o SERVE_STATIC] + FE --> APP ``` --- diff --git a/docs/es/monitor/self-service.md b/docs/es/monitor/self-service.md index 96a8227..37b02fd 100644 --- a/docs/es/monitor/self-service.md +++ b/docs/es/monitor/self-service.md @@ -41,6 +41,23 @@ La sesión se prolonga con actividad (**SelfServiceController** usa un timeout l ## Flujo extremo a extremo (cómo llegan las opciones al hotspot) +```mermaid +sequenceDiagram + participant UI as React /self-service + participant API as monitor FastAPI + participant DB as MySQL Clients + participant SRV as adn-server PROXY + participant HS as Hotspot + + UI->>API: POST /api/self-service/device/options + API->>DB: UPDATE options, modified = 1 + Note over SRV: bucle send_opts ~cada 10 s + SRV->>DB: lee filas con modified = 1 + SRV->>SRV: RPTO en pata MASTER + SRV->>HS: OPTIONS vía HBP + SRV->>DB: limpia modified +``` + 1. El usuario guarda opciones en la web → la **API del monitor** escribe **`Clients.options`** y **`modified = 1`**. 2. El **proxy hotspot** ejecuta **`send_opts`** en bucle (~cada **10 s**). Para filas con **`modified = 1`**, lee opciones de la BD, envía **RPTO** **al peer server** en **`(MASTER, puerto_destino_asignado)`**, luego limpia **`modified`** en la BD. 3. El **ADN DMR Peer Server** recibe **RPTO** en la pata MASTER y actualiza su estado **OPTIONS** / bridge (mismo camino que un refresco normal de registro del hotspot). diff --git a/docs/es/server/development/architecture.md b/docs/es/server/development/architecture.md index f628851..1db3780 100644 --- a/docs/es/server/development/architecture.md +++ b/docs/es/server/development/architecture.md @@ -8,6 +8,14 @@ **Regla de dependencias:** infraestructura → aplicación → dominio (solo hacia dentro). +```mermaid +flowchart TD + INF["Infraestructura
Twisted UDP/TCP · YAML · voz
report_server · persistencia · seguridad"] + APP["Aplicación
RoutingUseCases · VoiceUseCases
ports · SubscriptionStore · router"] + DOM["Dominio
entidades · value objects · errors · Result"] + INF --> APP --> DOM +``` + ## Punto de entrada `main.py` cablea configuración, temporizadores **LoopingCall**, fábricas para **HBPProtocol**, servidor de informes e inyecta casos de uso. @@ -16,6 +24,10 @@ El enrutado de voz lo gobierna **`SubscriptionStore`** (suscripciones de dominio). `RoutingUseCases` orquesta `dmrd_received` y delega la resolución de reenvío a **`SubscriptionRouter`**. +Comparación conceptual con **`BRIDGES`** legacy: [BRIDGES vs Subscriptions](bridges-vs-subscriptions.md). + +Cambios de rendimiento en 2.x (índices, informes, proxy integrado): [Rendimiento (2.x)](performance.md). + - **`InMemoryAclRouter`** (port `AclRouter`) — solo comprobaciones ACL (`acl_check`). - **`routing_table_for_report()`** — shim de exportación para monitor/informes (forma legacy BRIDGE_SND); no se usa para reenvíos en runtime. diff --git a/docs/es/server/development/bridges-vs-subscriptions.md b/docs/es/server/development/bridges-vs-subscriptions.md new file mode 100644 index 0000000..4022fc1 --- /dev/null +++ b/docs/es/server/development/bridges-vs-subscriptions.md @@ -0,0 +1,208 @@ +# BRIDGES (legacy) vs Subscriptions (servidor nuevo) + +**adn-dmr-server** y **adn-server 2.x** reenvían voz de grupo igual en el wire: una “tabla de bridge” por talkgroup decide qué **systems** reciben copia del stream, con **reescritura LC** por pata. Lo que cambia en 2.x es **cómo se representa esa tabla en código** — no las reglas visibles para el operador. + +## Para el operador + +En runtime el comportamiento de bridges es el mismo: TG, slots, OPTIONS, UA, TG 4000, OpenBridge. **Subscriptions** es el nombre interno del motor de enrutado en 2.x; no es un modo de operación distinto ni algo que configures aparte. + +**Sin cambios operativos** + +- Configuración: **`SYSTEMS`**, OPTIONS del hotspot, **`SELF_SERVICE`** / MariaDB — igual que con **adn-dmr-server**. No hay bloque `BRIDGES` en YAML en ninguno de los dos. +- Paridad de reglas: fila origen ACTIVE, tablas `#…`, timers UA, `GEN_STAT_BRIDGES`, etc. + +**Mejoras reales en 2.x** + +| Área | Efecto | +|------|--------| +| **Estabilidad del enrutado** | El estado de bridges vive en un store dedicado; informes y timers ya no comparten la misma estructura mutable que el forward de voz. Menos desvíos entre lo que reenvía el servidor y lo que muestra el panel bajo carga. | +| **Monitor** | **Informe v2** (`routing_table`, `topology`) hacia **adn-monitor 2.x** sustituye pickle/CSV; el BTABLE refleja mejor el estado del peer. | +| **TG dinámicos** | Con **`DATABASE`**, los dinámicos por peer se **persisten** y se restauran al reconectar (≥ 2.0.0-rc.3). | +| **Evolución** | Parches de timers, OpenBridge, ACL o self-service no pasan por un dict global compartido con todo el proceso. | + +El término **subscription** solo importa si lees código o esta guía; en el panel y en el aire sigues hablando de **bridges** y **talkgroups**. + +--- + +## Resumen + +| | **Legacy (`adn-dmr-server`)** | **Nuevo (`adn-server` 2.x)** | +|---|------------------------------|------------------------------| +| **Autoridad en runtime** | Dict global `BRIDGES` (`bridge_master.py`) | **`SubscriptionStore`** (objetos `Subscription` de dominio) | +| **Estructura** | `bridge_key → [ fila, fila, … ]` | Una **subscription** por pata de system en un canal | +| **Resolución de reenvío** | Recorrer filas, llamar `to_target` | **`SubscriptionRouter.resolve()`** → `ForwardLeg` | +| **Wire monitor / informes** | Pickle `BRIDGE_SND` = `BRIDGES` | JSON `routing_table` (v2) o export `BRIDGES` (compat v1) | +| **Bloque YAML `BRIDGES:`** | No se carga desde config en ninguno; filas en runtime | Igual — filas desde OPTIONS, UA, STAT, OpenBridge, bootstrap echo | + +El comportamiento observable (guardia de fila origen, UA dinámico, TG estática, claves reflector `#…`, match OpenBridge TS1, timers) sigue **paridad legacy** con `bridge_master.py`. + +## Legacy: dict `BRIDGES` + +En **adn-dmr-server**, el estado de enrutado es un **diccionario global**: + +```text +BRIDGES["52090"] = [ + { "SYSTEM": "MASTER-A", "TS": 2, "TGID": b'...', "ACTIVE": True, "TO_TYPE": "ON", "TIMER": …, … }, + { "SYSTEM": "MASTER-B", "TS": 2, "TGID": b'...', "ACTIVE": True, "TO_TYPE": "ON", … }, + { "SYSTEM": "OBP-UK", "TS": 1, "TGID": b'...', "ACTIVE": False, … }, +] +BRIDGES["#310"] = [ … ] # tablas reflector / marcado +``` + +Cada **fila** es una pata. Campos importantes: + +- **`SYSTEM`** — nombre del system configurado (master HBP o pata OpenBridge). +- **`TS`** — slot 1 o 2 (fuentes OpenBridge usan **TS 1** en el match). +- **`TGID`** — bytes de destino para **reescritura LC** hacia esa pata. +- **`ACTIVE`** — la pata participa en el reenvío si es true. +- **`TIMEOUT` / `TIMER` / `TO_TYPE` / `ON` / `OFF` / `RESET`** — timers UA, static/stat, reglas VTERM in-band. + +**Camino de voz (`dmrd_received`):** + +1. Obtener **clave de bridge** desde la TG destino (y tablas `#…` cuando aplica). +2. Crear tabla dinámica si no existe (UA / STAT / OPTIONS estáticas — mismos disparadores que legacy). +3. Buscar **fila origen ACTIVE** que coincida con **system + slot + TGID** actuales. +4. Por cada otra fila **ACTIVE** en **esa misma tabla**, llamar **`to_target`** (contención, ACL, LC/TA, control de bucle OpenBridge). + +```mermaid +flowchart TB + IN[Ingreso DMRD / DMRE] --> DM[dmrd_received] + DM --> KEY{¿existe bridge key?} + KEY -->|no| CREATE[ensure dynamic / stat / static] + CREATE --> BR + KEY -->|si| BR[(dict BRIDGES)] + BR --> SRC{¿fila origen ACTIVE\nSYSTEM + TS + TGID?} + SRC -->|si| SCAN[otras patas ACTIVE\nmisma tabla] + SCAN --> TT[to_target por pata] + TT --> OUT[Salida HBP / OpenBridge] + SRC -->|no| DROP[sin reenvío] +``` + +El monitor lee el **mismo dict** vía pickle **`BRIDGE_SND`**. + +## Servidor nuevo: `Subscription` + `SubscriptionStore` + +En **adn-server 2.x**, el **modelo de dominio** sustituye filas ad hoc: + +- **`AudioChannel`** — TG lógica + slot `(tgid, slot)`. +- **`Subscription`** — participación de un system: **role**, **política de activación**, **state** (fase, timer), **target_tgid** (LC), **`relay_table_key`** opcional (tablas `#…`). +- **`SubscriptionStore`** — única **autoridad de enrutado** en runtime (sin mutar `BRIDGES` en paralelo). + +**Camino de voz** (misma semántica, otros tipos): + +1. `RoutingUseCases.dmrd_received` actualiza el store (crear relay, TG estática, timeout UA — hooks legacy). +2. **`SubscriptionRouter.relay_tables_with_active_source`** — tablas donde el system de ingreso tiene subscription **ACTIVE** para slot/TG. +3. **`SubscriptionRouter.resolve`** — devuelve **`ForwardLeg`** (system, slot, tgid) del resto de subscriptions **ACTIVE** en esas tablas. +4. Mixins de forward envían paquetes (paridad `to_target`). + +```mermaid +flowchart TB + IN[Ingreso DMRD / DMRE] --> RU[RoutingUseCases.dmrd_received] + RU --> SS[(SubscriptionStore)] + RU --> SR[SubscriptionRouter] + SS --> SR + SR --> LEGS[Lista ForwardLeg] + LEGS --> FWD[Mixins HBP / OBP] + SS --> EXP[RoutingTableLegacyView\nsolo export] + EXP --> MON[Pickle BRIDGE_SND\no JSON routing_table] +``` + +**Importante:** `routing_table_for_report()` / **`BRIDGE_SND`** es un **export unidireccional** para paneles (`RoutingTableLegacyView`). **No** decide reenvíos. Evita el patrón legacy de mutar un dict global compartido con informes. + +## Mapeo fila → subscription + +| Fila legacy `BRIDGES` | `Subscription` de dominio | +|-----------------------|---------------------------| +| Clave tabla (`"52090"`, `"#310"`) | `relay_table_key` + TG del canal | +| `SYSTEM` | `system` (`SystemId`) | +| `TS` + contexto TG | `channel.slot` / `channel.tgid` | +| `TGID` (bytes) | `target_tgid` (reescritura LC) | +| `ACTIVE` | `state.phase` (`ACTIVE` / `IDLE`) | +| `TIMER` | `state.timer_expires_at` | +| `TIMEOUT` | `timeout_seconds` | +| `TO_TYPE` (`ON`, `OFF`, `STAT`, `NONE`) | `role` + `policy` | +| `ON` / `OFF` / `RESET` | `triggers` (`InbandTriggers`) | + +Helpers import/export: `routing_table_import.py`, `routing_table_export.py` (espejo de `bridges_export` legacy). + +## Comparación extremo a extremo (un frame de voz) + +```mermaid +sequenceDiagram + participant Radio + participant Server + participant Peer as Otro system + + Radio->>Server: Voz grupo TG 52090 TS2 + + rect rgb(40,40,50) + note right of Server: Legacy + Server->>Server: ¿Fila origen ACTIVE en BRIDGES["52090"]? + Server->>Server: foreach pata ACTIVE en tabla + Server->>Peer: to_target (reescribe LC) + end + + rect rgb(30,50,40) + note right of Server: Servidor nuevo (mismas reglas) + Server->>Server: SubscriptionRouter.resolve() + Server->>Peer: forward ForwardLeg(s) + end +``` + +## Lo que **no** cambió + +- **Claves** de bridge (`52090`, `#reflector`, …) y tablas **multi-pata**. +- **Guardia de fila origen** — reenviar solo desde una tabla donde **este** system es origen ACTIVE para ese contexto TG/slot. +- **UA dinámico**, **OPTIONS estáticas**, **`GEN_STAT_BRIDGES`**, **TG 4000**, bootstrap **echo 9990**. +- Pasadas de timer (`rule_timer`, `bridgeDebug`, …) — misma tabla lógica, implementada sobre el store en 2.x. + +## Ejemplo concreto + +**Bridge** (concepto de red): “la TG 52090 une estos systems y reenvía voz entre ellos”. +**Subscription** (solo en código 2.x): **una pata** de esa tabla — p. ej. “MASTER-A en TG 52090, slot 2, ACTIVE, con su LC y timer”. + +No es bridge *o* subscription: en 2.x un bridge **es** un conjunto de subscriptions sobre el mismo canal (TG + slot). + +### Escenario + +Alguien habla en **TG 52090** y deben oírlo **MASTER-A**, **MASTER-B** y una pata **OpenBridge**. En el panel y en el aire eso es un **bridge** (tabla de la TG 52090). + +**Legacy (`adn-dmr-server`)** — todo en un dict global: + +```text +BRIDGES["52090"] = [ + { SYSTEM: "MASTER-A", TS: 2, ACTIVE: True, TGID: …, TIMER: … }, + { SYSTEM: "MASTER-B", TS: 2, ACTIVE: True, TGID: … }, + { SYSTEM: "OBP-UK", TS: 1, ACTIVE: False, TGID: … }, +] +``` + +Cuando llega un frame de voz: + +1. Buscar la fila donde **este** system es origen **ACTIVE** (mismo TG/slot). +2. Recorrer las **demás filas ACTIVE** de la misma tabla. +3. Por cada una, **`to_target`** → reenvío con LC reescrito. + +El monitor lee **el mismo dict** (pickle **`BRIDGE_SND`**). + +**Servidor nuevo (`adn-server` 2.x)** — misma tabla, otro formato interno: + +```text +SubscriptionStore — TG 52090 / slot 2: + - subscription MASTER-A (ACTIVE, target_tgid, timer…) + - subscription MASTER-B (ACTIVE, …) + - subscription OBP-UK (IDLE, …) +``` + +Cuando llega voz, **`SubscriptionRouter.resolve()`** aplica las **mismas reglas** (origen ACTIVE, resto de patas ACTIVE) y devuelve **`ForwardLeg`** para reenviar. El monitor recibe una **vista exportada** (`BRIDGE_SND` o JSON **`routing_table`**); esa exportación **no** decide el reenvío. + +## Dónde leer código + +| Tema | Legacy | Nuevo | +|------|--------|-------| +| Ingreso voz | `adn-dmr-server/bridge_master.py` (`routerHBP.dmrd_received`) | `application/routing_use_cases.py` | +| Reenvío a pata | `to_target` | `application/routing/hbp_forward.py`, `obp_forward.py` | +| Estado tabla | `BRIDGES` global | `application/subscription/` (`store`, `router`, ops) | +| Export monitor | `send_routing_table` / pickle | `routing_table_legacy_view.py`, informe v2 `routing_table` | + +Ver también: [Bridges y talkgroups](../user-guide/bridges-and-talkgroups.md), [Arquitectura](architecture.md), [Rendimiento (2.x)](performance.md), [Protocolo de informes v2](../protocols/report-v2.md#routing_table). diff --git a/docs/es/server/development/performance.md b/docs/es/server/development/performance.md new file mode 100644 index 0000000..41a2179 --- /dev/null +++ b/docs/es/server/development/performance.md @@ -0,0 +1,99 @@ +# Rendimiento (2.x) + +**adn-server 2.x** y **adn-monitor 2.x** incluyen varios cambios que reducen trabajo de CPU y huella de memoria frente a **adn-dmr-server** y al stack antiguo de monitor/proxy. Esta página resume **qué** mejora y **qué lo provoca**. + +## Resumen + +| Área | Efecto típico | Causa principal | +|------|---------------|-----------------| +| **Downlink de voz (proxy inject)** | Menos CPU con tráfico de grupo intenso | **`PeerDownlinkIndex`** — fan-out solo a peers que encajan `(slot, TG)` en lugar de escanear todos los hotspots por paquete | +| **Origen ACTIVE en bridge** | Lookup más rápido | **Índices del `SubscriptionStore`** (`relay_tables_with_active_source`) — O(1) por `(system, slot, tgid)` frente a recorrer filas | +| **CPU de fondo** | Menos despertares | **OPTIONS / TG estática por eventos** — eliminado el bucle legacy cada **26 s** `options_config_loop` ([Comportamiento y temporizadores](behaviour-and-timers.md)) | +| **Ráfaga de logins** | Menos CONFIG redundante | **`ConfigPushThrottle`** — debounce adaptativo al empujar CONFIG al monitor | +| **Informes vs voz** | La voz se bloquea menos por informes | **`BoundedReportQueue`** — snapshots coalescidos, drenado acotado por tick | +| **Cable servidor → monitor** | Menos serializar/enviar | **Informe v2** JSON (`routing_table`, `topology`, `voice_event`) en lugar de pickle periódico de `CONFIG`/`BRIDGES` ([Protocolo de informes v2](../protocols/report-v2.md)) | +| **Procesos (RAM)** | Un proceso Python en lugar de dos | **`PROXY` integrado** en `adn-server.py` — sin proceso **adn-proxy** aparte ([Proxy hotspot](../user-guide/hotspot-proxy.md)) | +| **RAM / WS del monitor** | Estado de panel más compacto | **Wire slim `dashboard_state`**, `clean_sys_dict`, fingerprints WS más ligeros ([Arquitectura del monitor](../../monitor/architecture.md)) | + +## Servidor: índice de downlink inject-only + +La mayor ganancia de **CPU** en muchas redes ADN está en el camino **MASTER inject-only** (`PROXY` en modo inject-only). + +**Legacy:** `send_peers` recorre **todos los peers registrados** por cada paquete de downlink → coste **O(peers × paquetes/s)**. + +**2.x:** `PeerDownlinkIndex` precalcula candidatos desde **OPTIONS** (TG estáticas) y estado **UA** de cada peer. Por cada frame de voz de grupo solo se consideran peers que **podrían** querer ese `(slot, TG)`; cada candidato sigue pasando `peer_should_receive_group_voice`. + +```text +Legacy: cada DMRD → probar los N peers +2.x: cada DMRD → lookup en índice → probar k peers (k ≪ N en proxies cargados) +``` + +El parse de OPTIONS se **guarda en caché por peer** (`_CACHED_OPTIONS_STATIC`): si el blob OPTIONS no cambió, se reutilizan las TG estáticas ya parseadas en lugar de volver a interpretarlo en cada paquete. + +| Código | Rol | +|--------|-----| +| `application/routing/peer_downlink_index.py` | Construcción del índice y `(slot, tgid) → candidatos` | +| `infrastructure/twisted_adapters/udp_hbp.py` | `_iter_downlink_peers`, `send_peers` | +| `tests/infrastructure/test_peer_downlink_fanout.py` | Tests de fan-out inject-only | + +**Cuándo se nota:** proxy con **decenas o cientos** de hotspots y voz de grupo continua. En una conferencia pequeña con pocos peers, la diferencia es pequeña. + +## Servidor: índices de enrutado + +En cada frame de voz de grupo el servidor debe encontrar tablas donde **este system es origen ACTIVE**. + +**Legacy:** recorrer filas dentro de `BRIDGES[clave]`. + +**2.x:** `InMemorySubscriptionStore.relay_tables_with_active_source()` usa el índice **`_source_tables`** — lookup por `(system, slot, dst_tgid)` sin recorrer todas las patas. + +Está en la implementación del store; es un **índice algorítmico**, no una opción de configuración aparte. + +| Código | Rol | +|--------|-----| +| `infrastructure/subscription_store.py` | `_source_tables`, `_by_table`, `_active_target_counts` | +| `application/subscription/router.py` | `SubscriptionRouter.resolve()` | + +## Servidor: menos trabajo periódico y en tormenta de logins + +| Cambio | Qué evita | +|--------|-----------| +| **Sin bucle OPTIONS 26 s** | Timer cada 26 s en todos los systems cuando RPTO/arranque/reload ya refrescan bridges estáticos | +| **`ConfigPushThrottle`** | Inundar al monitor con snapshots CONFIG cuando muchos peers conectan en pocos segundos (debounce ~0,3 s → ~2 s en ráfaga) | +| **`BoundedReportQueue`** | Encode pickle/JSON y envío TCP en el hot path de voz; coalesce de snapshots config/bridge duplicados | + +## Servidor: informes y despliegue + +- **Informe v2** — JSON estructurado sustituye snapshots pickle opacos de bridge/config en el cable hacia **monitor 2.x**. Ver [Monitor e informes](../user-guide/monitoring.md) y [Protocolo de informes v2](../protocols/report-v2.md). +- **Proxy integrado** — `PROXY` **in-process**; quitar **adn-proxy** standalone ahorra **RAM** base (un intérprete, config compartida) y simplifica operación. + +## Monitor (adn-monitor 2.x) + +Empareja **adn-server 2.x** con **adn-monitor 2.x** para las mejoras del lado informes: + +| Cambio | Efecto | +|--------|--------| +| **Wire slim / `dashboard_state`** | El monitor ingiere JSON compacto en lugar de duplicar árboles pickle v1 | +| **`clean_sys_dict`** | Expulsión periódica de entradas obsoletas en memoria (tope de crecimiento en paneles largos) | +| **Caché lastheard, fingerprints WS ligeros** | Menos trabajo por refresco del dashboard | +| **Stack FastAPI unificado** | Eliminados API PHP y proceso **proxy** standalone del monitor | + +Detalle: [Arquitectura del monitor](../../monitor/architecture.md). + +## Cuándo se nota la diferencia + +| Despliegue | CPU | RAM | +|------------|-----|-----| +| Pocos masters, sin proxy inject, tráfico bajo | Poca | Poca | +| **Proxy inject-only, muchos hotspots, TG activa** | **Clara** (índice downlink) | Moderada (un proceso servidor vs servidor+proxy) | +| Monitor largo + informe v2 | Moderada (menos serializar en cable) | **Más clara** en monitor (estado slim, `clean_sys_dict`) | + +Crypto, AMBE y MAC OpenBridge siguen dominando en tramos OBP cargados — optimizar la tabla de bridge no elimina ese coste. + +## Lecturas relacionadas + +- [Arquitectura](architecture.md) — capas y entrypoint +- [BRIDGES vs Subscriptions](bridges-vs-subscriptions.md) — modelo de enrutado (no es feature de rendimiento) +- [Comportamiento y temporizadores](behaviour-and-timers.md) — OPTIONS por eventos vs bucle 26 s legacy +- [Proxy hotspot](../user-guide/hotspot-proxy.md) — `PROXY` integrado / inject-only +- [Protocolo de informes v2](../protocols/report-v2.md) — cable JSON al monitor +- Notas de versión: `CHANGELOG.md` en la raíz del repositorio (`Performance` en **2.0.0-rc.1**). diff --git a/docs/es/server/protocols/report-v2.md b/docs/es/server/protocols/report-v2.md index 404d614..46c40b1 100644 --- a/docs/es/server/protocols/report-v2.md +++ b/docs/es/server/protocols/report-v2.md @@ -28,6 +28,22 @@ Sustituir instantáneas al monitor que hoy usan **pickle** (`CONFIG_SND`, `BRIDG Los opcodes `0x10`–`0x13` están reservados en esta fase; pueden ajustarse antes de P1-002. +```mermaid +flowchart TB + subgraph v1 [Informe v1 — adn-server 1.0.x + monitor 1.0.x] + H1[HELLO protocol 1] --> C1[CONFIG_SND pickle] + C1 --> B1[BRIDGE_SND pickle] + B1 --> V1[BRDG_EVENT CSV] + end + + subgraph v2 [Informe v2 — adn-server 2.x + monitor 2.x] + H2[HELLO report_protocol 2] --> T2[TOPOLOGY_SND JSON] + T2 --> R2[ROUTING_TABLE_SND JSON] + R2 --> V2[VOICE_EVENT_SND JSON] + R2 -.-> D2[DELTA_SND opcional] + end +``` + ## Handshake (`hello`) Al conectar, el servidor envía **`HELLO` (`0xFF`)** primero. Clientes v2 miran `report_protocol`: diff --git a/docs/es/server/user-guide/bridges-and-talkgroups.md b/docs/es/server/user-guide/bridges-and-talkgroups.md index a56400d..6c8fb65 100644 --- a/docs/es/server/user-guide/bridges-and-talkgroups.md +++ b/docs/es/server/user-guide/bridges-and-talkgroups.md @@ -12,6 +12,8 @@ La tabla de bridges asocia **claves de talkgroup** (cadenas, p. ej. `"26811"`, ` El router recorre `BRIDGES` buscando una fila **ACTIVE** que coincida con el **sistema de origen actual**, **slot** y **TG de destino** antes de reenviar (`dmrd_received` → `to_target`). +En **adn-server 2.x** las mismas reglas están en **`SubscriptionStore`** / **`SubscriptionRouter`**; `BRIDGES` es solo forma de export para el monitor. Ver [BRIDGES vs Subscriptions](../development/bridges-vs-subscriptions.md). + ## Dinámico frente a estático - Los bridges **activados por usuario** se crean cuando alguien pulsa una TG sin fila previa (sujeto a `DEFAULT_UA_TIMER` y opciones). diff --git a/docs/es/server/user-guide/hotspot-proxy.md b/docs/es/server/user-guide/hotspot-proxy.md index 1308b66..457ebde 100644 --- a/docs/es/server/user-guide/hotspot-proxy.md +++ b/docs/es/server/user-guide/hotspot-proxy.md @@ -14,6 +14,13 @@ La configuración está en **`adn-server.yaml`**, bloques **`PROXY`** y opcional El proxy integrado usa **fan-in**: los hotspots solo necesitan **`PROXY.LISTEN_PORT`** (p. ej. **62031**). El **MASTER** destino es **solo inyección** — **no** abre su propio puerto UDP para ese system (sin rango de puertos por hotspot en el host del servidor). +```mermaid +flowchart LR + HS1[Hotspot A] -->|UDP HBP| LP[PROXY LISTEN_PORT] + HS2[Hotspot B] -->|UDP HBP| LP + LP -->|inyecta| MASTER[TARGET_SYSTEM MASTER] +``` + --- ## Dependencia opcional (self-service) diff --git a/docs/es/server/user-guide/introduction.md b/docs/es/server/user-guide/introduction.md index 9ccd999..a9a5758 100644 --- a/docs/es/server/user-guide/introduction.md +++ b/docs/es/server/user-guide/introduction.md @@ -37,4 +37,5 @@ Enrutado, temporizadores, control de bucle OpenBridge y manejo de protocolo est - [Números especiales](special-numbers.md) — TG 4000, servicios de información, eco. - [Proxy hotspot](hotspot-proxy.md) — **`PROXY`** / **`SELF_SERVICE`** integrados en `adn-server.yaml`. - [ADN Monitor](../../monitor/index.md) — panel, `adn-monitor.yaml`, UI self-service (repo aparte, desplegado con el servidor). +- [Rendimiento (2.x)](../development/performance.md) — mejoras de CPU/RAM en esta versión y qué las provoca. - [Créditos y licencia](attribution.md) — ADN → FreeDMR → hblink3, licencia. diff --git a/docs/es/server/user-guide/monitoring.md b/docs/es/server/user-guide/monitoring.md index 7b43f9f..3d072c9 100644 --- a/docs/es/server/user-guide/monitoring.md +++ b/docs/es/server/user-guide/monitoring.md @@ -24,6 +24,14 @@ El **monitor** decodifica estos mensajes, actualiza **CTABLE** / **BTABLE** y (c **Pila completa:** [Descripción general del ADN Monitor](../../monitor/index.md) (monitor FastAPI, WebSocket, self-service). +```mermaid +flowchart LR + SRV[adn-server\nREPORTS.REPORT_PORT] -->|TCP netstring| ING[ingest adn-monitor] + ING --> STATE[CTABLE / BTABLE] + STATE --> WS[WebSocket /ws] + WS --> UI[panel React] +``` + ### Líneas de log del canal de informes (logger `adn-monitor`) Python usa el nombre de logger **`adn-monitor`** (ver **`LOGGER.LOG_FILE`** en `adn-monitor.yaml`). **INFO** típicos del cliente TCP de informes: diff --git a/docs/es/server/user-guide/special-numbers.md b/docs/es/server/user-guide/special-numbers.md index e00e35b..ccabdf4 100644 --- a/docs/es/server/user-guide/special-numbers.md +++ b/docs/es/server/user-guide/special-numbers.md @@ -98,6 +98,8 @@ El **audio** se envía con **ID de fuente 5000** y **TG de destino 9** en el flu **Propósito:** las filas de bridge para **eco** suelen usar **9990** con el sistema **ECHO** (ver `BRIDGES` y opciones en tu YAML). +**`SINGLE=1`:** pulsar **9990** **no** crea sesión de escucha exclusiva (igual que **4000**). El downlink del eco vuelve siempre al hotspot llamante aunque otra TG tenga el bloqueo SINGLE. Ver [Proxy hotspot](hotspot-proxy.md#comportamiento-con-varios-hotspots). + **Nota:** Un **echo independiente** también está disponible como proceso aparte — [Echo](echo.md). ## Llamada privada al ID 4000 diff --git a/mkdocs.es.yml b/mkdocs.es.yml index abfadca..f5e97fa 100644 --- a/mkdocs.es.yml +++ b/mkdocs.es.yml @@ -32,10 +32,15 @@ theme: plugins: - search + - mermaid2 markdown_extensions: - attr_list - - pymdownx.superfences + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:mermaid2.fence_mermaid_custom - pymdownx.tabbed: alternate_style: true - admonition @@ -67,6 +72,8 @@ nav: - Protocolo de informes v2 (JSON): server/protocols/report-v2.md - Desarrollo: - Arquitectura: server/development/architecture.md + - Rendimiento (2.x): server/development/performance.md + - BRIDGES vs Subscriptions: server/development/bridges-vs-subscriptions.md - Comportamiento y temporizadores: server/development/behaviour-and-timers.md - Contribuir: - Traducciones: server/contributing/translations.md diff --git a/mkdocs.yml b/mkdocs.yml index 9d8c002..f817ca8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -32,10 +32,15 @@ theme: plugins: - search + - mermaid2 markdown_extensions: - attr_list - - pymdownx.superfences + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:mermaid2.fence_mermaid_custom - pymdownx.tabbed: alternate_style: true - admonition @@ -67,6 +72,8 @@ nav: - Report protocol v2 (JSON): server/protocols/report-v2.md - Development: - Architecture: server/development/architecture.md + - Performance (2.x): server/development/performance.md + - BRIDGES vs Subscriptions: server/development/bridges-vs-subscriptions.md - Behaviour and timers: server/development/behaviour-and-timers.md - Testing: server/development/testing.md - Contributing: diff --git a/pyproject.toml b/pyproject.toml index b3b2770..75f63c3 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -24,7 +24,7 @@ dependencies = [ mqtt = ["paho-mqtt>=2.0"] selfservice = ["mysqlclient>=2.0"] dev = ["pytest>=7.0", "jsonschema>=4.0", "paho-mqtt>=2.0"] -docs = ["mkdocs>=1.6", "mkdocs-material>=9.5", "pymdown-extensions>=10.3"] +docs = ["mkdocs>=1.6", "mkdocs-material>=9.5", "pymdown-extensions>=10.3", "mkdocs-mermaid2-plugin>=1.1"] [tool.setuptools.packages.find] where = ["src"]