Stream ids were handed out round robin from 1 to the stream limit
without checking whether the id was still taken. Each id has its own
UDP port (LISTEN_PORT + id), so when a long stream still held the id
the round robin came back to, binding its port failed with "Error
opening socket" and the request was rejected, although other ids were
free. xlxd then opened the stream without transcoding.
OpenStream now tries each id once, starting after the last one used,
and takes the first id not held by an open stream.
Patch by Patrice, F4GEN.
M17: compute the stream packet CRC on outbound frames
xlxd rebuilds every M17 stream packet it relays and wrote 0x0000 in the
CRC field. Add the M17 CRC-16 (poly 0x5935, init 0xFFFF, no reflection,
no final XOR) to CCRC and append it, big-endian, over bytes 0..51 of
every voice, last-frame and synthetic EOT packet.
M17: accept any unencrypted voice stream TYPE
Inbound stream packets were accepted only when the low byte of TYPE was
exactly 0x05, so a stream declaring GNSS or extended callsign META, or a
non-zero CAN, had every packet dropped without a log line and the whole
transmission was lost. Decode the full 16-bit TYPE and accept any stream
whose data type is voice (Codec2 3200) with no encryption, whatever its
META, CAN or signed bit. Unsupported streams (encrypted, voice+data,
data, packet mode) are still dropped, now with one log line per stream.
Signed streams carry their signature in the last four frames. xlxd
re-encodes every stream, so the signature cannot be passed on: the
three signature-only frames are dropped, the final frame is sent as
Codec2 silence, and the stream goes out unsigned.
M17: number inbound frames from the packet stream counter
Inbound M17 frames took their packet ID from the M17 frame number
(FN * 2, truncated to 8 bits). The D-Star side numbers frames modulo
21, and 256 is not a multiple of 21, so every 5.12 s D-Star listeners
received a 4-frame superframe and an early sync frame, restarting the
slow data. Leave the packet ID undefined, as other non-D-Star modes do,
so the packet stream numbers frames from its own continuous counter.
YSF: reassemble multi-frame Wires-X commands
A Wires-X command can span several data frames (FN 1..FT). Each frame
was decoded into a buffer local to the call, so the frames were never
combined: the final frame was CRC-checked without the earlier frames'
bytes, and a non-final frame was decoded as a whole command with no CRC
check. Two-frame requests, such as the FT2D's ALL_REQ for Search/All,
were dropped and the module list never appeared.
Reassemble each command per source (IP:port) in a zeroed buffer,
accepting frames only in sequence, with the same FT and no more than
1 s apart. A command is decoded once, on its final frame, after the end
marker and CRC check, which now only looks at the bytes received.
Frames 3 onwards use the 40-byte layout of G4KLX's YSFGateway; the
previous 20-byte stride overwrote frame 2's data.
xlxd.interlink: remove the REF030, XRF030 and DCS030 examples.
Evens out audio across modes and stations, and moves ambed's settings into
a config file so levels can be tuned without a rebuild or restart.
Audio chain:
- Every path now runs decode -> <MODE>_TO_PCM -> bandpass filter -> AGC ->
PCM_TO_<MODE> -> encode, including M17 <-> P25.
- One decode and one encode gain per mode replace the per-path gains.
- Signal processing is rebuilt for each stream, so a stream starts clean and
doesn't inherit the previous station's gain.
- The bandpass filter clamps its output to the short range.
AGC:
- Levels the speaker, not the syllable. It works like the AGC on an HF
receiver: fast down, hold through gaps, then up, faster the further a
station is below the target.
- Follows speech peaks, moving the gain between syllables and leaving the
shape of the speech alone for the vocoders.
- Pauses and noise below AGC_GATE leave the gain where it is.
- The gain ramps across each frame, so a change can't click.
- A peak limiter leaves room for the highest encode gain, so a boost can't
clip an encoder.
ambed.ini:
- The command line is now ambed [ambed.ini], and main.h holds the defaults.
The old ambed <ip> form is refused, so it can't bind to the wrong address.
- Holds the listen IP and port, DSCP marking, stream timeout, logging, gains,
AGC and filter settings. USE_AGC and USE_BANDPASSFILTER are now runtime
options.
- Checked every 3 minutes. Most changes apply from the next stream, DSCP
changes at once, and the listen IP and port need a restart. xlxd's
TRANSCODER_PORT must match LISTEN_PORT.
- A setting left out uses its default. A bad value is logged and the value
in use stays.
- make install copies the sample only if there is no ambed.ini, and the run
script and systemd service pass its path.
Tuning log: when a stream closes, ambed logs voice level, peak and clipping
after decode, after the decode gain, after the AGC and after each encode
gain, plus the speech peaks, AGC gain and limiter activity. With
AGC_CLAMPING=0 the audio is untouched and the levels are still logged.
TUNING_LOG turns the log on or off.
The readme is now README.md, rewritten to cover the audio chain, every
setting, how to tune and an example systemd unit.
NXDN ID lookups now come from the XLX API server, with radioid.net as the
fallback. Source and parser rework by Luc (LX1IQ).
- Reads the XLX API's "id;callsign;" format as well as the CSV.
- A failed or empty download keeps the current database.
- Connect, read/write and total timeouts on the HTTP fetch.
- Every ID gets its callsign, and a callsign with several IDs (one per
radio) maps back to the first of them.
- Only an HTTP 200 counts as a database, so a redirect or error page isn't
parsed as IDs. radioid.net now answers HTTP with a 301, so the fallback
needs HTTPS support to work.
- NXDNIDDB_USE_RLX_SERVER now defaults to 1, as for the DMR and YSF
lookups.
YSF/NXDN/P25 peer pages:
- Only shown when xlxd.interlink has a local (127.0.0.1 or localhost) peer
of that type, as they read the journal of a reflector on this host.
Otherwise the menu item is hidden, journalctl isn't run, and the page
explains why.
- config.inc.php notes the pages are local only.
Interlink file:
- Can live at any path set in config.inc.php, but must be a file named
xlxd.interlink. Before, anything outside /xlxd/ hid the peer pages and,
in dashboard1, sent calling home an empty interlink list.
- HasLocalPeer() logs why it can't read the file: not found, wrong name or
not readable.
- An empty InterlinkFile falls back to /xlxd/xlxd.interlink.
Calling home:
- dashboard1 now accepts host:port addresses, so YSF/NXDN/P25 peers are no
longer sent with an empty address.
- A blank line in xlxd.interlink no longer adds an empty interlink.
- Fixed PHP 8.1+ deprecation notices from null interlink fields.
Versions bumped to 2.4.7 (dashboard1) and 2.3.11 (dashboard2), with
changes.txt updated.
DVRef.com and RefCheck.radio validate reflectors with DVREFCHK. D-Star
callsigns are 7 chars + module, so it arrives as callsign "DVREFCH",
which did not match the existing DVREFCK exemption and was NAKed for
having no digit. Move the exemptions into IsValidatorCallsign() and
list both DVREFCK and DVREFCHK.
REGRESSION FIXED
Yaesu radios in System Fusion V/D mode transmit the operator's
selected DG-ID (Digital Group ID) in the FICH SQ field. xlxd
previously (2021, upstream commits 32c9e0c + e752b76 by narspt)
allowed a native YSF client to switch which reflector module they
were linked to by dialling the DG-ID on their radio. That feature
was silently removed in upstream commit 27f5678 -- part of a
large multi-protocol peering rewrite -- and stayed removed through
the recent hardening work in this fork.
Current behaviour before this fix: every YSF native client was
pinned to whichever module they auto-linked or Wires-X CONN_REQ'd
to, with no way to switch mid-session. This restores the DG-ID
mechanism with the correct scope and the gatekeeper-bypass fix
that narspt applied in the second 2021 commit.
DESIGN
Three-part change in cysfprotocol.cpp, no other files touched.
1. Task() -- new DG-ID stamping block between the isPeer check
(lines 139-158) and the gatekeeper check (lines 213+):
if ( !isPeer )
{
unsigned char sq = Fich.getSQ();
char dgidModule;
if ( (sq >= 10) && (sq < 10 + NB_OF_MODULES) )
{
dgidModule = 'A' + (char)(sq - 10);
}
else
{
dgidModule = clientModule; // client's current module
}
Header->SetRpt2Module(dgidModule);
}
DG-ID values 10..(10 + NB_OF_MODULES - 1) map to modules A..(letter).
Values outside that range (typically 0..9 for "no DG-ID intent",
or above the module range) leave the client on their currently-
linked module.
Applied to native clients ONLY (isPeer gate). Peers represent
external reflectors whose module identity is fixed by interlink
config -- a peer's upstream user's DG-ID must not move the peer's
own link.
2. Task() -- gatekeeper call at line 215 changed from passing
`clientModule` (client's current module) to passing
`Header->GetRpt2Module()` (the DG-ID destination if applicable,
otherwise the client's current module).
This closes the gatekeeper bypass narspt fixed in 2021:
without this, a callsign muted on module A could dial DG-ID
for module B and transmit under B's (unmuted) rules because
MayTransmit was checking module A's whitelist / loop-suppress
state instead of module B's.
3. OnDvHeaderPacketIn -- unconditional
`Header->SetRpt2Module(client->GetReflectorModule())` at line
422-423 replaced with peer-vs-native branching:
if ( client->IsPeer() )
{
// Peer traffic: always use peer's assigned link module.
Header->SetRpt2Module(client->GetReflectorModule());
}
else
{
// Native client: apply DG-ID module switch if the module
// stamped by Task() differs from the client's current one.
char headerModule = Header->GetRpt2Module();
bool validModule = (headerModule >= 'A') &&
(headerModule < 'A' + NB_OF_MODULES);
if ( validModule &&
(headerModule != client->GetReflectorModule()) )
{
std::cout << "YSF client " << client->GetCallsign()
<< " linking on module " << headerModule
<< " (DG-ID)" << std::endl;
client->SetReflectorModule(headerModule);
}
if ( !validModule )
{
Header->SetRpt2Module(client->GetReflectorModule());
}
}
The validModule test is sentinel-agnostic -- it accepts only
real module letters A..(A+NB_OF_MODULES-1). The parser's 'G'
default and any accidental ' ' value both fail the test and
trigger the fallback. This decouples the OnDvHeader logic from
whatever sentinel the parser uses for "unset" -- if the parser
default changes later, this code doesn't need updating.
PEER EXCLUSION -- three independent layers
Layer 1: Task() DG-ID stamping is gated on !isPeer, where isPeer
comes from FindPeerByIpPort at existing lines 143-144. Peer
packets never have DG-ID mapped to rpt2 in the first place.
Layer 2: OnDvHeaderPacketIn's client->IsPeer() branch
unconditionally overwrites rpt2 with the peer's assigned
ReflectorModule. Belt-and-braces against any future code path
that might stamp rpt2 for peer traffic.
Layer 3: Class hierarchy. CYsfPeerClient::IsPeer() returns true
(cysfpeerclient.h:48); CYsfClient (native user) inherits
CClient::IsPeer() returning false (cclient.h:90). Peer and native
clients are physically different types -- no runtime state can
make one look like the other.
WIRE-LEVEL SAFETY -- confirmed no DG-ID propagation
The three outgoing FICH builders in cysfprotocol.cpp (lines 1067,
1122, 1217) all set setSQ(0) hardcoded on every outbound packet.
The source's DG-ID is consumed on ingress (in the Fich local at
RxTask() line 94) and discarded when RxTask() returns; the
CDvHeaderPacket/CDvFramePacket objects that traverse xlxd's
internal packet routing carry no FICH bytes and no SQ field. TX
FICH is rebuilt from scratch with SQ=0 on the other end.
Consequence: DG-ID is a purely receive-side, per-reflector
decision. No cross-reflector propagation is possible even if
DG-ID handling malfunctioned upstream.
DG-ID MAPPING
DG-ID | Module | Behaviour
-------|--------|--------------------------------------------------
0..9 | - | No DG-ID intent -- keep client's current module
10 | A | Switch client to module A (if not already there)
11 | B | Switch client to module B
... | |
10+N-1 | (nth) | Switch client to nth module (default 10 modules: A..J)
36+ | - | Out of range -- keep client's current module
The 10-offset avoids collision with the SQ values a Yaesu radio
typically emits for non-DG-ID situations (0..9 span the
"no group set" states).
EXPLICITLY OUT OF SCOPE
- Parser default at IsValidDvHeaderPacket line 862-863
(`rpt2.SetModule('G')`) is unchanged.
- Dead ysfModule peer-lookup at lines 848-856 is unchanged.
- Pre-existing peer-classification gap where a peer packet
arriving on a mismatched source port could be misclassified
by Task()'s FindPeerByIpPort as !isPeer, while
OnDvHeaderPacketIn's IsPeer()-based fallback correctly
identifies it. In that divergent case: Task() would run
the native-client DG-ID path (potentially stamping a wrong
rpt2) and MayTransmit path (running the wrong gate). Layer 2
in OnDvHeaderPacketIn still correctly forces the peer's
module. Not addressed in this commit.
peer loop block, multi-peer-per-IP, protocol diagnostics
Comprehensive production-hardening release for xlxd + ambed. Covers
the D-Star egress pipeline, adaptive jitter, ambed decoupling and
PID preservation, XLX interlink codec routing (P25 support + BM-
peer compatibility), per-callsign peer loop suppression, multi-
peer-per-IP for server-style protocols, M17 synthetic EOT timing,
and enriched Wires-X diagnostics. Production-tested on XLX672
(March-June 2026) under real cross-mode traffic, WAN jitter,
source-side bursts, and peer-loop attack.
================================================================
1. Cross-mode -> D-Star egress: full JARL spec compliance + RP2C
================================================================
D-Star receivers vary in strictness. Software decoders (ircDDBGateway,
MMDVM) tolerate missing slow-data; Icom RP2C / G3 hardware does not.
Cross-mode -> AMBE+ output now generates a fully spec-compliant
21-frame slow-data cycle (cdstarslowdata.{h,cpp} new), with:
- Sync marker (0x55 0x2D 0x16 unscrambled) on frame 0
- Cycle 0: text message (4 x 6-byte scrambled segments, frames 1-8)
- Cycles 1+: header-sync (9 x 6-byte scrambled elements, frames 1-18)
- Pre-EoT marker (wire bytes 55 55 55) on the last voice frame
immediately before the EoT frame
Pre-EoT marker tracking works via m_pLastVoiceInJitter -- a non-owning
pointer to the most recent voice frame still in the jitter buffer.
When the EoT arrives, that pointer's DvData is overwritten in-place
with the pre-EoT marker before release. Strict Icom decoders that
gate on this signal can no longer mute audio at stream close.
================================================================
2. Decouple D-Star pass-through from ambed
================================================================
Pre-fix architecture: packets entered the jitter buffer ONLY after
ambed responded. UDP loss to/from ambed caused source frames to
pile up in m_LocalQueue, never reach the jitter buffer, and be
deleted at stream close -- silently destroying D-Star AMBE+ that
listeners didn't need ambed to play. Symptoms: broken tails on
D-Star -> D-Star, elevated reported BER from FEC alignment errors
at the abrupt cut-off.
New architecture: ambed is ENRICHMENT, not a GATE. Voice frames
flow through the jitter buffer on a fixed 20 ms cadence regardless
of ambed health. ambed responses fill in transcoded codec slots
(Codec2 / IMBE / AMBE+2) in-place via a PID-keyed lookup map
(m_PendingTranscode) while the packet sits in the jitter buffer.
Source-codec slot is filled at Push time; D-Star pass-through audio
reaches D-Star listeners in lockstep regardless of UDP loss to ambed,
ambed being slow, or ambed being completely offline.
PID-keyed lookup also fixes a latent bug in the previous FIFO-pop
design: a single dropped ambed return packet would silently mis-pair
every subsequent packet's transcoded slots until end of stream.
================================================================
3. Adaptive jitter buffer
================================================================
Replaces the fixed 160 ms jitter with per-stream adaptive sizing
based on observed ambed RTT and source-side inter-arrival jitter.
P95 of RTT samples (sliding window 100 = ~2 s at 50 fps) plus a
30 ms safety margin, OR max source arrival, whichever is larger.
Asymmetric hysteresis: increase target immediately when network
degrades (delta > +20 ms); decrease only after 3 consecutive
recomputes agree (delta < -20 ms each). Cold-start 160 ms, floor
80 ms (2x typical LAN RTT), ceiling 400 ms (above which real-time
conversation breaks down anyway). Stall-detection caps inter-
arrival samples at 500 ms so a silence-gap doesn't pin the buffer
at MAX for the next minute.
================================================================
4. ambed PID preservation through DVStick pipeline
================================================================
DVStick hardware response parser (IsValidChannelPacket) creates a
fresh CAmbePacket from raw USB bytes and has no way to recover the
PID that was on the corresponding input packet -- every output had
PID=0, breaking xlxd's PID-keyed response routing (every response
after the first looked like a stale duplicate).
Fix: each CVocodecChannel holds a FIFO of (pid, pushedAt) entries.
USB interface pushes when consuming input; pops when producing
output. The DVStick processes packets in FIFO order per physical
channel, so pop order matches push order. PID round-trips correctly
via FIFO pairing rather than byte-in-the-packet preservation that
the hardware protocol doesn't support.
Two desync-recovery protections:
- PopPid prunes orphaned entries (front older than 500 ms) before
returning. Bounds the mis-pairing window after a hardware drop
to ~timeout duration; without it the desync would be permanent
for the rest of the stream. 500 ms is well above healthy ambed
RTT (40-100 ms) and well below conversational silences.
- PushPid caps FIFO occupancy (drops oldest if > 200). Memory-
safety belt; only fires under pathological sustained input
overrun (4 s of frames at 50 fps).
Drops are tracked separately for diagnostics: orphan (timeout) vs
overflow (cap), different operational meanings.
================================================================
5. Stream-id allocation: per-source, randomly-seeded
================================================================
YSF/NXDN/P25/M17 now allocate per-transmission stream-ids via a
per-source map (m_SourceStates) rather than deterministic
IpToStreamId. Deterministic IDs collided on rapid rekey, producing
"multi-EoT" cross-mode incidents where a second transmission from
the same source within ~2 s appeared to be a continuation of the
first. Global monotonic counter seeded from std::random_device at
process start; per-protocol collision check against m_SourceStates
as defence-in-depth.
P25 additionally: HasActiveStreamIdForSource gate before
AllocateNewStreamIdForSource on every LDU1_VOICE1. P25 has no
explicit start-of-transmission frame type -- LDU1 fires every
360 ms mid-transmission, so allocating fresh on every LDU1_VOICE1
cut a single transmission into many short substreams and spammed
the loop detector. Allocate ONLY when there's no active mapping;
release on TDU.
================================================================
6. Overrun protection: initial-fill exemption + hybrid threshold
================================================================
The pre-codec ring buffer in CPacketStream (capped at 100 frames)
deliberately replays accumulated frames in microseconds when
AttachCodecStream completes after the AMBEd handshake. That replay
burst is *intended* to preserve audio across the handshake delay;
running overrun protection during it self-defeats. Gated on
m_bFirstReleaseDone (latches at first Phase 2 release).
Overrun threshold itself is now hybrid: max(target x 2.0,
target + 12 packets). Pure 2.0x was too tight at low targets
(8-pkt target x 2 = 16-pkt threshold, but P25 source-side WAN
jitter produced 17-packet momentary peaks). Hybrid:
- At low targets (cold-start 160 ms = 8 pkts): additive
dominates, threshold 20 (400 ms peak buffered).
- At high targets (400 ms ceiling = 20 pkts): ratio dominates,
threshold 40 (800 ms peak -- unchanged from before).
- Crossover at target=240 ms.
Slow-bleed drop cap (8 per Phase 2 tick) ensures sustained
overrun bleeds across multiple ticks rather than creating one
huge cadence-stutter event.
================================================================
7. Diagnostic instrumentation in close-time stats
================================================================
ambed-stats log line gains four new fields so the next overrun-
drop incident is self-explanatory:
ambed stats (ms) : X/Y/Z -- N sent, N returned, A late, B unfilled,
jitter X/Y/Z ms, N drops [peak=N arrival_max=Nms
drops@first=N/last=N]
Where:
- peak : max jitter-buffer occupancy seen during stream
- arrival_max : longest admitted source inter-arrival in ms
- drops@first : m_uiTotalPackets at first overrun drop
- drops@last : m_uiTotalPackets at last overrun drop
Pure additive -- no behaviour change, just observability. Caught
the 159 ms WAN gap that drove the hybrid-threshold tuning.
================================================================
8. Per-callsign peer loop block
================================================================
Production loop incident on a YSF peer: peer interlink echoed
one user's audio back at the reflector ~1 stream/second for
several minutes. The strike-based loop detector correctly fired
"strike 3 TX blocked 300s" repeatedly, but the block was dead state
-- two deliberate bypasses for peer traffic meant nothing checked it.
Bypass #1: every peer-aware protocol handler short-circuits the
gatekeeper for peer IPs: `if ( isPeer || MayTransmit(...) )`.
IsLoopSuppressed lives inside MayTransmit, so peer traffic never
sees the block.
Bypass #2: at strike-3, m_bLinkBlocked and shouldDisconnect are
gated on !isPeer. Peer interlink itself cannot be auto-killed
(correctly preserved).
P25 was the silent case: zero MayTransmit calls anywhere in
cp25protocol.cpp, so strikes accumulated unenforced.
Fix: new CGateKeeper::IsCallsignLoopBlocked() -- pure read of
m_LoopPenalties, returns true iff the callsign is inside an
active strike-backoff window. Called from six peer-aware
protocol handlers (YSF, DCS, DPlus, DExtra, NXDN, P25) on every
incoming peer-sourced header. Drop at protocol layer if blocked;
no MayTransmit whitelist/blacklist/callsign-validity work for
peer traffic (peers are trusted interlinks).
Rate-limited drop log:
Loop block: dropping incoming header from <CALLSIGN>
(block expires in <N>s via <PROTOCOL> peer)
NXDN required restructuring to defer OnDvHeaderPacketIn outside
the Peers lock (mirrors the existing deferredLastFrame pattern)
so IsCallsignLoopBlocked -> LoopMutex doesn't introduce a novel
Peers -> LoopMutex lock ordering. P25 releases the per-source
SID mapping on block so subsequent LDU cycles re-allocate and
get re-checked at Voice5 rather than orphaning frames through
OnDvFramePacketIn with no open stream. Peer link itself stays
up -- only the offending callsign is suppressed.
CTimePoint default-constructs to now() not epoch, so
m_bDropEverLogged bool gates the first rate-limited emission;
subsequent ones gate on elapsed time.
================================================================
9. Multi-peer-per-IP for YSF/NXDN/P25
================================================================
Two YSF (or NXDN, or P25) reflectors hosted on the same machine
could not both be attached as peers. The interlink-file lookup and
runtime peer routing both matched by IP-addr only, so the second
peer at the same IP collapsed into the first one's bookkeeping
silently. Scope: YSF, NXDN, and P25 -- protocols where peers listen
on well-known ports declared in xlxd.interlink and the inbound
source port equals the peer reflector's listening port (server-
style semantics). DExtra, DPlus, DCS, XLX, and M17 keep their
existing IP-only matching unchanged because they have NAT-friendly
/ fluctuating-source-port behaviour that the IP-only comment at
CPeers::FindPeer specifically documents.
New method CPeers::FindPeerByIpPort(const CIp &, int Protocol),
declaring match semantics via signature rather than a boolean
parameter or protocol-conditional switch. Nine runtime call sites
switched from FindPeer to FindPeerByIpPort (inbound-packet routing).
Callsign-keyed lookups (used by periodic reconnect-poll paths) stay
unchanged.
Three protocol-specific interlink-file helpers made port-aware
(FindYsfPeerByIp, FindNxdnPeerByIp, FindP25PeerByIp): if the
interlink entry has an explicit port (item->GetPort() != 0), match
(addr, port); if no explicit port (legacy entries), fall back to
addr-only -- preserves single-peer-per-IP behaviour for existing
operator configs.
CPeers::AddPeer conflict warning updated from GetAddr()==GetAddr()
to GetIp()==GetIp() (full endpoint match) -- otherwise two
legitimate same-IP-different-port peers would trigger a spurious
warning on every connect.
================================================================
10. CM17Protocol: synthetic EOT timing during ambed starvation
================================================================
Production incident: a 59-second D-Star -> M17 cross-mode
transmission triggered a synthetic M17 EOT just 4 seconds in while
the upstream stream was still active. Late-entry attempts from
other callsigns piled up. Close stats: 2942 sent / 2942 returned /
2774 unfilled (94% empty Codec2), peak=18, no drops. ambed itself
was healthy; the M17 output had nothing to send for most of the
transmission and the EOT timer misfired off that silence.
Root cause: m_LastFrameSentTime was updated only when buffer.size()
> 0 -- meaning only when M17 emitted wire bytes. With 94% of frames
lacking Codec2 transcode results, the timestamp froze, and 500 ms
later CheckForMissedEOT fired a synthetic EOT, tearing down the
listener-side stream state.
Fix:
- Renamed m_LastFrameSentTime -> m_LastUpstreamActivityTime to
match the actually-needed semantic.
- Moved the .Now() update out of the `if (buffer.size() > 0)`
block into a separate `if (packet->IsDvFrame())` guard. Refreshes
the timestamp on every upstream voice frame regardless of
whether ambed delivered Codec2 in time.
- Left m_bStreamActive gated on buffer.size() > 0 (its semantic
is "we told M17 listeners about this stream by emitting wire
bytes").
Does NOT address the underlying ambed starvation (94% unfilled) --
that's a separate concern. M17 listeners still hear silence on
unfilled frames, but the stream state machine stays coherent.
================================================================
11. CYsfProtocol: Wires-X diagnostic enrichment + UB closure
================================================================
Production complaint: the bare "Wires-X unknown command" log line
was useless for diagnosis -- no callsign, no IP, no bytes.
IsValidwirexPacket signature gains `const CIp &Ip` for diagnostic
labelling. Single unified log line shape:
Wires-X command from <CALLSIGN> @ <IP>:<PORT>
(<peer-or-client>) -- code XX XX XX (TAG, Dropped)
Where TAG is one of:
UNKNOWN -- code not recognised
DX_RESP -- server response 5D 51 5F 26
CONN/DISC_RESP -- server response 5D 41 5F 26
ALL_RESP -- server response 5D 46 5F 26
Server-response detection (matched with full 4-byte signature
including the 0x26 marker byte). A client sending a response code
TO xlxd is anomalous -- misconfigured peer, relay loop, RF bridge,
or probe. Packet dropped; suspicious source surfaced loudly.
ALL_REQ sub-code dispatch -- UB closure:
Pre-existing bug: the ALL_REQ family branch (5D 66 5F) checks
command[5..6] for "01" (ALL) or "11" (SEARCH) sub-codes. If
neither matched, the original code left *Cmd unset and returned
valid=true. Caller constructed CWiresxCmd with uninitialised
memory; C++ standard says this is undefined behaviour. Fixed by
always setting *Cmd = WIRESX_CMD_UNKNOWN, *Arg = 0, valid = false
on any unrecognised ALL_REQ sub-code variant. Three permanent
assignments tagged "// KEEP (UB-closure)" to document survival
requirement.
Bounds-safety guards added:
The CRC loop's end-marker position is now plumbed out into
endMarkerPos so downstream byte reads can gate on it:
- Response-signature 4-byte memcmp gated on endMarkerPos >= 4
- ALL_REQ sub-code memcmps gated on endMarkerPos >= 6
Without these, a degenerate-but-CRC-valid packet with an early
end marker could read uninitialised stack memory inside the
300-byte command buffer.
================================================================
12. XLX interlink codec routing: needed-vs-available with
source-authoritative Has-flags
================================================================
Cross-XLX P25 calls produced silence on the receiving side. Path:
USER1(P25) <-> P25REF1 <-> XLX1 <-> interlink <-> XLX2 <-> P25REF2
<-> USER2(P25). XLX1 transcoded P25 IMBE -> AMBE+/AMBE+2/Codec2
for the rev-3 interlink. The XLX rev-3 wire format has NO IMBE slot
(D-Star/DMR/M17 only). XLX2's CXlxClient::GetCodec() returns
CODEC_NONE for rev 2/3 on the assumption "all codecs are pre-
provided by the wire frame" -- true when XLX was D-Star + DMR + M17
but wrong for P25. OpenStream's gate `if ( inputCodec != CODEC_NONE )`
skipped the transcoder, IMBE was never filled, USER2 heard silence.
Same bug class affected M17 on rev-1/rev-2 XLX peers.
DESIGN -- Two-axis model:
1. Each client provides a SET of codecs (bitmask). New virtual
CClient::GetAvailableCodecs() defaults to {GetCodec()}.
CXlxClient overrides per rev:
rev 0/1: {AMBE+}
rev 2 : {AMBE+, AMBE+2}
rev 3 : {AMBE+, AMBE+2, Codec2}
2. Each module has a NEEDED codec set. AMBE+ and AMBE+2 are
always-on "core" codecs (D-Star, DMR-family coverage) --
operator policy. Codec2 added iff an M17 client is on the
module; IMBE added iff a P25 client is.
OpenStream: compute needed minus available. If non-empty AND
transcoder connected, pick source codec by preference (AMBE+2 >
AMBE+ > Codec2 > IMBE), engage transcoder. Otherwise no transcode.
HAS-FLAG GUARDS -- the wire-data preservation half:
CDvFramePacket previously had m_bHasCodec2 and m_bHasImbe flags but
not AMBE+/AMBE+2. Added m_bHasAmbe and m_bHasAmbePlus to complete
the four-codec Has-flag symmetry. SetAmbe atomically sets the
matching flag with the memcpy. All 9 constructors initialise the
flags explicitly. operator== extended.
CCodecStream Phase 1 response handler now uses fillIfMissing lambda
that guards each write on the corresponding Has-flag -- pre-populated
slots (from the wire frame) are preserved; only missing slots get
filled by the transcoder. Prevents re-transcoded data from
overwriting authoritative wire data (avoids an extra vocoder hop
for D-Star / DMR / YSF / NXDN listeners).
SOURCE-AUTHORITATIVE HAS-FLAG SEMANTIC:
A Has-flag means "authoritative data is in this slot -- the source
genuinely provides this codec, or a downstream transcoder filled
it." Authority comes from GetAvailableCodecs(), NOT the wire format.
The XLX wire-frame constructor cannot know the source (called by
IsValidDvFramePacket with just a buffer, no peer context) so it
does NOT set m_bHasAmbe or m_bHasAmbePlus -- they default false.
Source-aware code takes over: CXlxProtocol::OnDvFramePacket{,Last}In
calls a new ConformAvailableCodecs(uint32 mask) method with the
mask from the source peer's GetAvailableCodecs(). Assignment-based
(not OR) -- clears flags for codecs the source doesn't provide, sets
them for codecs it does.
This is critical for BM-style XLX peers (CXlxDmrPeer with
CXlxDmrClient -- BrandMeister, FreeDMR, FreeStar, DVSPH bridges
matching BM*/FD*/FS*/PH*). Those peers are forced to rev-2, provide
only AMBE+2, but their rev-2 wire frame reserves an AMBE+ slot that
they populate with empty/stale bytes. Without source-authoritative
Has-flags, the wire-format-set m_bHasAmbe = true would lock the
Phase 1 guard out of the very slot the transcoder needs to fill.
D-Star listeners on a BM-bridged stream would hear silence.
The mask-extract-under-lock + Conform-after-release helper
(ConformXlxPacketToSource) guards on codecMask != 0U -- if the peer
can't be identified (Ip NULL, race during disconnect), Conform is
skipped and default-false flags stand. Codec stream fills every
slot from the transcoder -- extra vocoder hops, never silence.
EFFECT MATRIX:
Source | Result
------------------------------|----------------------------------
Native CXlxClient rev 0/1 | Transcode AMBE+2/Codec2/IMBE
Native CXlxClient rev 2 | Transcode Codec2 (M17) / IMBE (P25)
Native CXlxClient rev 3 | Transcode IMBE only when P25 needed
CXlxDmrClient (BM/FD/FS/PH) | Transcode AMBE+/Codec2/IMBE
================================================================
Files changed
================================================================
ambed: cstream, cusb3xxxinterface, cvocodecchannel (PID FIFO)
xlxd: ccodecstream (jitter buffer + adaptive + hybrid threshold
+ diagnostic instrumentation + EoT marker + Phase 1
fillIfMissing guard)
cpacketstream (pre-codec ring buffer)
cgatekeeper (IsCallsignLoopBlocked + rate-limit state)
cdstarslowdata (new -- JARL slow-data synthesiser)
cdvframepacket (four-codec Has-flag symmetry + Conform)
cdvheaderpacket (DvData accessors)
cclient (GetAvailableCodecs virtual)
cxlxclient (rev-conditional GetAvailableCodecs override)
cxlxprotocol (ConformXlxPacketToSource helper)
cprotocol + cnxdnprotocol, cp25protocol, cysfprotocol,
cm17protocol (stream-id allocation, per-source state,
loop-block enforcement, EOT timing, Wires-X diagnostics)
cdcsprotocol, cdplusprotocol, cdextraprotocol
(loop-block enforcement)
cpeers (FindPeerByIpPort)
creflector (GetNeededCodecsOnModule + OpenStream rewire)
ctranscoder, main (plumbing)
First-class M17 protocol support:
- Full M17 client handling: connect (CONN/LSTN), disconnect (DISC),
keepalive (PING/PONG), voice streaming with 40ms→20ms frame split
- M17 base-40 callsign encoding/decoding per specification
- M17 wire stream ID generation (random-seeded per module)
- Codec2 3200bps frame pairing (two 8-byte frames per 40ms packet)
- Synthetic EOT detection (500ms timer) for clients that don't send
proper end-of-transmission, with cross-mode distribution via
CDvLastFramePacket pushed to reflector stream
- Byte-swapped stream ID fallback for M17 EOT packets
- DISC packet sent to all M17 clients on reflector shutdown
- Listen-only client support
P25 and NXDN peering:
- P25 peer protocol with IMBE voice frame handling
- NXDN peer protocol with AMBE2+ voice frame handling
- AMBE2+↔AMBE2+ gain levelling for NXDN within xlxd
- AMBE2+↔AMBE2+ gain levelling for YSF mode within xlxd
DCS, DPlus (REF), and DExtra (XRF) peer link management:
- Automatic peer connect/disconnect from gatekeeper peer list
- Peer keepalives with timeout detection and reconnection
- Connection state machine (connecting/connected/disconnected)
- Peer client objects for voice routing through existing reflector
- Peer mirror prevention via full IP:port owner matching on DCS,
DPlus, and DExtra peer send paths
AMBEd software vocoder support:
- Async Codec2 encoder thread in CVocodecChannel with condition
variable wake-up, PCM queue, and encoded output queue
- Async IMBE encoder thread (same pattern) using imbe_vocoder library
- Embedded Codec2 library for M17 codec support
- Embedded IMBE vocoder library for P25 codec support
- Staging mechanism in CStream::Task(): hold hardware AMBE result
until software codecs catch up (1s timeout safety net)
- Three-codec output per stream (e.g., AMBEPLUS→AMBE2PLUS+CODEC2+IMBE)
- Precomputed gain multipliers to avoid pow() per packet
- Codec2 decode gain (+6dB) applied in AMBEd stream processing
- IMBE encode gain (-4dB) applied to all PCM→IMBE paths
- Software queue depth caps (SOFTWARE_QUEUE_MAX_DEPTH=50)
Transcoding gains (tuned from live testing):
- Codec2 path: CODEC2_TO_AMBEPLUS -3dB, CODEC2_TO_AMBE2PLUS -13dB,
CODEC2_TO_IMBE -13dB
- IMBE path: IMBE_DECODE_GAIN +18dB, IMBE_TO_AMBE2PLUS -13dB,
IMBE_TO_CODEC2 -7dB
- Hardware: CODECGAIN_AMBEPLUS -10dB, CODECGAIN_AMBE2PLUS +10dB
- Cross-mode AMBE2+: NXDN↔DMR -3dB, DMR→YSF -4dB, YSF→DMR +10dB
RX/TX thread split:
- CProtocol base class: dual-thread infrastructure with RxTask/TxTask,
condition variable on m_Queue for TX wake-up, backward-compatible
via UsesSplitThreads() virtual. Shutdown joins RX first then TX
with final drain loop
- Split protocols: DCS, M17, DExtra (XRF), DPlus (REF), YSF, NXDN, P25
- Legacy single-thread: XLX interlink, DMR+ (DMRplus), DMR MMDVM
- Per-slot std::mutex on stream cache items protecting shared state
between RX and TX threads
- Prevents transmission tail loss by ensuring TX processing never
blocks inbound packet reception
XLX interlink DMR peer generalisation:
- Rename CBmPeer/CBmClient to CXlxDmrPeer/CXlxDmrClient
- Callsign prefix matching for BM (BrandMeister), FD (FreeDMR),
FS (FreeStar), PH (DVSPH) — all rev 2 AMBE2+ only
D-Star end-of-transmission fixes:
- Strip 0x40 end flag from D-Star PID (& 0x1F) in all five D-Star
protocol parsers — the flag shifted EncodeDvPacket's sequence by 1
- Preserve DCS wire format tail (bytes 58-99) via per-module cache
with m_bHasDcsTail guard for cross-mode sources
- DCS HandleQueue sends EOT reliably via independent TX thread
Thread safety and memory safety hardening:
- CPacketStream: cache owner IP at Open() time, eliminating use-after-
free on m_OwnerClient. NULL pointer under stream lock in CloseStream
- Fix ABBA deadlock in all peer management (DCS, DExtra, DPlus, YSF):
snapshot GateKeeper peer list before acquiring Peers/Clients locks
- Fix lock ordering across all protocols to prevent ABBA deadlocks
between Peers, Clients, and GateKeeper mutexes
- Collect items for removal after iteration instead of modifying
containers while iterating
- CUdpSocket::SendVoice: EAGAIN retry (5 attempts, 1ms backoff)
- AMBEd Phase 4 timeout reverted to counter-only (no packet deletion)
- Remove dead CPacketQueue shadow fields, debug RouterThread logging,
IsStreaming() stub. Make g_bDscpLogged atomic
- Client match in keepalive/header paths uses full CIp::operator==
(IP:port) preventing false matches with shared-IP peers
Security hardening:
- Send-time owner exclusion across all protocols using per-module
client cache with snapshotted owner fields under slot lock
- Echo detection for DCS, DExtra (XRF), and DPlus (REF) to reject
reflected audio from peers
- Reject tracker with progressive backoff for invalid connections,
capped map size to prevent memory exhaustion from spoofed IPs
- Sanitized log output to prevent terminal injection
- std::random_device for DMR authentication seed
- Bounds checks on module indices across all protocol handlers
- Use-after-free fixes in OnDvHeaderPacketIn across all protocols
- ARM-portable unaligned access fixes (memcpy for uint16/uint32)
Performance:
- Per-module client cache for DExtra/DPlus/DCS outbound packets
- DSCP EF marking on voice packet sockets for QoS
- Peer lookup maps built once per drain cycle instead of per-packet
- Skip M17 idle packet encoding when no clients connected
LX1IQ
Andy Taylor
Chipster