50 Commits (5.04a_maint)

Author	SHA1	Message	Date
Bryan Biedenkapp	274b805517	Merge R05A04 (r05a04_dev) into master (#114) ... * rebuild r05a04_dev from nasty Git merge bullshit; implement handling of SNDCP on the FNE instead of dvmhost; add quick sanity Catch2 testcases; BUGFIX: NXDN SACCH was incorrectly handling the RAN and structure causing the structure value to become overwritten; correct badly set example IP range in FNE config; add AI generated documentation for the network statck, FNE REST and DVMHost REST; update version number for next dev version; * remove old documents; * BUGFIX: fix some unbounded array checking; * enhance InfluxDB implementation: fix bad handling on Windows; fix bad use of free() for address info freeaddrinfo(); fix partial data writes for bigger payloads; enhance getting responses back from InfluxDB (this makes it so that InfluxDB cannot fail silently easily); * EXPERIMENTAL: add an adaptive jitter buffer implementation to the FNE to better deal with peers on bad connections that may send packets out-of-sequence; * remove unnecessary garbage in AI generated doc; use references for autos when looping peers; normalize jitter buffer log messages; * cleanup doc; * properly move hastily erected debug YAML config to the peer ACL file instead; * update jitter buffer doc; * BUGFIX: correct naive implementation that did not consider the length of the packet being sent, which could result in a nullptr after casting the ip struct; BUGFIX: send actual bytes read from VTUN vs the default MTU size; * BUGFIX: correct similar ip struct casting problem; * implement a very preliminary DMR to VTUN IP dispatcher based on the P25 implementation (while I am sure this is probably going to be quite iffy, I am quite sure the VTUN TAP interface method we are using *is* the right direction; even though the P25 implementation is iffy, with iteration over time I suspect it is the correct approach, as such I have implemented a DMR equiviliant based on the P25 implementation); * use 3/4 rate instead of 1 rate; add some dev notes future me; * implement preliminary positive response to KMM HELLO (this still will fail with No-Service due to missing implementation for per-SU UKEK implementation from the crypto container); * add some extra logging for VTUN PDU; * better implement V(R) V(S) checking; * add prereq logging of last load time for various lookup tables; * add fne-stats; add reload peers and crypto; * update FNE REST API doc; * WIN32: fix ctime_r and add Win32 compat for ctime_s so we can compile on Win32 again; * Win32: FNE's P25PacketData clock() in its entirety should do nothing on Windows; * shut the compiler up about order of constructor initializers; * EXPERIMENTAL: add option for 1 second delayed network TDU when a subscriber dekeys; * slight TUI color palette change; * implement support to configure adaptive jitter buffer from peered; * initialize FinalCut palette earlier; * document that key IDs are hex not dec; * BUGFIX: lets not memset past the possible end of the key array....; * track active call counts (this might be slightly error prone, so we will at best call this an approximate number); * attempt to validate source IP/port during a RPTL, RPTK or RPTC event when RPTL, RPTK or RPTC is sent while not in the correct login state; * BUGFIX: dvmpatch did not properly release the network stream ID at the termination of a call, resulting in a hang up; * BUGFIX: for R05A04s 1s delayed TDU, make sure we have valid IDs before allowing a delayed TDU; * BUGFIX: add more stream ID lockout reset handling; * BUGFIX: add more stream ID lockout reset handling; * add some experiemental support to dvmpatch to support clear to enc one-way patching; * BUGFIX: only consider jitter buffer for RTP frames not carrying the end of call sequence (65535); * BUGFIX: always allow the network stack to process and pass RTP frames carrying the end of call sequence (65535); * BUGFIX: dvmbridge, ignore UDP frame timeouts when using RTP or USRP UDP streams; * add dropped call support to dvmpatch; refactor/rewrite dvmpatch encryption support; * hide debug messages; * BUGFIX: validate destination matches before handling TDU; * report ssrc for late packets; * prevent issue with 0 callDstId on dvmpatch; reject TDUs with no destination (these are invalid and can fuck off); * add separate packet dumping option to the network stack, to allow for less-verbose debugging messaging (debugigng where packet dump/trace is not required); * BUGFIX: fix condition in P25 call handler where if the TSBK was not decoded it could result in a crash; * add network debug trace around stream ID reset calls; * better handle framequeue debugging when not performing packet dumping; * implement proper packet reordering for P25 PDU data; BUGFIX: add boundary checking for P25 PDU disassembly; * BUGFIX: fix startup crash of dvmbridge when using analog audio mode; add transmit voice frame status for analog audio; correct core network issue determining length of analog audio frames; * bump copyright dates; * BUGFIX: for unauthorized, unconnected failures the NAK response was not sending the appropriate RTP sequence; * enhance REST API logging; add dev REST API commands to force send NAK packets; * add PDU block reordering support to dvmhost; * minor bugfix, correct incorrect log module for host sourced analog audio; * very experimental fix for #111, along with additional enhancements to repeat parrot traffic to all VCs of a trunk site to ensure parrot traffic repeats; * update bridge config and expressly document the need to properly configure timing parameters; * make the program status variable m_running a static s_running for dvmpatch; ensure PeerNetwork for dvmpatch uses m_packetDump instead of m_debug for packet dumping; * (we're still hunting the cause of bridge ceasing UDP socket operations after a period of time, it *seems* to occur on Win32 only though); make the program status variable m_running a static s_running for dvmbridge; ensure PeerNetwork for dvmbridge uses m_packetDump instead of m_debug for packet dumping; fix shutdown issue with local audio causing a crash for dvmbridge; refactor how RTP and uLaw frames work; remove and deprecate the no include length configuration parameter; refactor and centralize UDP audio writing functionality; add verbose log messages for if the s_running flag becomes non-true during operation; * fix concurrency lockup for DMR granting; * remove main thread sleep control this causes WinSock on Win32 to lose its mind when polled too fast; * move sample level dumping option to CLI; * always reset the call streams at the end of a call; * increase packet processing latency warning to 250ms (1/4 second); * begin adding support for P25P2 network transiting; * add some verbose debug trace for checkNetTrafficCollision(); * begin defining P25 Phase 2 primitives; * P2 abbreviated/explicit partition MCOs are essentially just TSBK opcodes; * add MFID specific partition type; * refactor MAC MCO defines; * correct comment; * make slot numerical; * whoops meant == not =; * implement RTP audio pacing by timestamp sent by source; * for maintainability split modes into separate source CPP files; BUGFIX: fix buffer overflow when copying PCM data; allow uLaw encoded RTP frames to also carry source and dest metadata; * preamble tones are not supported for UDP audio; * BUGFIX: dvmpatch do not send LDU2s until network is in non-idle state; * convert direct MMDVM TDU to a timed TDU; correct some handling of LDU1 srcId and dstId; * remove dvmmon, R05A02 will be the last version to support dvmmon, R05A04+ will not support it in favor of using sysview from the FNE for monitoring; * add more P25 Phase 2 primitives for S-OEMI sync and Hamming (8,4,4) for P2 DUID handling; * more P25 Phase 2 constants; * extreme preliminary work to handle P25 Phase 2 MAC messages (nothing uses this yet this is for future proofing); * continued work on preliminary instrumentation for Phase 2 MAC handling; * fix missing parens; * split the decodeVCH_MACPDU into two functions, one for IEMI the other for OEMI; * bugfix and correct encode/decode for RS 52,30,23, 46,26,21, 45,26,20, 44,16,29; implement catch2 cases for P25 Phase 2 RS codes; correct bad EC check for RS 24,12,13, 24,16,9 and 36,20,17 that could cause false positive failures in edge case conditions; * implement and add more EDAC verification test; * add DMR EMB and QR 16,7,6 tests; correct issue where QR 16,7,6 decode() was correcting and returning the wrong number of bits; correct issue where DMR EMB would not actually use the corrected QR 16,7,6 codeword; * finally what I was trying to get to -- add testcases for the new P2 MAC PDU logic; correct implementation problems with the P2 MAC PDU handling for OEMI and IEMI (the implementation still requires handling scrambling); * add missing Hamming FEC test cases; * finish out the base crypto test suite and include the DES algo; * (this first part is subject to Git revert if it becomes problematic, but because this is a dev branch I am gonna roll with it) revert FrameQueue timestamp list/map changes back to pre-R04J32, the original implementation was far cleaner and faster with O(1) speed for timestamp lookup vs the O(n) lookup, additionally, the newer R04J32+ implementation introduces a pointer that can be dereferenced incorrectly causing a edge case crash; fix the implementation for handling the mutex for timestamp map locking, the implementation used a incorrect instance mutex which could in high-traffic conditions cause a race condition and ultimately a crash; * for condition where a downstream peer is trying to connect to us, only validate the IP instead of IP/port; * EXPERIMENTAL (and untested): allow source ID overriding for parrot playback, this feature will rewrite the source ID to a static ID configured in the FNE configuration, for P25, NXDN and Analog this rewrite is striaghtfoward, for DMR this will likely work oddly due to the way DMR has source ID data embedded in the transited data frames; * validate the RPC data length includes the message; * BUGFIX: handle edge case where a opcode $27 is not an ISP_EMERG_ALARM_REQ but rather its a network sourced OSP_DENY_RSP; BUGFIX: handle incorrect decoding of OSP_DENY_RSP by SysView; BUGFIX: for DVRS operation properly set MFG_MOT for DENY and QUEUE outbounds; * do not use [] access for grantTimers and unitRegTimers, use find() lookups these are safer; * issue a notify CC channel release on a ChanRelease TDULC; * add explicit TDU TG release option, this option allows a CC to process incoming TDUs to determine if a channel grant should be released; add more explicit TDU process logging to the FNE, ensure TDUs being sent outside a call are being logged properly; during a TG unpermit on a non-authoritative VC, transmit a burst of TDUs on the outbound VC RF interface; add extra verbosity to the active TG logging; * add option displayModemDebugMessages to optionally disable or enable debug log messages coming *from* the modem; properly check if we're operating in DFSI or not and if we are do not attempt to set the FIFO buffer lengths or clear buffers related to DMR or NXDN; correct deletion of array types in the DFSICallData structure; correct bad length of the VHDR1 during DFSICallData initialization, the length defaulted to TIA mode which would cause a crash when trying to delete and deallocate VHDR1 when used in V.24 mode; * correct TSBK, TDULC and PDU transmission using V.24 DFSI, timing must be applied like IMBE; lengthen the high-level CC generation timer by 10ms, this has no impact on regular air and hotspot modems, but better times CC frames for V.24 DFSI modems; refactor naming to be clearer for the V.24 jitter buffer transmit types; because V.24 uses OSP_TSBK_RAW and LC_TDULC_RAW correct several off by 2 shift bugs; remove unnecessary V.24 start of stream and end of stream calls; * allow TDUs to be sent regardless of whether a V.24 call was started or not; * do work on V.24 PDU support (inbound should be working pretty much perfect now, outbound -- is still broken); * V.24 PDU work; * allow the user to adjust the frame timeout length when using frame timing at the bridge; ensure audio frames aren't greater then x2 the size of an expected audio frame; * after consideration entirely refactor how UDP frame timing is generated internally when we're using raw PCM; fix a variety of bugs related to locking and handling of the deque for UDP audio frames; better insert silence at the start and end of calls; * disable this debug trace, its too noisy; * do not consider active call or call counts for parrot; * add call collisions to the stats counters; add REST APIs to reset and zero call counters; * log non-call DMR terminators like P25; * correct endpoints for counter reset APIs; * reset active call count on a maintainence loop; * increase V.24 outbound rate for TSDUs; * properly set the V.24 Tx buffer size to the FIFO length like done for air and hotspot modems; pass whether or not the frame being written to the modem is an immediate frame; modify V.24 modem to properly maintain two independant buffers, one for background/normal priority frames, and one for immediate priority frames (this helps with CC mode delays, there are more buffering issues, likely at the modem to mitgate but thats a future project); * update gitignore to include some python stuff; add very very preliminary Python tool that helps generate dvmhost configurations; * add option to disable deny responses in conventional for radios that are braindead and interpret an OSP_DENY as a ISP_EMERG because of opcode reuse (thanks TIA-102); * delete array instead of delete; * add protections to prevent the network mutex from locking too long; add proper network watchdog to prevent unterminated calls from hanging bridge; * ICC and GRANT shouldnt NAK, as this will cause a straight disconnect/reconnect, ignore them instead; * fix issue where in-call control might start looping requests; * lets make sure RID0 is never used, treat as WUID_FNE instead; * only send ICC upstream when the ICC request originates on the local FNE; * Revise cross-compilation instructions in README Updated cross-compilation instructions for ARM 32-bit and ARM 64-bit, including examples and required packages. * relabel some source files in the FNE, FNENetwork -> TrafficNetwork, DiagNetwork -> MetadataNetwork; remove support for disabling the alternate port, this feature is mandatory now; * identify a software SDR; * add support to dvmcfggen for logging configuration; add support to dvmcfggen for user supplied answers files to automate skipping certain wizard questions; * typo; * HIGHLY_UNECESSARY_DISCLAIMER_FOR_THE_MENTAL; * the typo of that define was driving my OCD nuts; * include new statement in README.md; * enforce a users usage understanding by adding a startup configuration parameter that must be set; * these flags are root level; * make not be stupid error more clear; * copy legalese from config.yml to README.md; * policy updates * Update usage_guidelines.md * create issue & pr templates * remove dvmmon from bug & pr template * revert how timestamping is generated; * BUGFIX: fix incorrectly set length for RTP packets, we were incorrectly adding 4U bytes which does not apply to RTP frames; * enrage the entire community by cleaning up the bridge configuration to be more sane -- a lot of these parameters had nothing to do with FNE network configuration; * code cleanup (no functional change); * fix missing marker bit on RTP seq 0 to make jsb stop complaining; fix missing CTS controller teardown; * properly set the FIRST seq (when seq is reset to 0) as a marker; because JSB is *REALLY* hell bent on this, implement continuous RTP seq to make him happy; * wtf; * cleanup typo and bad spacing due to bad VSCode configuration; correct memory leak when handling buffer drop due to new stream packet wraparound; * correct issue where RTP sequences were not stepped appropriately; * ignore m_audioDetect flag for UDP audio outbound to the FNE; attempt to correct becoming stuck on incoming RTP timestamps; * whoops remove debug trace line; * remove incorrect shift; * ensure network watchdog resets all states; ignore TDU/TDULC when local audio or UDP traffic is running; * reset callStartTime for TGs who receive Non-Call TDU/Terminators; * remove duration log from Non-Call TDU; * disable the expected TS code for RTP frames; * add mutex locking when making a copy of the RadioId table to prevent thread-based race conditions; --------- Co-authored-by: Dev_Ranger <30966416+DevRanger@users.noreply.github.com>	9 months ago
W3AXL	49ff1a461b	ensure DTR/RTS logic matches between normal host startup and cal/boot modes (#112) ... Co-authored-by: W3AXL <29879554+W3AXL@users.noreply.github.com>	10 months ago
Bryan Biedenkapp	274a8f23fc	Merge R05A02 (r04k32_dev branch) into Master (#110) ... Below is a summary from the various commits, this release updates the major version number. Backward compatibility, with previous versions should not be a problem. But YMMV and its important to note that FNE backward compatibility, specifically might be problematic (FNE's should be at the same release level). This release changes significant under the hood implementations, including the lowest level of our network stack. * update version number for next dev version; * implement passing complex types as refs in lambda functions; * move KMM NoService response into its own helper function; * remove data call collisions (this code was really iffy); refactor log messages; refactor default handling of packets the FNE doesn't process; insert null bits before PDU; * set RSI data properly for outgoing KMM frame; * implement TEK encryption with an AES-256 KEK; * add support for unwrapping (decrypting) a KEK encrypted TEK; * reorganize code and organization for handling P25 OTAR KMMs for better separation and robustness (this is still a WIP and does not function!); * force hard disable KMF services if OpenSSL isn't compiled in; add instance of P25 crypto to P25OTARService; * add support to expose KMF services via DLI UDP; add plumbing to support encrypted KMM frames; * add some debug dumping; * BUGFIX: null reference when trying to perform old style lookup of timestamp when debugging is enabled; * standardize KMM logging; * begin adding support for V.24 PDU data; * fix C++ namespace shenanigans; * fix documentation error; * add support for outgoing V.24 PDU data; * minor alterations to bridge UDP audio logic; set better default for bridge udpJitter buffer; * re-engineer entirely how source untimed raw PCM frames over UDP are handled and timed; refactor how udp end of call is handled; * add support to resize the recv and send buffers on the raw UDP socket; adjust the recv buffer on bridge to use 131K system buffer for the socket (this allows us to hold ~394 frames worth of *raw* PCM + metadata in the socket's internal buffer in the system kernel; * set socket buffer sizes to larger values then the default; * annotate Linux limiting the maximum send and recv socket buffer sizes * update dvmhost submodule; * use a 2M buffer for bridge UDP audio; * display the socket buffer resize as warnings and not errors; * reduce UDP recv/send buffer size to a lower reasonable value of 512K; * update README.md; * add more class copy safety; * correct missing parens; * correct loop indexing; remove unused variable; * deprecate unused DIU flag (this isn't what this byte meant in the first place); * add call start marker to main application log; * remove SIP classes (this will be done by a different team, and done differently); begin refactoring Log and ActivityLog implementations into C++-type functions to do away with the C-style va_args functions for log message handling; correct -Wstringop-trunction warnings from BaseNetwork.cpp (we hide the warning for these because we are intentionally copying these strings without the nul terminator); * whoops accidentally blew away activity log transmission to the FNE, fix that...; * add better concurrency protection to AffiliationLookup; fix issues when getting the granted source ID from a destination ID by properly ensuring the mapping table has an entry first; add srcId to the release grant callback (callback should never call AffiliationLookup gets as it can deadlock, so adding this parameter gives the source ID to the callback so that it doesn't have to do the lookup); * remove __spinlock() from touchGrant() and isGranted(); * correct bad ordering of log message for call source switching; * remove blockTrafficTo; implement promiscuous hub mode for FNE (this mode allows the FNE to pass any and all traffic transparently); * fix typo; * for the purposes of my OCD fix incorrect one-liner Doxygen documentation (its //!< and not just //!); * better document peer ID and rid ACL list files; * add backward for stacktrace support on crash; * fix FNE compilation when SSL is not available; make Win32 builds work again; * remove double error message; * generate stacktrace file if main logger file is not initialized or unavailable; * reduce the use of unordered_map::at(); * begin relabeling peer-link to peer replication/peer replica; * implement identity with qualifier, this makes logs (and only logs) easier to trace by uniquely identifying certain peer types, a peer qualifier is simply a symbol appended to the beginning of a resolved peer identity in the FNE logs (@ = SysView, + = External/Linked FNE, % = Replica FNE); silence the Call Source Switched log messages, we will only actually print these if the call source ID changes; fix missed Peer-Link log branding/naming; * lock m_status and m_statusPVCall before trying to update data elements; * don't drop out-of-order packets (this was a bad idea, instead log an issue, the real fix for this will be some sort of RTP jitter buffer); if the current packet sequence reaches the maximum allowed, roll over to 0; * update order of operations for peer ident; * normalize log messages; * differentiate a call end collision from a call collision; * differentiate a call grant collision from a call collision; * fix missing code commenting; * initial implementation of naive round-robin HA mechanism. this mechanism allows the FNE master to communicate to a peer connected to it, to announce alternate IPs for the peer to connect to if the primary configured FNE master becomes inaccessible. the HA mechanism requires peer replication to create a loose cluster of FNEs, each FNE in the cluster is configured with the external WAN IP and port for connection to the FNE, and these IPs are then disseminated to all FNE replicas (and downstream connected peers) in the cluster automatically; * hide debug messages unless debugging is enabled; * remove old call in progress global; refactor call collisions, allow the call collision timeout be user definable (with 0 disabling call collisions, user beware); * simplify sendmmsg implementation; * BUGFIX: correct scenario where traffic from an upstream master to a downstream peer FNE would lose the RTP sequence numbering; * refactor how RTP multiplexing by stream is handled; * fix some odd behavior with very fast calls locking up grants on on DVRS channels; * refactor RTP sequence count handling; * whoops this stream ID check was intended for non-promiscuous operation only; * fix memory leak with PacketBuffer::decode(), dont use a direct heap allocated buffer, instead use a unique_ptr buffer; * simplify implementation; * rename backtrace namespace to log_stacktrace, when using simple mode the namespace conflicts with a global function backtrace(); add package element for libdw-dev:arm64 for cross compile instructions; * whoops forgot to update BridgeMain.cpp for log_stacktrace; * lock the peer list when writing traffic data to prevent peer removal during traffic operations; * utilize a shared_timed_mutex for peer list locking on the FNE, this better keeps the peer list locked during traffic operations using lock counting vs the original spinlock mechanism; * better handling locking peer connections during critical state changes; * whoops this lock should only take place for a connected peer; * reflect higher requirements for FNE; fix issue on host where sometimes a stuck network call would cause network traffic to stop incorrectly hanging on the previous TG; * change around some naming; * typo; * [EXPERIMENTAL] implement rudimentary spanning tree mechanism to prevent peer looping; refactor how FNENetwork handled peer disconnect cleanups and consolidate into a singular routine; * [EXPERIMENTAL] enhance detection for case where FNE A and FNE B are cross-peered to each other (dont do this); * add REST API endpoint to fetch the master tree; * make sure master peer ID 0 doesnt ever happen; * log condition where masterPeerId is 0 * SysView peers announce themselves as an external FNE, but we do not consider them FNE peer links; * SysView masquarades its masterPeerId as itself; * instead of killing a peer connection instantly on a duplicate conn drop, increase retry time to 30 minutes and allow up to 3 duplicate conn failures before killing; * better handle duplicate connection disconnect NAKs; add REST API on FNE to force reset a upstream peer connection; * fix issue with purely ACL virtual FNEs not being able to replicate configuration further down the master tree; * deprecate the "external FNE" terming for upstream FNE connections and instead call them "neighbor FNE"; * more code cleanup, simplify naming; * cleanup code; refactor log messages from the FNE to better categorize them; correct issue where a peer reconnecting may trip tree duplicate conn checking; * enhance STP peer reconnect logic to allow peers announcing the same peerId and masterId to reconnect between spanning tree updates; * update log colorizer to match new logging categories in the FNE; * update log colorizer to match new logging categories in the FNE (round 2); * update log colorizer to match new logging categories in the FNE (round 3); * update log colorizer to match new logging categories in the FNE (round 4); * bump major version numbering in a preliminary fashion, at least until group talks about it are done (so this could be permenant); * fix messaging for fast peer reconnect (it was misleading as a RPTC NAK which it isnt); * allow reparenting of a STP node if it moves from one tree node to another; * implement tree cleanups if the downstream announcement removes child leaves or reports no children at all; * relabel MasterTree to SpanningTree proper; * whoops errant i++; * fix up some concurrency problems when dealing with parrot transmissions, due to migration of parrot playback into its own thread; * make sure to share lock peers while processing in maintainence loop; * add mutex locking around spanning tree updates; * simplify log levels, deprecate LogMessage log level in favor of just using LogInfoEx (message and info logs are basically the same thing); * better classify log messages; * simplify FNE configuration for peers, make identity a global applying to all peer connections, remove bogus frequency data; * better report global identity; * allow overriding the global identity for upstream connections; implement use of identity for spanning tree; * update config examples to reflect new log levels; * add back peer "name" field, this is strictly informational to make the config file easier on the brain; * Win32: bump version number for file resource metadata; * add documentation for packet payloads for: writeLogin, writeAuthorisation, writeConfig and writePing; * add some more packet payload documentation; * continue packet payload documenting; * minor comment alteration; * correct DMR data handling; refactor FNE DMR data calls to be structured more akin to P25 data calls; * use WASAPI by default on Windows; * better handle out of order blocks for PDUs; reduce packet retry to 2; correct handling ack response packets; * refactor how buffered UDP datagram queuing is performed; * relabel static class variables to use s_ and globals to use g_ * enhance colorize-host.sh to terminate color properly; * enhance colorize-host.sh to terminate color properly; * RTS PTT will only assert when audio is present. Add holdoff timer before removing RTS. (#103) * add option to enable/disable upstream call start/end event logging; * more work towards a working DLD-type OTAR service for P25; * fix typos; * Add carrier operated relay support to dvmbridge. (#104) * refactor KMMFrame to properly support message number and add framework to support MAC; refactor all derived KMM classes to properly determine KMM frame length and body offsets; change collision in class naming for KeyItem in the CryptoContainer and KeyItem in the p25::kmm namespace; cleanup magic numbers; add initial code for transmitting a rekey command; add KEK crypto wrapping testcase; * begin work on generating KMM CMAC for message authentication; * document sections of doc where a test originates test data; * begin work on generating CBC-MAC for message authentication; * minor cleanups; * test should be using generated key; * CBC-MAC now works properly; CMAC MAC generation works (just gotta fix CMAC MAC key generation); * complete CBC-MAC implemenation, at least for the provided samples from TIA-102.AACA-C this is now passing for the rekey example provided in section 14.3.4; anything compiling against libcommon.a while OpenSSL is enabled will require OpenSSL -lssl; fix delete vs free in various tests; * typo * implement OFB data encryption per TIA-102.AAAD-B; test proper final encrypted output for CBC-MAC check; implement use of cryptAES_PDU() in P25OTARService; * add experimental DES crypto; * move Git hash global defines out of Defines.h and into GitHash.h; * fix copyrights; * cleanup file headers containing lingering old comments; update fw/modem submodule; update fw/hotspot submodule; * remove debug trace not needed anymore; * add support for PDU auxiliary ES headers; * complete refactor of how P25 PDUs are assembled from user data and disassembled to user data; various corrections for data path nullptr reference issues; implement several tests for testing the P25 PDU assembler; * remove left over debug code; fix AMBT CRC-32 calculation error, AMBTs calculate the CRC-32 for the PDU themselves the Assembler does not need to do it; correct default value of p25TxNAC; * add some better error handling for NetRPC; * validate LC_CALL_TERM peer ID before allowing them to repeat, this fixes an issue where an errant peer on the network could spam LC_CALL_TERM to cause trunked nodes to terminate/kill a call in progress; * fix issue with dvmpatch not properly evaluating the destination TEK; * add support to patch to *disable* enc processing and allow frames to pass transparently; implement DES for bridge and patch; * reorganize application files, I've wanted to do this for a while JSON isn't really a network service, and REST API while it is a network service, I want to be in a separate namespace; * add multi-block PDU for V.24; * add some guard rails on PDU reception so we dont overflow ourselves; * it seems as if PDU_(UN)CONF_END is variable length, it will always be at least 1 block containing the last PDU block, and PDU_(UN)CONF_BLOCK_X is always 4 blocks; * cleanup dataBlocks pointer array; * preliminary code for sending PDU frames >3 blocks over V.24; * use unsigned numbers for these loops the values should never be negative anyway; * properly encode sequence opcode; * whoops missing parens; * initialize these unordered_maps to allow the max connection cap count number of entries (before it was zero and could result in many reallocations of these maps); * well that was a little naive of me, I didnt multiply the count by the size of the stored elements...; * reverse course lets do this more intelligently and implement the passthru for the reserve() function on the concurrency classes; * add STP check to see if a downstream leaf is blown itself away on the tree; * implement DMR data PDU assembler based on the P25 PDU assembler; * expand on the files related to an FNE instance for clarity; * add preliminary support for enabling peers with call priority, this will give those peers the ability to override any current call in progress; * implement a more solidified peer call priority mechanism using ICC; * bump WebSocketPP version to conform to new CMake minimum version; (#108) * fix typo; * add some documentation around newer protocol additions; * whoops typo; * start adding foundational work for future P25 Phase 2; * decomplicate foundational changes to RS coders for Phase 2; * added new peer config options to FNE REST api * add some locks around pkt maps in DiagNetwork; * better document the master peerId importance; * ensure buffers are set to nullptr after deleting to prevent double frees; * skip trying to transmit any buffer that is null; * add thread safties to SpanningTree; * correct bad memory allocation; * revert previous change to add locking to SpanningTree (this is handled externally in FNENetwork and DiagNetwork); fix issue in FNENetwork which could cause a STP deadlock; fix issue incorrectly labeling a peer as allowing call priority in the log when infact the peer was not configured that way; * make sure during resetPeer() we lock the connection; * readd SpanningTree internal locking mechanisms; * make sure pointers are set to null after delete; * ensure when a FNE loses all its downstream leaves, that it will properly notify upstream FNEs; fix issue where dangling tree nodes were being incorrectly left in the flat peer node list for the spanning tree; * fix defaultNetIdleTalkgroup being treated as a hex value instead of dec; enhance P25 defaultNetIdleTalkgroup slightly to better pass traffic after RF traffic; enhance P25 defaultNetIdleTalkgroup to pass traffic if there are affiliations to the group; * how about we dont blatently leave debug messages enabled...; * BUGFIX: hopefully correct crash condition when trying to erase child nodes; * followup for last commit, simplify implementation; * this will be unpopular, but I am deprecating support for cross-compiling for armhf/legacy RPi, maintaining this is already causing problems with OpenSSL and will ultimately handcuff us in the future if we upgrade C++ versions because the legacy toolchain uses GCC 4.9; * remove deprecated patch; * update README.md; * correct some cross-compile shennigans for OpenSSL; * correct some cross-compile shennigans for aarch64; * BUGFIX: fix issue where the host would incorrectly reset the voice stream ID during a call; * match code change from previous bugfix to DMR and NXDN; * dont use the deprecated OpenSSL1.1 functions, use portable AESCrypto functions instead for low-level AES crypto; * disable STP reparenting when deserializing children of a tree; * BUGFIX: fix tged and peered in the case where an empty file or file with no entries is provided for editing; * BUGFIX: add some validation checks around deserialization reparenting; * implement P25P2 reed-solomon codes for future use; * BUGFIX: correct bad check for parrot frames end of call; --------- Co-authored-by: Lorenzo L. Romero <lorenzolrom@gmail.com> Co-authored-by: Natalie <jelimoore@gmail.com> Co-authored-by: W3AXL <29879554+W3AXL@users.noreply.github.com>	11 months ago
Bryan Biedenkapp	7c2bfb3914	R04J32 Merge to Master (#95) ... * implement user-defined adj site mapping to allow fine grained control over neighboring site announcements; * BUGFIX: patch issue where the FNE would not maintain the source peer ID for certain routing scenarios; * fix typo in original implementation, peerId should be the peerId and ssrc should become the originating peer ID; remove peer check throwing a warning; * log ssrc for various data points (the ssrc is the RTP originating sync source which would be the peer ID of the origination of a RTP packet); * bump build number; * add peer ID masking support (this is useful for FNEs that are "public" links where internal peer IDs don't need to be retained); more work for maintaining the originating stream sync source; refactor how the FNE handles peer network connections; * throw a warning if the user configured more then 8 upstream peer connections (more than this number can cause performance problems); * make non-peer-link peer ID masking optional; * add checking for traffic repeat to verify we are not trying to send traffic back to the source; * add checking for traffic repeat to verify we are not trying to send traffic back to an external source; * convert peer network protocol packet processing to a threaded model; * update copyright dates; * implement REST API for manipulating adjacent site map entries; * fix missed REST API initializer, need to pass the adj site map; * add DFSI/V.24 full duplex option (this allows dvmhost to repeat incoming frames back out); * Fix incorrect check of DFSI and FSC operating modes that resulted in a segfault (#92) Co-authored-by: faulty <faulty@evilcomputing.net> * document p25CorrCount in depth more; set the default p25CorrCount to 2; * properly call PacketBuffers clear() before leaving a scope to ensure contained buffers are deleted (otherwise we'll leak memory); * skip the first 6 bits of the TIA-102 DFSI VHDR (this is very strange, and is probably not kosher); * transparently pass voice frames with FID $20 (assumed to be Kenwood) when the PF flag is set (this allows Kenwoods flavor of encryption to pass); * relabel FID_DMRA to FID_MOT (this feature ID is assigned to Motorola); handle more conditions for FID $20 (Kenwood) on voice frames; * remove filterHeaders (this is deprecated, HDUs aren't sent over the network); perform frame length validation for P25 network frames (unlike DMR and NXDN P25 frames are variable length, requring length validation to prevent buffer under or overflows); fix issue with addr variable not being freed in InfluxDB handler; * [EXPERIMENTAL] add very initial support for dvmpatch to talk to a MMDVM P25 Gateway in P25 mode; * fixup dvmpatch support for talking to MMDVM P25 Gateway; * fix up handling of MMDVM call termination; * more work on better signalling end of call for MMDVM P25 gateway patches; better log traffic from MMDVM; * correct some incorrect timing; * add support for properly authorizing a peer to send Inhibit/Uninhibit commands; add some NXDN constants for remote control; * add columns to main peered peer list to represent if a peer is allowed to inhibit; * fixup informational logging; * add auto generation of a peer password; slightly increase the dialog size of the peer edit window; * fix TIA VHDR incorrect length; correct gatekeep accidentally setting TIA StartOfStream to 4 bytes instead of 3, because you know the TIA spec is incomprehensible with its bit alignment; * add some permanent log trace for DFSI over UDP tracing; correct startOfStreamTIA incorrectly sending a LDU1 NID; correct some offsets; * make sure we send out heartbeats *before* the heartbeat time; * send Start of Stream with voice data; * free memory in error case; remove unused variables; * correct some buffer allocations; ensure the AES class is deleted after use; * implement support for legacy radio affiliation (like P25) for DMR and NXDN (this fixes an issue where conventional DMR and NXDN systems would be unable to transmit onto affiliation only TGs); correct bad CSBK decoding in the FNE (forgot to check for dataSync); * code cleanup to correct compiler warnings; * add string file meta data info to Win32 EXEs; add icons to Win32 EXEs; * migrate and condense analog audio helper and G.711 codec routines into a common class called AnalogAudio for easy reuse; stub network protocol entries to eventually handle analog audio; * add support in the DVM network protocol for analog FM audio traffic; add support to the DVM FNE to properly switch analog FM audio traffic; bump revision from H to J; * add analog enable flag to fne config YAML; * prelim work to make dvmbridge pass analog audio directly into FNE network; * handle locking inside deleteBuffers(); * refactor FrameQueue mutex handling; don't start and stop the thread pool when the peer network opens/closes; * use unique_ptr for compression class instead of passing raw pointers; * refactor FrameQueue from using a std::unordered_map to a simple fixed array that is directly controlled; * refactor compression slightly; * better handle multi-threaded problems; * use a std::vector instead of a classical C allocated array for timestamp list; add error/logic checking for V.24/DFSI where the VHDR may send a TGID0 during a call causing a call rejection; * make timestamps vector static; * dump out of VHDR processing if call is in progress; * bump tarball version build stamp; * add option to forcibly allow TGID0 for badly behaved systems this will cause dvmhost to accept the call and not drop it, with the caveat that the TGID will be rewritten to TGID1, the network stack simply cannot service a TGID of 0 and as such we must rewrite the TGID a TIA-102 valid TGID; * simplify late entry handling where V.24 may send a dstId of 0; * whoops correct my own stupidity; * whoops correct my own stupidity (again); * add opcodes for Harris User Alias and GPS; add opcode for Motorola GPS on PTT; decode Harris User Alias in LDU LCs; * attempt at decoding the LDU RS values for both V.24 and TIA DFSI, this does not abort decoding in the ModemV24 right now and will simply log an error if the RS values for the LDU1 or LDU2 are to badly invalid; * alter field data based on recent testing; enhance and update debug log entries; add debug logging for Voice 1 and 10 start of voice frames for V.24; * add missing tag information for V.24 PDU; * fix incorrect handling of RFSS ID in host setup; * experimental fix to ignore wildly illegal TGIDs in the HDU; * simplify the V.24 HDU TGID logic, entirely ignore the TGID presented in the HDU and just wait for the LDU1 to set the rfLastDstId; * add grantDemand documentation to dvmpatch; * [EXPERIMENTAL] add experimental support for cross-encrypting patched P25 traffic; * reset call algo when a P25 call ends; * properly handle algo ID's for source and destinations; * missed reverseCrypto check; * [EXPERIMENTAL] for some configurations (**you know who you are god damn it**), allow dvmhost to be configured to "idle" on a non-zero TG when the RF talkgroup hangtimer stops, this effectively disables the promiscuous network Rx of the host for certain conventional operations, and requires RF-based TG steering to change the active talkgroup; cleanup the macros for DMR and NXDN that perform various repeated checks; * major refactor for V.24 support; * hide RSSI2 ICW errors; report on any voice frames reporting more then 0 errors; * properly sent VHDR1 and VHDR2 with proper opcodes (whoops); * add experimental support for TDULC over V.24; add some documentation for the V.24 and TIA voice header byte layout; * experimental support for transporting V.24 over IP; * initial super frame should start at 1; make sure to use proper constants instead of magic numbers; * reorganize code slightly; * cleanup format for, and make slightly more precise trace and error dumping log messaging; * add option for the FNE to directly log traffic denials to the system log; * centralize string for illegal RID access; * log total voice frame errors for TIA/DFSI mode; * minor PDU refactoring on when network PDU data is sent; correct issue with accepting a conventional data reg; * disable allowExplicitSourceId by default (not all subscribers support this); correct some bad handling of LC data; * log sysId and netId info for call start and end on the FNE log; * add setting netId and sysId to dvmbridge and dvmpatch for future use; * reorganize and expose decodeLC/encodeLC from LC; correct typo in TDULC headers; add explicit source id TDULC; add LC_GROUP explicit ID flag; * use shorthand macros where able; * enable or disable explicit source ID directly, don't require control to be enabled (important for trunked VC); * minor correction to handling explicit source ID; * properly set dummy SiteData for dvmbridge and dvmpatch (necessary to set the WACN and SysId); * handle condition defaulting WACN and SysId if network data for WACN and SysId is 0; * ensure unencrypted parameters if encryption is disabled; * copy dvmbridge change to dvmpatch for: ensure unencrypted parameters if encryption is disabled; * ignore extra ICW opcodes; * use raw LDU values from the call data instead of processed values if the MFId on the LC isn't TIA standard; * collate bit errors reported from V.24 or TIA; * typos; * allow a group affiliation on Conv+ (DVRS) to terminate a running TG hang timer; * add missing stop to the timer; * add WACN/SysId logging to SysView; * add global to disable transmitting logging to the network (this is used when the UDP socket fails to send data to prevent a crash condition talking to a null socket); unify errno to string message processing for clearity in logs; refactor network reconnect and retry logic to better handle a full connection reset cycle (this is useful for conditions where the network loss isn't transient and is something like a Ethernet or WiFi link drop causing the entire interface to become down temporarily); * remove unnecessary debug logging; * allow encoding user alias at the LC level; * see if we can calculate error percentages; * get rid of magic numbers for properly defined constants; * implement proper support for P25 LDU1 and LDU2 to pass call control bytes; allow a end-point to signal that a call is handing over/switching over from one stream ID/source ID to another; implement support on dvmbridge to properly handle stream/source ID switch over; * log call source switch over events; * whoops this should be a OR equals not equals; * correct issue where network frames would be ignored during RF calls causing buffer overflows; * cleanup and remove unnecessary and confusing C compiler macros; * cleanup and unify design a bit, for P25 move traffic collision checking into separate function calls; * add colorize-host.sh helper tool; update colorize-fne.sh tool for DMR and NXDN; * store status data for a private call; * during unit registration store the originating source peer ID for a given unit registration; use unit registration source ID to select the destination peer to send private call data to; * initial experimental implementation of private call routing: this introduces a new configuration flag "restrictPrivateCallToRegOnly", when set, this flag will influence how private calls are routed through the system private calls using restrictPrivateCallToRegOnly require both the source and destination subscribers to be unit registered with the system, when a private call occurs the system will utilize the source peer ID to restrict repeating the private call traffic only to the 2 peers involved in the private call FNEs will *always* receive private call traffic * rework stream validation for U2U calls slightly for P25; ensure P25 always sends the U2U control byte for private calls; * enhance handling of NXDN LC on the FNE; fix issue where dvmhost would not process a network RCCH burst; * refactor NXDN CC handling a bit; * add support to translate a raw DENY/QUEUE/CAUSE/REASON value for DMR/P25/NXDN into a human readable string; * whoops hastily missed std::string -> c_str conversion; * ensure SSRC is maintained for unit registration announcements; * more implementation for private call routing; * add missing clear flag to DMR payload activate RPC; * correct crash for fsc set to true when not using DFSI TIA-102/UDP; * add support to disable a failed CRC-32 for P25 PDU data; ignore CRC-32 errors for AMBT PDUs; * Add support in dvmbridge for a serial PTT activation switch. RTS is asserted on the serial port defined in bridge-config.yml for the duration of audio received, then is removed. (#102) * P25 PDU packet handling refactor (for future use); * remove test code; * fix issue where PDU RSPs weren't being sent to the FNE; correct timing around ARP and packet retry when subscriber is not ready; * add some more verbose logging; * experimental changes to PDU data handling; * correct CSV parsing for iden, peer list and RID lookup tables (we would skip parameters if they were empty); make the FNE P25 packet data handler operate in the TIA-102 asymmetric addressing mode (as would be required with FNE configurations); --------- Co-authored-by: Jamie <25770089+faultywarrior@users.noreply.github.com> Co-authored-by: faulty <faulty@evilcomputing.net> Co-authored-by: Lorenzo L. Romero <lorenzolrom@gmail.com>	1 year ago
Bryan Biedenkapp	fcdec00f32	BUGFIX: correct issue where a completely incorrectly formatted identity table missing the appropriate number of entities would cause a startup crash; minor formatting cleanups for readability;	1 year ago
Bryan Biedenkapp	738ee918d1	R04H31 Merge to Master (#89) ... * simplify and refactor peer-link packet handling; * bump version number to match dev branch; alter logging for peer-link data; * warn on packets generated over 4K; * don't clear map directly, use clear() helper; * use parens vs braces for initializers; correct case fall-thru on SysView; * for clarity relabel RPT_CLOSING and MST_CLOSING to RPT_DISC and MST_DISC; * discontinue support for TRANSFER command on the traffic port entirely; hide useAlternatePortForDiagnostics and allowActivityTransfer options in FNE configuration; add stern warnings and alerts if the FNE disabled either useAlternatePortForDiagnostics or allowActivityTransfer (these are really critical operations); alert in peer log if the master does not report support for the alternate port for diagnostics; * make oversized packet warning clearer (this warning is not a end-user warning, the user can't do antyhing about this this is a developer BUGBUG warning); * apply recv timeout on InfluxDB operations (this hopefully will prevent stuck queries if the InfluxDB server dies while query is in progress); * prevent InfluxDB query from becoming stuck in a tight infinite loop within select(); * don't bother handling responses from the influx query, remove fluxQL (we don't make queries with this library); * cleanup unused vars; * generate an error log if a fluxql worker fails to write data to influx; * implement support to save/retain configuration pushed from peer link connections; * fix memory leak caused by not deleting unused packetbuffer; * refactor ACL updates; * refactor log messages for channels to be <channel ID>-<channel No> in most cases where applicable; * add more <channel ID>-<channel No> changes; * whoops wrong variable type; * correct fatal bug where update lookup timer was never started; * check the RF state at the bottom of a talkgroup hang expiration timer for RF talkgroup activity, if the state is not listening, handle the end of call as if it was a frame loss; * fix potential memory leak in PacketBuffer fragmenter, the encoder was not deleting the storage buffer before end of scope; * ensure decompressed buffer is deleted; * ensure decompressed buffer is deleted in error conditions too; * add support to disable adjacent site broadcast; * lay the groundwork for explicit channel configuration; * [EXPERIMENTAL] implement experimental support for a CC to define explicit channel identity information for voice channels (NOTE: this does *NOT* implement support for hotspots to operate in explicit channel mode!); * don't use the concurrent::unordered_map for this (this gives me great pause for a problem...); * add some extra debug trace; * add some extra debug trace; * display MI data during calibration; update modem submodule; * this is ugly, and I hate it, but to fix very strange WIN32 deadlock situations, we'll selectively use our concurrent unordered_map on non-WIN32, and on WIN32 use the std unordered_map + a local mutex; * update modem submodule; change P25 Voice MI logging messages from DEBUG to MESSAGE; fix issue in HostSetup/HostCal where PDU header decoding was broken; * only dump MI data if the call is encrypted (don't bother displaying zeros for a clear call); * whoops check right variable; * cleanup and clarify MI logging; * cleanup and clarify MI logging (again LDU2 was out of log sequence); * hate me later, dont use a colon as the string separator for the MI bytes, use equals; * log DMR PI MI dta; * refactor debug messaging for HDU; * reorganize core meta-programming macros, move class property macros into ClassProperties.h, move bit manipulation into BitManipulation.h, move VLA into VariableLengthArray.h; rename meta-programming macros for clarity most of these macros declare some sort of variable and/or functions, as such I'm dropping the notation for __BLAHBLAH and using DECLARE_BLAHBLAH instead; better document class property meta-programming macros; refactor __GET/__SET_UINT16 into __GET/__SET_UINT24 these macros set 24-bit values from and to buffers, this will reduce programming confusion; * add property documentation for some KMM classes; * correct file copyright headers; * change commenting to reflect section more clearly; * use proper GPL license for file; * detect arch and properly update internal variable; * handle instance where the system does not return a valid processor type for arch detection; * display target arch along side host arch if we're cross compiling; * strip newlines; * fix missing parens around response encoding;	1 year ago
Bryan Biedenkapp	af2e7c9c8d	R04H30 Merge to Master (#88) ... * implement support for LC_GROUP_UPDT (this includes the RPC between the CC and VC's to announce active TG lists); completely and entirely refactor how packet handling threads are done, use a new method introduced for this, thread pool resources. this will ultimately be more resource consuming depending on configuration as the worker threads for packet processing stay alive along side the main process. this should be more performant (because we're not constantly creating and destroying threads) and will prevent error conditions that can cause an extreme number of threads to spawn; * reduce influx and V.24 thread pool sizes; add calculation of how long between when a packet was Rx to when it began proper processing; * don't use void* for the task routines; * rename ThreadPoolCallback to ThreadPoolTask; add some checking around task validness; correct a valgrind issue with RawFrameQueue() write not deleting the buffer before return; * valgrind cleanups; * gate active TG from CC to VC updates at 5s (prevent API spam); * update package version; * don't attempt to send active TG updates to 0.0.0.0; * make notification of active TGs CC -> VC optional; * correct ThreadPool issue on Win32; split UDP PCM audio processing into its own thread; implement user control of inter-audio frame delay and jitter buffer (if using inter-audio frame delay); * lock queue; * add custom classes for STL containers that support mutex locking for thread safe operation; modify ChannelLookup and AffiliationLookup to use concurrent containers; modify FNE to use concurrent containers for internal lists; * remove mutex used for protecting udp packet deque; * more concurrency solidification; * further concurrency class usage; bump version number to R04H30; * add --boot commandline argument to reboot modem into bootloader without any interactive interface; * cleanup program -h usage display; * incorrect opcode define; * update README.md; * correct incorrect string format for non-useAlternatePortForDiagnostics; * update README.md; * simplify influxdb worker task function; * refactor PL_ACT_PEER_LIST opcode entirely, use zlib and compress list sent and properly block data sent; * don't waste cycles on building the peer list repeatedly, build it once for the cycle; * deduplicate compress/decompress code into a static C++ class; * stylecop file formatting; * add table locking and remove at find; * deduplicate implementation; * cleanup unused label; change FrameQueue's unordered_map to a concurrent one; fix incorrect setting of __lock() for concurrent containers; * add more timestream map locking; * fix incorrect behavior when deriving initial timestamp for a call stream; correct incorrect behavior inserting new stream in to timestamps table for frame queue; fix incorrect behavior deriving timestamp for bridge RTP; * fix issue with naive approach to handling PL_ACT_PEER_LIST data fragementation; * disable accidental debug code; * instead of asserting, throw a log error message and discard network packet; * lock the talkgroup tables when a find is in progress (this will prevent some weird concurrency behaviors because talkgroup rules does not use the concurrent vector); * if filter headers or terminators is enabled, and the target peer is in the exclusion list, do not send headers or terminators; * exclusion check should happen before the rewrite check; * implement dvmpatch, this is a new utility that allows simple TG to TG patching; * ensure FNE downstream peers that report as peer link have implicit always rules applied to them (i.e. they will *always* receive *all* traffic); correct order of operations when deleting a peer entry (delete connection *AFTER* removing from peers table); * document concern over possible null ref concurrency issue; * don't be overly aggressive with FNE process niceness, nice of -10 is more then sufficient;	2 years ago
Bryan Biedenkapp	32494fb19d	change RF power level of hotspots to 95 instead of 100 to prevent a condition where spurious emissions may be generated; update README.md to include methods to calibrate a hotspot if a service monitor is available;	2 years ago
Bryan Biedenkapp	ef4e3104ac	you know what helps? if you actually set the channel number;	2 years ago
Bryan Biedenkapp	28153fbab9	fix cross-thread ARM shennigans (this should resolve #78);	2 years ago
Bryan Biedenkapp	1b4ea4d8c7	refactor status bit handling slightly;	2 years ago
Bryan Biedenkapp	2e2e8bf957	fix all instances of (uint32_t) for calcTxOffset;	2 years ago
Bryan Biedenkapp	fe66aff8ab	correct long lingering frequency rounding error when calculating freq in MHz;	2 years ago
Bryan Biedenkapp	ae39ae816a	adjust color palette for TUI; add alpha version for the TUI talkgroup rules file editor;	2 years ago
Bryan Biedenkapp	65fa91b7cb	[THIS MAY BREAK THINGS, BE AWARE] Merge Win32 support and C++ dvmbridge into master. (#67) ... * experimental support for WIN32 compilation (tested only with VS2022); * fix up incorrect return; * make debug compile verbose; ensure ws2_32.lib is included; * initial commit of vocoder library to main source tree; initial commit of C++ port of bridge (non-functional); * calculate the length of the preamble tone and properly write it; * implement MDC callback to detect MDC unit ID; implement UDP audio reception; implement local audio reception; * very alpha working verison; * correct buffer size; * correct issue with enabled Tx mode not properly being set on the network; correct issue with DMR not setting the appropriate frame length after packing AMBE bits; correct false call starts by faking the txStreamId (this isn't correct but works for now); fix issue with P25 non-GROUP LCOs causing audio breaks; correct bad logic for call lockup thread; * cleanup and properly align log messages; * implement missing Rx/Tx manual gain control; fix missing srcId overrides; * eliminate the __ALLOC_VLA macro (this really only works on VS2022, GCC explodes in a fit of horror); * fix missing comment; * fix bad CFLAGS; correct missing dl library when linking dvmbridge on linux; fix various typos and bad variable types in bridge; * fix some bad memset calls not taking array size into account; fix some missing parens; * implement missing cal options; * don't start the timer till the first LDU1 for cal mode; * I'm just full of bad mistakes today; * ignore logging excessive sync frame errors; * update modem submodule; * Add DVM-V24 Firmware Submodule And Enable Building It (#66) * Added v24 firmware as a submodule * Added v24 firmware compilation * V24 doesn't have make clean, so remove it from cmake * Added V24 firmware to tarballs * Fix my 1D107 error... --------- Co-authored-by: faulty <faulty@evilcomputing.net> * correct compilation flags for Win32; add support to dump sample levels on bridge (this is useful for tuning the vox level); add various fixes for local audio flag; add various fixes for not starting loops until completely running; ensure audio input/output device settings are output to the log; fix issue with Rx and Tx gain controls not being applied appropriately; * fix compilation issues for RPI_ARM; ensure math constants are defined; remove unused command line arguments; --------- Co-authored-by: Jim <25770089+faultywarrior@users.noreply.github.com> Co-authored-by: faulty <faulty@evilcomputing.net>	2 years ago
Bryan Biedenkapp	26457d7d85	implement support to software reboot into the system bootloader for flashing (using stm32flash); update modem and hotspot submodules;	2 years ago
Bryan Biedenkapp	cda8427c6c	don't start the timer till the first LDU1 for cal mode;	2 years ago
Bryan Biedenkapp	da2f6130a8	[NOTICE: this commit is *EXPERIMENTAL* and implements *very* early data support, it is expected to be buggy, incomplete or broken] implement VTUN interface (fne0) on the dvmfne (this requires dvmfne to be run as root on Linux system [untested on non-Linux] to create the tun interface); implement VIFace random MAC generation; implement P25 TIA-102.BAEB SCEP ARP (SCEP is always used for conventional data) on dvmfne; refactor P25 packet status bits (this may break things else where, beware!); properly implement P25 status bit flipping on inbound channel activity; implement functionality in the P25 and DMR DataHeader classes to calculate the proper raw PDU frame length; implement dvmfne ARP table creation and maintainence (this is buggy and requires more implementation and test); implement dvmfne VTUN -> PDU and PDU -> VTUN IP traffic forwarding (this is incomplete and requires more implementation and test); correct several issues on dvmhost p25::packet::Data with overlapped buffers;	2 years ago
Bryan Biedenkapp	882c2564ca	correct double spaced formatting of CPP files in common; convert dvmhost to Doxygen documentation format; convert dvmfne to Doxygen documentation format; convert dvmcmd to Doxygen documentation format;	2 years ago
Bryan Biedenkapp	35c07715bc	don't use HUGE as a SiteModel because certain versions of GCC are awful;	2 years ago
Bryan Biedenkapp	cb0500b2e7	treewide: refactor constant variables into proper enumerations where appropriate (this applies to DMR, P25 and NXDN, and may introduce gremlins);	2 years ago
Bryan Biedenkapp	24082b9652	P25 data (#52) ... NOTE: this does not indicate full P25 data support as working yet; this commit refactors a lot of the baseline PDU handling code, and better handles both confirmed and unconfirmed PDU data blocks, there is still some more work to do, I just didn't want p25_data to start getting overly out of sync with master. * rework modem protocol to support variable length frames up to 64k; refactor P25 PDU handling to support TIA-102 minimum 512 byte frames; * fix PDU length its 512 bytes not 500; ignore instead of report invalid modem frames; add double length checking to HostSetup; * add support to send P25 long frames to the modem; * adjust FIFO size to 522 (fits a single PDU frame); remove extra debug displays; match fixed FIFO sizes between modem and host; * correct weird inverted CRC-9 that sometimes happen; * ensure netPDU is reset at the beginning of a network data call; fix lengh used by data block dumping; * fix some buffer lengths;	3 years ago
Bryan Biedenkapp	f8aa80a63f	continued work on P25 PDU handling (this has a long way to go...);	3 years ago
Bryan Biedenkapp	86d736b0bb	revert change from 0ea6ee0 that changed the RingBuffer const char* to std::string to correct issue where the RingBuffer name is being copied incorrectly; change method for copying name std::string passed to AffiliationLookup; remove accidentally committed debug code;	3 years ago
Bryan Biedenkapp	f000639c7f	ensure m_forceHotspot is initialized to a sane default;	3 years ago
Bryan Biedenkapp	9dadc06130	I was asked for ASCII art, and I deliver, thus, ASCII art;	3 years ago
Bryan Biedenkapp	5a5104ba2f	treewide: perform general refactor of file headers, I've modified the file headers to follow rules from: https://www.kernel.org/doc/html/next/process/license-rules.html, this trims/compresses the file header section a bit reducing the number of lines at the top of every file quite a bit, going forward all new files should follow this new format (if its good enough for the Linux kernel, its good enough for me);	3 years ago
Bryan Biedenkapp	bad8129d7d	file header updates;	3 years ago
Bryan Biedenkapp	d9e2458722	implement support for low-level TCP sockets; deprecate and remove new_unique macro and use appropriate std::make_unique; apply various clang linting;	3 years ago
Bryan Biedenkapp	cf0d8bb430	port RTP header changes from fnecore for better handling the RTP timestamp; relabel conference FNE to converged FNE; refactor version slightly so that the running DVM host version is properly reported to the FNE and when saved to modem configuration area;	3 years ago
Bryan Biedenkapp	506c59c52b	fix range for softpots in TUI to be 0 thru 255;	3 years ago
Bryan Biedenkapp	2d809e0a61	complete the job Faulty started and remove remnants of deprecated ENABLE_DMR, ENABLE_P25 and ENABLE_NXDN compiler options;	3 years ago
Bryan Biedenkapp	ddfb6aaedc	properly handle swapping between Rx and Tx antennas for BER operations on hotspots (this requires updated firmware);	3 years ago
Bryan Biedenkapp	97af96a6ad	be verbose in cmake output when enabling various debugs; correct issue with startup messages;	3 years ago
Bryan Biedenkapp	1379482fcc	reorganize entire codebase structure; move common classes, files into a compile time static library; reorganize CMake files to make use of common static library to reduce compile time;	3 years ago
Bryan Biedenkapp	fe3d0b7e35	add command line argument to override enable modem debug (faster then continually modifying the YAML config file);	3 years ago
Bryan Biedenkapp	243ad6ce0c	increase NXDN voice call verbosity; correct incorrectly calculated data offset for NXDN voice data;	3 years ago
Bryan Biedenkapp	12edb913b4	correct issue with AMBEFEC not properly calculating FEC for NXDN; correct buffer offset issue for P25 and NXDN;	3 years ago
Bryan Biedenkapp	4e758bba91	Merge branch 'master' into dev_355	3 years ago
Bryan Biedenkapp	388e23c67c	fix issue where the gain mode wasn't being appropriately selected;	3 years ago
Bryan Biedenkapp	c67314e49d	rename getData to get in RingBuffer; ensure the RingBuffer contains a data to the given length before attempting to return that data;	3 years ago
Bryan Biedenkapp	1eb24608a0	add tiny AES library (for future LLA use);	3 years ago
Bryan Biedenkapp	31f6b072fb	correct some problems reported for --cal not writing the modem tuning partition; correct some issues with Rx tuning not properly applying during calibration procedures;	3 years ago
Bryan Biedenkapp	d45af90c07	initial bare-bones implementation of dvmmon; enhance REST API to return more detailed status information (for use by dvmmon); cleanup file code style;	3 years ago
Bryan Biedenkapp	8f7763cac4	initial implementation for a passive monitor tool; correct bad code style; implement feedback of last destination ID; enhance status REST API;	3 years ago
Bryan Biedenkapp	77ff4a551b	typo cleanup;	3 years ago
Bryan Biedenkapp	82155893aa	fix issue where the setup event loop may try to talk to a unconnected modem;	3 years ago
Bryan Biedenkapp	2b6b0c574d	merge 3.5-dev into master, this marks 3.5 alpha/beta release; (#31) ... * implement inital code to support embedded FNE; * continued implementation of embedded FNE; * implement initial core functionality for handling FNE side of DMRD, P25D and NXDD; * update README.md; * refactor and rename files; clarify that the embedded FNE is not a "routing" FNE, but rather, a "conference bridge" FNE; * more cleanups making the conference bridge FNE implementation clearer; * implement proper code for handling peer inclusion/exclusion; implement proper code for validating a call stream; * add missing comments; * make the GCC compiler happy when compiling on 32-bit ARM; make old RPI_ARM compiler happy; * process parrot flag for talkgroup configuration; * deprecate and remove old tg_acl.dat file infavor of the YAML-based talkgroup rules file for the entire host; deprecate and remove custom Mutex class in favor of std::mutex; expand helper routines for the TalkgroupRulesLookup class (backwards compatibility essentially with TalkgroupIdLookup); fix issue in network core that could cause deactivated TGs to try using talkgroup lookups when talkgroup lookups may be unavailable; * support code for future use; * fixup CMakeLists to fix build warnings for ARM 32-bit platform; * update detection for sendmsg and sendmmsg (handles some platforms that don't define sendmmsg); implement container classes for RTP framing; * update CMakeLists to handle new files appropriately; * initial implementation of FrameQueue to generate FNE RTP frames; * fix incorrect return; * implement a typedef for unique_ptr uint8_t arrays; fix issue where DMR, P25 and NXDN control classes were accepting BaseNetwork instead of Network; refactor how FrameQueue handles returning messages read from the network; refactor BaseNetwork, Network and FNENetwork to use FrameQueue instead of raw network writes; * fix issue with handling network state; fix issue handling global network enable flag; fix buffer pointer cleanup; * add option to CMake to selectively utilize the legacy (non-RTP) network protocol (embedded FNE does not support this and will be disabled for USE_LEGACY_NETWORK; cleanup some error messages when trying to initialize and read the configuration file; fix some minor issues in parsing the new talkgroup rules YAML file; complete conversion of BaseNetwork from raw socket to using the new FrameQueue class; refactor the multi-buffer UDPSocket write function to properly send multiple messages in batch down to the kernel driver (on Linux using sendmmsg); add flag to UDPSocket to indicate whether the socket is "open" or not; correct some RTP formatting issues with the extension header (the payload length is in 32-bit units); * remove remaining native WIN32/WIN64 type support code (it was unmaintained anyway); correct issues with network socket reading; cleanup some error messages;q * finally resolve lingering issue with frame queue having random socket issues; * refactor how BufferVector and sendmmsg UDPSocket write operates; refactor FrameQueue to properly queue messages; * properly queue packets to send in bursts to peers; add missing ACKs; add missing MSTPONG; * fix up formatting; remove unnecessary debug statements; * minor cleanups; * remove unused function; * add fields for function and sub-function; * describe network functions and subfunctions; * reorganize some code; add checking for packet sequence; * roll next sequence properly; * utilize typedef for std::unique_ptr<uint8_t[]>; * fix usage of __UNIQUE_BUFFER to __UNIQUE_UINT8_ARRAY; * clean up macro definitions; * implement sequence counting for the FNE conference bridge; * remove USE_LEGACY_NETWORK support; * use macro expansions for this instead of raw buffer lists; * implement actual transmission of new network func/sub-func behavior; * deprecate use of packet tags for determining operation, instead use RTP FNE header function and subfunction bytes; * fix issue with send talkgroups not setting flag to send talkgroups to peers; fix issue when compiling list of activated or deactivated TGs to send to peers; correct strangeness with the RTP sequence counting; fix talkgroup rule lookup on peer when activating or deactivating TGs; * update copyright dates; * complain if the RTP and FNE headers don't agree on stream ID; * for when you forget to update a comment so things actually make sense when you look at it later...; * remove extraneous check, this is really not necessary since FrameQueue does the size validation; * clarify the description of the "embedded FNE"; * merge changes from https://github.com/CVSoft/dvmhost to place spec specific limits on channel spacing; * refactor how network traffic is handled, we don't handle protocol specific stuff within BaseNetwork anymore and offload that to the specific protocol controller instead; implement support to receive peer traffic and repeat (still need to do master to peer); * convert fatal error about Rx frequency being below base frequency to a warning instead; * fix incorrect redefine of slotNo; * correct ret flag for readDMR, readP25 and readNXDN not being set to true by default (stupid me); correct FNE-mode payload offsets for diagnostic packets; * implement option to selectively enable/disable FNE mode verbose logging information; enhance FNE display of network diagnostic logs; refactor dmr::Control::processNetwork(); refactor RTP packet sequence counting; * fix incorrect variable used for out-of-sequence log message; * fix issues with login RTP stream; specially handle certain control opcodes with a different payload type; continued enhancements to sequence counting; * whoops readd accidentally removed block; * backout previous lastPeerId code; properly ensure RTP packet sequence is repeated for traffic calls; * transmit DVM LC_CALL_TERM TSDU at the end of call or if a call drops (hopefully); * don't reset the packet sequence for a RPTPING; * rename some class variables for clarity and prep for some future changes; * add support for priority/immediate Tx queues, these queues are intended to be used sparingly (and mostly for control data) to transmit priority messages [this is an experimental change and may break the build!]; * fix issue where connecting to a non-existent REST API endpoint would crash with an unhandled ASIO exception; * increase timeout delay for the modem by 4 seconds (the original 4 second delay would be too short for some longer operations); * use the immediate queue to prioritize control data in some situations [this is experimental and may break things!]; * reformat headers; * implement toString() functions on TSBKs, CSBKs and RCCHs for future use to replace fixed strings; cleanup DMRDefines slightly; * fix up and simplify RTP timestamp generation; * reduce reliance on fixed strings, instead rely on toString() functions from TSBK, CSBK, and RCCH to ensure string consistency for logging purpsoes; * refactor how and when releaseGrant is called in some situations; ensure channel grant messages traverse the network; * fix reset of error count between network super-frames; * add development helper file for VS Code to configure debug launch parameters; * update CMakeLists to be more condusive to debugging; * cleanup AUTHORS.md; * implement parrot for the conference bridge FNE; implement appropriate call detection logic for conference bridge FNE and attempt to prevent call collisons for the same destination; * add support for altering softpot levels from calibration mode; * bump copyright information; * reorganize CMakeLists; * remove finalcut from the libraries list (this isn't supposed to be included yet!); * implement initial rework of setup/calibration mode into a TUI; add ENABLE_TUI_SUPPORT CMake option to disable *all* project TUI support; add ENABLE_TUI_SETUP CMake option to disable setup TUI support for dvmhost; * properly update isConnected flag; * fix issue with incorrectly placed control; * part 1 of a fix for #30, the change here would be to allow the VCs to know about the CC to be able to report back to it to release or other wise update grants; * part 2 of the fix for #30, this implements the actual logic for a VC to notify the CC of a grant status change (touch, release) [NOTE: this could have undesirable consequences and this commit is experimental!]; * update README.md; * fix terrible copy and paste job resulting in a poor logic check (SMH); print to log the control channel address and port at startup; use the appropriate REST API for touch (whoops); * remove transmit overlay on shell; add F12 hotkey to trigger transmit; implement support to use F2 and F12 from various configuration/calibration modals; correct some mode strangeness when saving; * implement support to adjust FIFO buffer sizes from the host; * add engineering menu to adjust FIFO buffer sizes from setup; * ensure set button state is set properly for these windows; * fix stupid --tags non-sense in the git hash reporting; * ensure certain error conditions for setup mode are displayed when exiting at initial startup; * more fixes to attempt to satisfy #30 (why is this being so difficult!); correct situation where RESTClient may throw an assert instead of handling an error condition nicely; * for #30 allow release tg grant and touch tg grant to work regardless of dedicated CC setting; * file cleanups; * C++11/14 cleanups, class overrides and pure-virtual implementors /worked/ but were not really correct; * stringify modem command and reason messages (so log errors are easier on the eyes); * late update during a branch merge, to fix some latent issues that were missed;	3 years ago
Bryan Biedenkapp	ca161aa402	cleanup trailing whitespace;	4 years ago
K4YT3X	c487873db2	Reorganizing directory structure and minor housekeeping (#21) ... * reorganized files into appropriate directories * updated README, AUTHOR, and LICENSE to conform to conventions * adjusted CMakeLists and pipeline for the new directory structure * moved examples->configs; scripts->tools; src/debian->/debian * updated authors information * created new section in author information * removed deprecated iden_channel_calc tool	4 years ago