KB8PMY
/
openhamclock
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

init commit of spider dx service

Browse Source
pull/27/head
accius 4 days ago
parent f6ea975bad
commit 675c4c9495
5 changed files with 680 additions and 0 deletions
Show Stats Download Patch File Download Diff File

22
dxspider-proxy/Dockerfile
Unescape View File

@ -0,0 +1,22 @@
FROM node:20-alpine
WORKDIR /app
# Copy package files
COPY package*.json ./
# Install dependencies
RUN npm install --production
# Copy source
COPY server.js ./
# Expose port
EXPOSE 3001
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=10s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:3001/health || exit 1
# Start
CMD ["node", "server.js"]

189
dxspider-proxy/README.md
Unescape View File

@ -0,0 +1,189 @@
# DX Spider Telnet Proxy
A microservice that maintains a persistent telnet connection to DX Spider cluster nodes and exposes the spots via a simple HTTP REST API.
## Why This Exists
Many cloud hosting platforms (including Railway) don't support outbound telnet connections. This proxy service solves that by:
1. Maintaining a persistent telnet connection to DX Spider
2. Accumulating spots in memory (30-minute retention)
3. Exposing spots via HTTP API that any app can consume
## Features
- **Auto-reconnect** - Automatically reconnects on disconnect
- **Multi-node failover** - Cycles through multiple DX Spider nodes if one fails
- **Keepalive** - Sends periodic keepalive to maintain connection
- **Spot deduplication** - Prevents duplicate spots within 2-minute window
- **30-minute retention** - Accumulates spots for richer data
- **Mode detection** - Automatically detects CW, SSB, FT8, etc. from comments
- **CORS enabled** - Can be called from any frontend
## Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `PORT` | 3001 | HTTP server port |
| `CALLSIGN` | OPENHAMCLOCK | Callsign used for DX Spider login |
## API Endpoints
### `GET /health`
Health check endpoint.
```json
{
"status": "ok",
"connected": true,
"currentNode": "DX Spider UK",
"spotsInMemory": 142,
"totalSpotsReceived": 1847,
"lastSpotTime": "2025-01-31T12:34:56.789Z",
"connectionUptime": "3600s",
"uptime": "7200s"
}
```
### `GET /api/spots`
Get accumulated spots with full details.
Query parameters:
- `limit` (default: 50, max: 200) - Number of spots to return
- `since` (timestamp) - Only return spots after this timestamp
```json
{
"spots": [
{
"spotter": "W3ABC",
"freq": "14.025",
"freqKhz": 14025,
"call": "JA1XYZ",
"comment": "CW 599",
"time": "12:34z",
"mode": "CW",
"timestamp": 1706704496789,
"source": "DX Spider"
}
],
"total": 142,
"connected": true,
"source": "DX Spider UK",
"timestamp": 1706704500000
}
```
### `GET /api/dxcluster/spots`
Get spots in simplified format (compatible with OpenHamClock).
Query parameters:
- `limit` (default: 25, max: 100)
```json
[
{
"spotter": "W3ABC",
"freq": "14.025",
"call": "JA1XYZ",
"comment": "CW 599",
"time": "12:34z",
"mode": "CW",
"source": "DX Spider Proxy"
}
]
```
### `GET /api/stats`
Get statistics about spots.
```json
{
"connected": true,
"currentNode": "DX Spider UK",
"totalSpots": 142,
"totalReceived": 1847,
"lastSpotTime": "2025-01-31T12:34:56.789Z",
"retentionMinutes": 30,
"bandCounts": {
"20m": 45,
"40m": 32,
"15m": 28,
"10m": 20
},
"modeCounts": {
"FT8": 67,
"CW": 35,
"SSB": 25
}
}
```
### `GET /api/nodes`
List available DX Spider nodes.
```json
{
"nodes": [
{ "index": 0, "name": "DX Spider UK", "host": "dxspider.co.uk", "port": 7300, "active": true },
{ "index": 1, "name": "W6KK", "host": "w6kk.no-ip.org", "port": 7300, "active": false }
],
"currentIndex": 0
}
```
### `POST /api/reconnect`
Force reconnection to current node.
### `POST /api/switch-node`
Switch to a different node.
```json
{ "index": 1 }
```
## Deployment
### Railway
1. Create a new project in Railway
2. Connect your GitHub repo or upload files
3. Set environment variable: `CALLSIGN=YOURCALL`
4. Deploy!
The service will automatically start and connect to DX Spider.
### Docker
```bash
docker build -t dxspider-proxy .
docker run -p 3001:3001 -e CALLSIGN=YOURCALL dxspider-proxy
```
### Local Development
```bash
npm install
CALLSIGN=YOURCALL npm start
```
## Using with OpenHamClock
Once deployed, update your OpenHamClock configuration to use this proxy as a DX cluster source:
```
https://your-proxy-url.railway.app/api/dxcluster/spots
```
## DX Spider Nodes
The proxy cycles through these nodes on failure:
1. dxspider.co.uk:7300 (UK)
2. w6kk.no-ip.org:7300 (California)
3. dxc.nc7j.com:7373 (NC7J)
4. dx.k3lr.com:7300 (K3LR)
## License
MIT

26
dxspider-proxy/package.json
Unescape View File

@ -0,0 +1,26 @@
{
"name": "dxspider-proxy",
"version": "1.0.0",
"description": "DX Spider Telnet to HTTP Proxy - Maintains persistent telnet connection and serves spots via REST API",
"main": "server.js",
"scripts": {
"start": "node server.js",
"dev": "node server.js"
},
"keywords": [
"ham-radio",
"dx-cluster",
"dxspider",
"amateur-radio",
"telnet-proxy"
],
"author": "OpenHamClock",
"license": "MIT",
"dependencies": {
"cors": "^2.8.5",
"express": "^4.18.2"
},
"engines": {
"node": ">=18.0.0"
}
}

8
dxspider-proxy/railway.toml
Unescape View File

@ -0,0 +1,8 @@
[build]
builder = "nixpacks"
[deploy]
startCommand = "node server.js"
healthcheckPath = "/health"
healthcheckTimeout = 30
restartPolicyType = "always"

435
dxspider-proxy/server.js
Unescape View File

@ -0,0 +1,435 @@
/**
* DX Spider Telnet Proxy Service
*
* A microservice that maintains a persistent telnet connection to DX Spider,
* accumulates spots, and serves them via HTTP API.
*
* Designed to run on Railway as a standalone service.
*/
const net = require('net');
const express = require('express');
const cors = require('cors');
const app = express();
app.use(cors());
app.use(express.json());
// Configuration
const CONFIG = {
// DX Spider nodes to try (in order)
nodes: [
{ host: 'dxspider.co.uk', port: 7300, name: 'DX Spider UK' },
{ host: 'w6kk.no-ip.org', port: 7300, name: 'W6KK' },
{ host: 'dxc.nc7j.com', port: 7373, name: 'NC7J' },
{ host: 'dx.k3lr.com', port: 7300, name: 'K3LR' }
],
callsign: process.env.CALLSIGN || 'OPENHAMCLOCK',
spotRetentionMs: 30 * 60 * 1000, // 30 minutes
reconnectDelayMs: 5000, // 5 seconds between reconnect attempts
maxReconnectAttempts: 5,
cleanupIntervalMs: 60000, // 1 minute
keepAliveIntervalMs: 120000 // 2 minutes - send keepalive
};
// State
let spots = [];
let client = null;
let connected = false;
let currentNode = null;
let currentNodeIndex = 0;
let reconnectAttempts = 0;
let lastSpotTime = null;
let totalSpotsReceived = 0;
let connectionStartTime = null;
let buffer = '';
let reconnectTimer = null;
let keepAliveTimer = null;
// Logging helper
const log = (level, message, data = null) => {
const timestamp = new Date().toISOString();
const logLine = `[${timestamp}] [${level}] ${message}`;
if (data) {
console.log(logLine, typeof data === 'object' ? JSON.stringify(data) : data);
} else {
console.log(logLine);
}
};
// Parse a DX spot line from telnet
// Format: DX de SPOTTER: FREQ DXCALL comment time
const parseSpotLine = (line) => {
try {
// Match: DX de W3ABC: 14025.0 JA1XYZ CW 599 1234Z
const match = line.match(/^DX de\s+([A-Z0-9/]+):\s+(\d+\.?\d*)\s+([A-Z0-9/]+)\s+(.*)$/i);
if (!match) return null;
const spotter = match[1].toUpperCase();
const freqKhz = parseFloat(match[2]);
const dxCall = match[3].toUpperCase();
let comment = match[4].trim();
// Extract time from end of comment (format: 1234Z or 1234z)
let time = '';
const timeMatch = comment.match(/(\d{4})[Zz]\s*$/);
if (timeMatch) {
time = timeMatch[1].substring(0, 2) + ':' + timeMatch[1].substring(2, 4) + 'z';
comment = comment.replace(/\d{4}[Zz]\s*$/, '').trim();
} else {
// Use current UTC time
const now = new Date();
time = String(now.getUTCHours()).padStart(2, '0') + ':' +
String(now.getUTCMinutes()).padStart(2, '0') + 'z';
}
// Detect mode from comment
let mode = null;
const upperComment = comment.toUpperCase();
if (upperComment.includes('FT8')) mode = 'FT8';
else if (upperComment.includes('FT4')) mode = 'FT4';
else if (upperComment.includes('CW')) mode = 'CW';
else if (upperComment.includes('SSB') || upperComment.includes('USB') || upperComment.includes('LSB')) mode = 'SSB';
else if (upperComment.includes('RTTY')) mode = 'RTTY';
else if (upperComment.includes('PSK')) mode = 'PSK';
else if (upperComment.includes('FM')) mode = 'FM';
else if (upperComment.includes('AM')) mode = 'AM';
return {
spotter,
freq: (freqKhz / 1000).toFixed(3), // Convert kHz to MHz string
freqKhz,
call: dxCall,
comment,
time,
mode,
timestamp: Date.now(),
source: 'DX Spider'
};
} catch (err) {
log('ERROR', 'Failed to parse spot line', { line, error: err.message });
return null;
}
};
// Add a spot to the accumulator
const addSpot = (spot) => {
if (!spot) return;
// Check for duplicate (same call + freq within 2 minutes)
const isDuplicate = spots.some(existing =>
existing.call === spot.call &&
existing.freq === spot.freq &&
(spot.timestamp - existing.timestamp) < 120000
);
if (!isDuplicate) {
spots.unshift(spot); // Add to beginning (newest first)
totalSpotsReceived++;
lastSpotTime = new Date();
log('SPOT', `${spot.call} on ${spot.freq} MHz by ${spot.spotter}`);
}
};
// Clean up old spots
const cleanupSpots = () => {
const cutoff = Date.now() - CONFIG.spotRetentionMs;
const before = spots.length;
spots = spots.filter(s => s.timestamp > cutoff);
const removed = before - spots.length;
if (removed > 0) {
log('CLEANUP', `Removed ${removed} expired spots, ${spots.length} remaining`);
}
};
// Connect to DX Spider
const connect = () => {
if (client) {
try {
client.destroy();
} catch (e) {}
client = null;
}
if (reconnectTimer) {
clearTimeout(reconnectTimer);
reconnectTimer = null;
}
const node = CONFIG.nodes[currentNodeIndex];
currentNode = node;
log('CONNECT', `Attempting connection to ${node.name} (${node.host}:${node.port})`);
client = new net.Socket();
client.setTimeout(30000);
client.connect(node.port, node.host, () => {
connected = true;
reconnectAttempts = 0;
connectionStartTime = new Date();
buffer = '';
log('CONNECT', `Connected to ${node.name}`);
// Send login after short delay
setTimeout(() => {
if (client && connected) {
client.write(CONFIG.callsign + '\r\n');
log('AUTH', `Sent callsign: ${CONFIG.callsign}`);
}
}, 1000);
// Start keepalive
startKeepAlive();
});
client.on('data', (data) => {
buffer += data.toString();
// Process complete lines
const lines = buffer.split('\n');
buffer = lines.pop() || ''; // Keep incomplete line in buffer
for (const line of lines) {
const trimmed = line.trim();
if (!trimmed) continue;
// Check if it's a DX spot
if (trimmed.startsWith('DX de ')) {
const spot = parseSpotLine(trimmed);
if (spot) {
addSpot(spot);
}
}
}
});
client.on('timeout', () => {
log('TIMEOUT', 'Connection timed out');
handleDisconnect();
});
client.on('error', (err) => {
log('ERROR', `Connection error: ${err.message}`);
handleDisconnect();
});
client.on('close', () => {
log('CLOSE', 'Connection closed');
handleDisconnect();
});
};
// Start keepalive timer
const startKeepAlive = () => {
if (keepAliveTimer) {
clearInterval(keepAliveTimer);
}
keepAliveTimer = setInterval(() => {
if (client && connected) {
try {
// Send a harmless command to keep connection alive
client.write('\r\n');
log('KEEPALIVE', 'Sent keepalive');
} catch (e) {
log('ERROR', 'Keepalive failed', e.message);
}
}
}, CONFIG.keepAliveIntervalMs);
};
// Handle disconnection and reconnection
const handleDisconnect = () => {
connected = false;
if (keepAliveTimer) {
clearInterval(keepAliveTimer);
keepAliveTimer = null;
}
reconnectAttempts++;
if (reconnectAttempts >= CONFIG.maxReconnectAttempts) {
// Try next node
currentNodeIndex = (currentNodeIndex + 1) % CONFIG.nodes.length;
reconnectAttempts = 0;
log('FAILOVER', `Switching to node: ${CONFIG.nodes[currentNodeIndex].name}`);
}
log('RECONNECT', `Attempting reconnect in ${CONFIG.reconnectDelayMs}ms (attempt ${reconnectAttempts + 1})`);
reconnectTimer = setTimeout(() => {
connect();
}, CONFIG.reconnectDelayMs);
};
// ============================================
// HTTP API ENDPOINTS
// ============================================
// Health check
app.get('/health', (req, res) => {
res.json({
status: 'ok',
connected,
currentNode: currentNode?.name || 'none',
spotsInMemory: spots.length,
totalSpotsReceived,
lastSpotTime: lastSpotTime?.toISOString() || null,
connectionUptime: connectionStartTime ?
Math.floor((Date.now() - connectionStartTime.getTime()) / 1000) + 's' : null,
uptime: process.uptime() + 's'
});
});
// Get spots
app.get('/api/spots', (req, res) => {
const limit = Math.min(parseInt(req.query.limit) || 50, 200);
const since = parseInt(req.query.since) || 0; // Timestamp filter
let filteredSpots = spots;
// Filter by timestamp if provided
if (since > 0) {
filteredSpots = spots.filter(s => s.timestamp > since);
}
res.json({
spots: filteredSpots.slice(0, limit),
total: filteredSpots.length,
connected,
source: currentNode?.name || 'disconnected',
timestamp: Date.now()
});
});
// Get spots in simple format (for compatibility with existing DX cluster endpoint)
app.get('/api/dxcluster/spots', (req, res) => {
const limit = Math.min(parseInt(req.query.limit) || 25, 100);
const formattedSpots = spots.slice(0, limit).map(s => ({
spotter: s.spotter,
freq: s.freq,
call: s.call,
comment: s.comment,
time: s.time,
mode: s.mode,
source: 'DX Spider Proxy'
}));
res.json(formattedSpots);
});
// Stats endpoint
app.get('/api/stats', (req, res) => {
// Calculate spots per band
const bandCounts = {};
spots.forEach(s => {
const freq = s.freqKhz;
let band = 'other';
if (freq >= 1800 && freq <= 2000) band = '160m';
else if (freq >= 3500 && freq <= 4000) band = '80m';
else if (freq >= 7000 && freq <= 7300) band = '40m';
else if (freq >= 10100 && freq <= 10150) band = '30m';
else if (freq >= 14000 && freq <= 14350) band = '20m';
else if (freq >= 18068 && freq <= 18168) band = '17m';
else if (freq >= 21000 && freq <= 21450) band = '15m';
else if (freq >= 24890 && freq <= 24990) band = '12m';
else if (freq >= 28000 && freq <= 29700) band = '10m';
else if (freq >= 50000 && freq <= 54000) band = '6m';
bandCounts[band] = (bandCounts[band] || 0) + 1;
});
// Calculate spots per mode
const modeCounts = {};
spots.forEach(s => {
const mode = s.mode || 'unknown';
modeCounts[mode] = (modeCounts[mode] || 0) + 1;
});
res.json({
connected,
currentNode: currentNode?.name || 'none',
totalSpots: spots.length,
totalReceived: totalSpotsReceived,
lastSpotTime: lastSpotTime?.toISOString() || null,
retentionMinutes: CONFIG.spotRetentionMs / 60000,
bandCounts,
modeCounts
});
});
// Force reconnect
app.post('/api/reconnect', (req, res) => {
log('API', 'Force reconnect requested');
handleDisconnect();
res.json({ status: 'reconnecting' });
});
// Switch node
app.post('/api/switch-node', (req, res) => {
const { index } = req.body;
if (typeof index === 'number' && index >= 0 && index < CONFIG.nodes.length) {
currentNodeIndex = index;
reconnectAttempts = 0;
log('API', `Switching to node index ${index}: ${CONFIG.nodes[index].name}`);
handleDisconnect();
res.json({ status: 'switching', node: CONFIG.nodes[index].name });
} else {
res.status(400).json({ error: 'Invalid node index', availableNodes: CONFIG.nodes.map(n => n.name) });
}
});
// List available nodes
app.get('/api/nodes', (req, res) => {
res.json({
nodes: CONFIG.nodes.map((n, i) => ({
index: i,
name: n.name,
host: n.host,
port: n.port,
active: i === currentNodeIndex
})),
currentIndex: currentNodeIndex
});
});
// ============================================
// STARTUP
// ============================================
const PORT = process.env.PORT || 3001;
// Start cleanup interval
setInterval(cleanupSpots, CONFIG.cleanupIntervalMs);
// Start server
app.listen(PORT, () => {
log('START', `DX Spider Proxy listening on port ${PORT}`);
log('CONFIG', `Callsign: ${CONFIG.callsign}`);
log('CONFIG', `Spot retention: ${CONFIG.spotRetentionMs / 60000} minutes`);
log('CONFIG', `Available nodes: ${CONFIG.nodes.map(n => n.name).join(', ')}`);
// Connect to DX Spider
connect();
});
// Handle graceful shutdown
process.on('SIGTERM', () => {
log('SHUTDOWN', 'Received SIGTERM, shutting down...');
if (client) {
client.destroy();
}
process.exit(0);
});
process.on('SIGINT', () => {
log('SHUTDOWN', 'Received SIGINT, shutting down...');
if (client) {
client.destroy();
}
process.exit(0);
});
Write Preview
Loading…
Cancel
Save

Powered by TurnKey Linux.