Add Joanna telemetry and topology diagram tooling

pull/1706/head
Carlo Costanzo 2 months ago
parent e6bfe444bd
commit bc1997dccb

@ -27,6 +27,7 @@ Codex skills stored in-repo so they can be shared with the community. These are
- `homeassistant-dashboard-designer/`: Constrained, button-card-first Lovelace dashboard design system + YAML lint helper.
- `homeassistant-yaml-dry-verifier/`: Home Assistant YAML DRY verifier to detect redundant triggers/conditions/actions/sequence blocks and suggest refactors.
- `infrastructure-doc-sync/`: Session closeout workflow to update AGENTS/README/Dashy shortcuts/Infra Info snapshot consistently after infra changes.
- `network-architecture-diagrammer/`: Mermaid-first homelab/network architecture diagram workflow for Excalidraw-friendly topology and service maps.
### Notes
- Codex loads skills from your local Codex home. See each skill folder for install steps.

@ -0,0 +1,91 @@
---
name: network-architecture-diagrammer
description: "Create homelab and network architecture diagrams from plain-English prompts by producing Mermaid-first artifacts that import cleanly into Excalidraw. Use for Docker host topology, service dependency maps, Cloudflare/public edge diagrams, and before/after infrastructure sketches."
---
# Network Architecture Diagrammer
Use text-first architecture diagrams that round-trip well: Mermaid source first, Excalidraw second.
## When to Use
- The user asks for a network diagram, architecture sketch, topology map, or service relationship view.
- You need a reviewable artifact that belongs in git, docs, or an issue comment.
- You need a fast homelab diagram from AGENTS, compose files, README notes, or runtime inventory.
## Primary Artifact
- Produce Mermaid flowcharts first.
- Keep the output Excalidraw-importable instead of Mermaid-fancy.
- If the user wants a file, write a `.mmd` file.
- Add a short `.md` legend or assumptions file only when it materially helps.
## Workflow
1. Collect topology truth from `AGENTS.md`, compose files, repo docs, and MCP/runtime inventory when available.
2. Pick one view:
- context: user, cloud, edge, host
- deployment: hosts, containers/apps, data stores
- flow: request/data/event path
- delta: before/after architecture change
3. Normalize nodes into a small set:
- users
- internet/cloud
- edge/tunnel/proxy
- hosts
- containers/apps
- data stores
- external systems
4. Split large requests into two diagrams instead of forcing one dense canvas.
5. Write Mermaid with one subgraph per host or trust zone and short, stable labels.
6. Run `scripts/validate_mermaid_excalidraw.py` on any Mermaid file you create.
7. Deliver the Mermaid artifact plus short assumptions and the Excalidraw import hint when useful.
## Excalidraw Compatibility Rules
- Use only Mermaid `flowchart` or `graph` syntax.
- Prefer plain labels, simple arrows, and `subgraph`.
- Avoid features that commonly import poorly into Excalidraw:
- non-flowchart diagram types such as `sequenceDiagram`, `classDiagram`, `stateDiagram`, `erDiagram`, `journey`, `gantt`, `mindmap`, `timeline`, `pie`, `gitGraph`, `block-beta`, `packet-beta`, `sankey-beta`, `quadrantChart`, and `zenuml`
- expanded shape syntax like `@{ ... }`
- `classDef`, `class`, `style`, `linkStyle`, `click`, and HTML labels/tags
- Prefer stable hostnames over LAN IPs unless the IPs are the point of the diagram.
- Keep edge labels short: protocol, purpose, or port only when it changes understanding.
- Keep each diagram roughly under 15 nodes. Split by audience or scope when needed.
## Output Contract
Always produce:
1. A Mermaid code block.
2. A short assumptions or omissions list when anything was inferred.
3. An optional recommended filename such as `docs/diagrams/docker-host-topology.mmd`.
4. A short note that the Mermaid can be imported into Excalidraw for cleanup when the user wants a polished visual.
If asked to write files, prefer:
- `docs/diagrams/<name>.mmd`
- `docs/diagrams/<name>.md` for legend/notes only when helpful
## Prompt Pattern
Natural-language prompts are the default. Helpful details include:
- scope
- audience
- desired granularity
- whether public vs LAN edge matters
- whether a before/after comparison is needed
Example:
- "Create a deployment diagram for docker10/docker14/docker17/docker69, show Cloudflare edge publishing, and make clear which services are LAN-only."
## References
- Read `references/excalidraw_mermaid_rules.md` for the diagram playbook and example patterns.
## Validation
- Run `python scripts/validate_mermaid_excalidraw.py <file.mmd>` when a Mermaid file is created.
- If the validator flags unsupported syntax, simplify the diagram instead of forcing Mermaid-only features.

@ -0,0 +1,4 @@
interface:
display_name: "Network Architecture Diagrammer"
short_description: "Generate Excalidraw-ready homelab diagrams"
default_prompt: "Use $network-architecture-diagrammer to turn a plain-English homelab or network architecture request into a Mermaid-first diagram that imports cleanly into Excalidraw."

@ -0,0 +1,134 @@
# Excalidraw Mermaid Playbook
Use this reference when you need a stable, reviewable architecture diagram artifact.
## Goal
Produce Mermaid that:
- is easy to diff in git
- is readable in markdown or issue comments
- imports into Excalidraw without heavy cleanup
## Recommended Views
### Context diagram
Use when the audience only needs major boundaries.
```mermaid
flowchart LR
User[Carlo]
Internet[Internet]
Cloudflare[Cloudflare Tunnel]
subgraph docker17[docker17]
Dashy[Dashy]
Codex[Codex Appliance]
end
User --> Internet --> Cloudflare --> Dashy
Cloudflare --> Codex
```
### Deployment diagram
Use when the audience needs host and service placement.
```mermaid
flowchart TD
subgraph docker10[docker10]
HA[Home Assistant]
MQTT[MQTT]
UniFi[UniFi]
end
subgraph docker14[docker14]
Frigate[Frigate]
end
subgraph docker17[docker17]
Appliance[Codex Appliance]
Dashy[Dashy]
end
subgraph docker69[docker69]
Tunnel[Cloudflared]
Info[Infra Info]
end
Tunnel --> Appliance
Tunnel --> Info
HA --> MQTT
Frigate --> HA
```
### Flow diagram
Use when the audience cares about one path, not the full estate.
```mermaid
flowchart LR
User[User]
Telegram[Telegram]
Joanna[Joanna]
HA[Home Assistant]
Notify[notify_engine]
User --> Telegram --> Joanna --> HA --> Notify
```
## Label Rules
- Prefer `docker17` over raw IP addresses.
- Prefer `Cloudflare Tunnel` over product-internal nouns unless the audience already knows them.
- Keep labels human-readable; IDs belong in notes, not in node titles.
## Edge Rules
- Use unlabeled arrows for obvious containment or traffic.
- Add short labels only when they matter:
- `RTSP`
- `MQTT`
- `HTTPS`
- `Webhook`
- `8124`
## Split Rules
Split into multiple diagrams when any of these are true:
- more than four hosts
- more than 15 nodes
- mixed audiences (operators vs app users)
- both runtime placement and request flow are important
Good split example:
- diagram 1: public edge and user entry points
- diagram 2: host/container placement
## Excalidraw Guardrails
Avoid these Mermaid features in Excalidraw-bound artifacts:
- non-flowchart diagram types
- expanded shape syntax (`@{ ... }`)
- CSS classes or class attachments
- inline style directives
- HTML inside labels
- click handlers
If you want polish, do it after import inside Excalidraw.
## File Naming
Prefer short, obvious filenames:
- `docs/diagrams/bear-stone-context.mmd`
- `docs/diagrams/docker-host-topology.mmd`
- `docs/diagrams/cloudflare-edge-flow.mmd`
## Review Checklist
- Does the diagram answer one question clearly?
- Are host boundaries obvious?
- Are public entry points distinguishable from LAN-only services?
- Is the artifact still readable as raw Mermaid text?
- Will Excalidraw import it without advanced Mermaid features?

@ -0,0 +1,77 @@
from __future__ import annotations
import argparse
import re
import sys
from pathlib import Path
DISALLOWED_PATTERNS: list[tuple[re.Pattern[str], str]] = [
(
re.compile(
r"^\s*(sequenceDiagram|classDiagram|stateDiagram|erDiagram|journey|gantt|pie|mindmap|timeline|gitGraph|quadrantChart|zenuml|block-beta|packet-beta|sankey-beta)\b",
re.MULTILINE,
),
"Use Mermaid flowchart syntax only for Excalidraw import.",
),
(
re.compile(r"@\{"),
"Avoid expanded Mermaid shape syntax (`@{ ... }`); simplify the node shape.",
),
(
re.compile(r"^\s*(classDef|class|style|linkStyle|click)\b", re.MULTILINE),
"Avoid Mermaid styling/class directives; keep the diagram plain for Excalidraw.",
),
(
re.compile(r":::\w+"),
"Avoid Mermaid class attachments (`:::`); keep the diagram plain for Excalidraw.",
),
(
re.compile(r"<[^>\n]+>"),
"Avoid HTML inside Mermaid labels; use plain text labels instead.",
),
]
def validate_mermaid(text: str) -> list[str]:
findings: list[str] = []
if not re.search(r"^\s*(flowchart|graph)\s+(LR|RL|TB|TD|BT)\b", text, re.MULTILINE):
findings.append("Expected a Mermaid flowchart header such as `flowchart LR` or `flowchart TD`.")
for pattern, message in DISALLOWED_PATTERNS:
if pattern.search(text):
findings.append(message)
return findings
def _read_text(path: str | None) -> str:
if path:
return Path(path).read_text(encoding="utf-8")
return sys.stdin.read()
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(
description="Validate Mermaid syntax for clean Excalidraw import."
)
parser.add_argument("path", nargs="?", help="Optional Mermaid file to validate")
args = parser.parse_args(argv)
text = _read_text(args.path)
findings = validate_mermaid(text)
if findings:
print("Mermaid is not Excalidraw-friendly:")
for finding in findings:
print(f"- {finding}")
return 1
print("Mermaid looks compatible with the Mermaid-to-Excalidraw workflow.")
return 0
if __name__ == "__main__":
raise SystemExit(main())

@ -0,0 +1,46 @@
from __future__ import annotations
import sys
import unittest
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts"))
import validate_mermaid_excalidraw as validator
class ValidateMermaidExcalidrawTests(unittest.TestCase):
def test_simple_flowchart_passes(self) -> None:
text = """flowchart LR
User[User] --> Tunnel[Cloudflare Tunnel] --> App[Dashy]
"""
self.assertEqual(validator.validate_mermaid(text), [])
def test_non_flowchart_diagram_fails(self) -> None:
findings = validator.validate_mermaid(
"""sequenceDiagram
Alice->>Bob: hello
"""
)
self.assertTrue(any("flowchart syntax only" in finding for finding in findings))
def test_expanded_shape_fails(self) -> None:
findings = validator.validate_mermaid(
"""flowchart TD
A@{ shape: rect, label: "App" }
"""
)
self.assertTrue(any("expanded Mermaid shape syntax" in finding for finding in findings))
def test_html_label_fails(self) -> None:
findings = validator.validate_mermaid(
"""flowchart TD
A["App<br/>UI"] --> B[API]
"""
)
self.assertTrue(any("Avoid HTML" in finding for finding in findings))
if __name__ == "__main__":
unittest.main()

@ -137,6 +137,42 @@
const v = states['sensor.joanna_ha_success_rate_24h']?.state ?? '0';
return `${v}%`;
]]]
- type: custom:button-card
template: bearstone_infra_list_row
entity: binary_sensor.joanna_qmd_healthy
name: QMD Health
icon: mdi:heart-pulse
state_display: >
[[[
const qmd = states['sensor.bearclaw_status_telemetry']?.attributes?.qmdHealth || {};
const healthy = states['binary_sensor.joanna_qmd_healthy']?.state === 'on';
const status = qmd?.status || (healthy ? 'healthy' : 'unknown');
const failures = states['sensor.joanna_qmd_consecutive_failures']?.state ?? '0';
return `${status} | fail ${failures}`;
]]]
- type: custom:button-card
template: bearstone_infra_list_row
entity: sensor.joanna_memory_docs
name: Memory Index
icon: mdi:database-search
state_display: >
[[[
const docs = states['sensor.joanna_memory_docs']?.state ?? '0';
const chunks = states['sensor.joanna_memory_chunks']?.state ?? '0';
const stale = states['binary_sensor.joanna_memory_index_stale']?.state === 'on';
return `${docs} docs | ${chunks} chunks${stale ? ' | stale' : ''}`;
]]]
- type: custom:button-card
template: bearstone_infra_list_row
entity: binary_sensor.joanna_session_source_indexed
name: Session Transcript Indexing
icon: mdi:message-text-clock-outline
state_display: >
[[[
const enabled = states['binary_sensor.joanna_session_source_indexed']?.state === 'on';
const age = states['sensor.joanna_minutes_since_memory_index']?.state ?? '0';
return `${enabled ? 'enabled' : 'disabled'} | ${age} min ago`;
]]]
- type: grid
column_span: 4

@ -53,7 +53,7 @@ Live collection of plug-and-play Home Assistant packages. Each YAML file in this
| [mqtt_status.yaml](mqtt_status.yaml) | Command-line MQTT broker reachability probe with Spook Repairs escalation and Joanna troubleshooting dispatch on outage. | `binary_sensor.mqtt_status_raw`, `binary_sensor.mqtt_broker_problem`, `repairs.create`, `rest_command.bearclaw_command` |
| [mariadb.yaml](mariadb.yaml) | MariaDB recorder health and capacity SQL sensors. | `sensor.mariadb_status`, `sensor.database_size` |
| [tugtainer_updates.yaml](tugtainer_updates.yaml) | Tugtainer container update notifications via webhook + persistent alerts, plus event-based Joanna dispatch when reports include `### Available:` (24h cooldown via `mode: single` + delay, no new helpers). | `persistent_notification.create`, `event: tugtainer_available_detected`, `script.joanna_dispatch`, `input_datetime.tugtainer_last_update` |
| [bearclaw.yaml](bearclaw.yaml) | Joanna/BearClaw bridge automations that forward Telegram commands to codex_appliance, include LLM-first routing context for freeform text, relay replies back, ingest `/api/bearclaw/status` telemetry, and expose dispatch KPI sensors for Infrastructure dashboards. | `rest_command.bearclaw_*`, `sensor.bearclaw_status_telemetry`, `sensor.joanna_*`, `automation.bearclaw_*`, `script.send_to_logbook` |
| [bearclaw.yaml](bearclaw.yaml) | Joanna/BearClaw bridge automations that forward Telegram commands to codex_appliance, include LLM-first routing context for freeform text, relay replies back, ingest `/api/bearclaw/status` telemetry, and expose dispatch plus QMD/memory-index sensors for Infrastructure dashboards. | `rest_command.bearclaw_*`, `sensor.bearclaw_status_telemetry`, `sensor.joanna_*`, `binary_sensor.joanna_*`, `automation.bearclaw_*`, `script.send_to_logbook` |
| [telegram_bot.yaml](telegram_bot.yaml) | Telegram script wrappers used by BearClaw and other ops flows (UI config remains source of truth; wrappers work with polling or webhook transport). | `script.joanna_send_telegram`, `telegram_bot.send_message` |
| [phynplus.yaml](phynplus.yaml) | Phyn shutoff automations with push + Activity feed + Repairs issues for leak events. | `valve.phyn_shutoff_valve`, `binary_sensor.phyn_leak_test_running`, `repairs.create` |
| [water_delivery.yaml](water_delivery.yaml) | ReadyRefresh delivery date helper with night-before + garage door Alexa reminders, plus helper-change audit logging and Telegram confirmations. | `input_datetime.water_delivery_date`, `script.send_to_logbook`, `script.joanna_send_telegram`, `notify.alexa_media_garage` |

@ -64,6 +64,8 @@ sensor:
- dispatchStats
- queue
- active
- qmdHealth
- memoryIndex
template:
- sensor:
@ -182,6 +184,71 @@ template:
{% else %}
0
{% endif %}
- name: Joanna QMD Consecutive Failures
unique_id: joanna_qmd_consecutive_failures
icon: mdi:heart-pulse
unit_of_measurement: checks
state_class: measurement
state: >-
{% set qmd = state_attr('sensor.bearclaw_status_telemetry', 'qmdHealth') | default({}, true) %}
{{ qmd.get('consecutiveFailures', 0) | int(0) }}
- name: Joanna Memory Docs
unique_id: joanna_memory_docs
icon: mdi:file-document-multiple-outline
unit_of_measurement: docs
state_class: measurement
state: >-
{% set memory = state_attr('sensor.bearclaw_status_telemetry', 'memoryIndex') | default({}, true) %}
{{ memory.get('docs', 0) | int(0) }}
- name: Joanna Memory Chunks
unique_id: joanna_memory_chunks
icon: mdi:text-box-search-outline
unit_of_measurement: chunks
state_class: measurement
state: >-
{% set memory = state_attr('sensor.bearclaw_status_telemetry', 'memoryIndex') | default({}, true) %}
{{ memory.get('chunks', 0) | int(0) }}
- name: Joanna Minutes Since Memory Index
unique_id: joanna_minutes_since_memory_index
icon: mdi:database-clock-outline
unit_of_measurement: min
state_class: measurement
state: >-
{% set memory = state_attr('sensor.bearclaw_status_telemetry', 'memoryIndex') | default({}, true) %}
{% set indexed_at = memory.get('indexedAt') %}
{% set indexed_ts = as_timestamp(indexed_at, 0) %}
{% if indexed_ts > 0 %}
{{ ((as_timestamp(now()) - indexed_ts) / 60) | round(1) }}
{% else %}
0
{% endif %}
- binary_sensor:
- name: Joanna QMD Healthy
unique_id: joanna_qmd_healthy
icon: mdi:heart-pulse
state: >-
{% set qmd = state_attr('sensor.bearclaw_status_telemetry', 'qmdHealth') | default({}, true) %}
{{ qmd.get('healthy', false) in [true, 'true', 'True', 'on', 'yes', 1, '1'] }}
- name: Joanna Session Source Indexed
unique_id: joanna_session_source_indexed
icon: mdi:message-text-clock-outline
state: >-
{% set memory = state_attr('sensor.bearclaw_status_telemetry', 'memoryIndex') | default({}, true) %}
{% set options = memory.get('options', {}) if memory is mapping else {} %}
{{ options.get('includeSessionSource', false) in [true, 'true', 'True', 'on', 'yes', 1, '1'] }}
- name: Joanna Memory Index Stale
unique_id: joanna_memory_index_stale
icon: mdi:database-alert-outline
state: >-
{% set memory = state_attr('sensor.bearclaw_status_telemetry', 'memoryIndex') | default({}, true) %}
{{ memory.get('stale', false) in [true, 'true', 'True', 'on', 'yes', 1, '1'] }}
automation:
- id: bearclaw_telegram_bear_command
alias: BearClaw Telegram Bear Command

@ -7,6 +7,7 @@
# HASS.Agent session automations plus Wake on LAN for CARLO-HOMEPC.
# -------------------------------------------------------------------
# Related Issue: 24
# - Blog: https://www.vcloudinfo.com/2026/04/home-assistant-wake-on-lan-workday-pc-automation.html
# Notes: Sleep Number nighttime lock/monitor-sleep logic lives in
# config/packages/sleepiq.yaml.
# Notes: Wake on LAN button entity is managed through the HA UI

@ -0,0 +1,7 @@
# Bear Stone topology notes
- This is a deployment inventory diagram for blog and documentation use.
- `docker10` is pinned to ProxMox1 (`qemu/105`) based on the current workspace inventory.
- `docker14`, `docker17`, and `docker69` are shown as cluster-managed Docker VMs on shared storage because the current AGENTS inventory does not pin them to a single Proxmox node.
- The diagram is intentionally hierarchy-first. It shows hosts and containers, not every runtime network edge between services.
- Use the Mermaid source as the editable system-of-record, then import it into Excalidraw for spacing and visual cleanup when a polished graphic is needed.

@ -0,0 +1,78 @@
flowchart TD
subgraph PX[Bear Stone Proxmox cluster]
P1[ProxMox1]
P2[ProxMox2]
NFS[Prox-NFS01 shared storage]
FLOAT[Cluster-managed Docker VMs]
end
P1 --> D10VM[docker10 VM]
P1 --> FLOAT
P2 --> FLOAT
NFS --> D10VM
NFS --> D14VM[docker14 VM]
NFS --> D17VM[docker17 VM]
NFS --> D69VM[docker69 VM]
FLOAT --> D14VM
FLOAT --> D17VM
FLOAT --> D69VM
subgraph D10[docker10 - primary home stack]
D10_HA[home-assistant]
D10_PI[pihole]
D10_MQTT[mqtt]
D10_DB[mariadb]
D10_BK[mariadb-backup]
D10_UNIFI[unifi]
D10_WYZE[wyze-bridge]
D10_PORT[portainer]
D10_ESP[esphome]
D10_MATTER[matter-server]
D10_DOZZLE[dozzle_agent]
D10_TUG[tugtainer-agent]
D10_PROXY[tugtainer_socket_proxy]
end
subgraph D14[docker14 - camera and DNS standby]
D14_FRIGATE[frigate]
D14_PI[pihole_secondary]
D14_NEBULA[nebula_sync]
D14_PORT[portainer_agent]
D14_DOZZLE[dozzle_agent]
D14_TUG[tugtainer-agent]
D14_PROXY[tugtainer_socket_proxy]
end
subgraph D17[docker17 - internal apps and tooling]
D17_CODEX[codex_appliance]
D17_DASHY[dashy]
D17_DOZZLE[dozzle]
D17_TUG_UI[tugtainer]
D17_DUP[duplicati]
D17_PANEL[panel_notes]
D17_CRUISE[cruise_tracker]
D17_POKER[poker_tracker]
D17_RV[rvtools_ppt_web]
D17_AGENT[docker_socket_proxy]
D17_DOZZLE_AGENT[dozzle_agent]
D17_TUG_AGENT[tugtainer-agent]
D17_TUG_PROXY[tugtainer_socket_proxy]
end
subgraph D69[docker69 - public edge and utility apps]
D69_INFO[infra_info]
D69_CF_WP[cloudflared_wp]
D69_CF_KCH[cloudflared_kch]
D69_WP_DB[wordpress_db]
D69_WP[wordpress_wp]
D69_FOODIE[foodie_tracker]
D69_IMPOSTER[imposter]
D69_BUDGET[college_budget_app]
D69_TAPPLE[tapple]
D69_GAMES[games_hub]
D69_KCH[kingcrafthomes]
D69_LMEDIA[lmediaservices]
D69_PORT[portainer_agent]
D69_DOZZLE[dozzle_agent]
D69_TUG[tugtainer-agent]
end
Loading…
Cancel
Save

Powered by TurnKey Linux.