From d706d5b431f6ec53cd1b6799c6a8daf0ab370ee3 Mon Sep 17 00:00:00 2001 From: Carlo Costanzo Date: Thu, 8 Oct 2026 17:09:59 -0400 Subject: [PATCH] Add daily Tugtainer rollout verification --- config/packages/README.md | 2 +- config/packages/tugtainer_updates.yaml | 59 ++++++++++++++++++++------ 2 files changed, 48 insertions(+), 13 deletions(-) diff --git a/config/packages/README.md b/config/packages/README.md index 681de5e7..ce9fff59 100755 --- a/config/packages/README.md +++ b/config/packages/README.md @@ -67,7 +67,7 @@ Kitchen voice uses Kiosk Satellite's [native Assist satellite](https://kiosksate | [![YAML source: mqtt_status](https://img.shields.io/static/v1?label=YAML&message=mqtt_status&color=lightgrey&logo=github&logoColor=181717)](mqtt_status.yaml) | Command-line MQTT broker reachability probe with Spook Repairs escalation and Joanna troubleshooting dispatch on outage. | `binary_sensor.mqtt_status_raw`, `binary_sensor.mqtt_broker_problem`, `repairs.create`, `rest_command.bearclaw_command` | | [![YAML source: mariadb](https://img.shields.io/static/v1?label=YAML&message=mariadb&color=lightgrey&logo=github&logoColor=181717)](mariadb.yaml) | MariaDB recorder health and capacity snapshots with hourly live metrics, weekly admin/recorder polling, and stats-ready numeric sensors. | `sensor.mariadb_status`, `sensor.database_size` | | [![YAML source: processmonitor](https://img.shields.io/static/v1?label=YAML&message=processmonitor&color=lightgrey&logo=github&logoColor=181717)](processmonitor.yaml) | Root filesystem disk-pressure monitoring with immediate digest/logbook notes at 80%, Joanna review after 10 minutes above 80%, and delayed phone alerts only if the issue stays unresolved after dispatch. | `sensor.disk_use_percent`, `repairs.create`, `script.joanna_dispatch`, `tts.clear_cache` | -| [![YAML source: tugtainer_updates](https://img.shields.io/static/v1?label=YAML&message=tugtainer_updates&color=lightgrey&logo=github&logoColor=181717)](tugtainer_updates.yaml) | Tugtainer container update reports via webhook, with Available-only reports processed without persistent alerts, visible `### Updated:` / `### Failed:` / `### Rolled-back:` outcomes, immediate Joanna recovery dispatch for failures, and Joanna review dispatch for every `### Available:` report. Home Assistant core participates in the 07:00 automatic rollout after a mandatory Tugtainer pre-update config check, with HTTP health and image rollback safeguards. Protected Tugtainer control-plane containers are updated one host at a time through their owning Compose stacks; duplicate Portainer image-update entities are auto-disabled because Tugtainer and Compose own rollout decisions. [![Watch on YouTube](https://img.shields.io/badge/Watch-YouTube-FF0000?logo=youtube&logoColor=white)](https://youtu.be/4NNOkXzUyYw) [![vCloudInfo Blog Post](https://img.shields.io/static/v1?label=vCloudInfo&message=Blog%20Post&color=21759B&logo=wordpress&logoColor=white)](https://www.vcloudinfo.com/2026/07/home-assistant-tugtainer-docker-updates.html) | `automation.tugtainer_disable_duplicate_portainer_image_updates`, `persistent_notification.create`, `event: tugtainer_available_detected`, `script.joanna_dispatch`, `input_datetime.tugtainer_last_update` | +| [![YAML source: tugtainer_updates](https://img.shields.io/static/v1?label=YAML&message=tugtainer_updates&color=lightgrey&logo=github&logoColor=181717)](tugtainer_updates.yaml) | Tugtainer container update reports via webhook, with Available-only reports processed without persistent alerts, visible `### Updated:` / `### Failed:` / `### Rolled-back:` outcomes, immediate Joanna recovery dispatch for failures, and Joanna review dispatch for every `### Available:` report. The notification template must include still-pending `available(notified)` items so unresolved updates remain visible. A daily 07:30 dispatch verifies the completed rollout, application health, and all protected updater components independently of report contents. Home Assistant core participates in the 07:00 automatic rollout after a mandatory Tugtainer pre-update config check, with HTTP health and image rollback safeguards. Protected Tugtainer control-plane containers are updated one host at a time through their owning Compose stacks; duplicate Portainer image-update entities are auto-disabled because Tugtainer and Compose own rollout decisions. [![Watch on YouTube](https://img.shields.io/badge/Watch-YouTube-FF0000?logo=youtube&logoColor=white)](https://youtu.be/4NNOkXzUyYw) [![vCloudInfo Blog Post](https://img.shields.io/static/v1?label=vCloudInfo&message=Blog%20Post&color=21759B&logo=wordpress&logoColor=white)](https://www.vcloudinfo.com/2026/07/home-assistant-tugtainer-docker-updates.html) | `automation.tugtainer_disable_duplicate_portainer_image_updates`, `persistent_notification.create`, `event: tugtainer_available_detected`, `script.joanna_dispatch`, `input_datetime.tugtainer_last_update` | | [![YAML source: printer](https://img.shields.io/static/v1?label=YAML&message=printer&color=lightgrey&logo=github&logoColor=181717)](printer.yaml) | Epson ink watchdog with one-day and one-week mobile snooze actions for low-ink reminders. | `input_datetime.printer_ink_snooze_until`, `sensor.epson_*`, mobile app action events | | [![YAML source: bearclaw](https://img.shields.io/static/v1?label=YAML&message=bearclaw&color=lightgrey&logo=github&logoColor=181717)](bearclaw.yaml) | Joanna/BearClaw bridge automations dispatch trusted Home Assistant machine jobs to codex_appliance, relay lifecycle callbacks and outbound notifications, deliver compact adaptive TeslaMate road-trip milestone, destination-arrival, and final summaries to Carlo and Stacey, emit an active-trip vacation signal, ingest `/api/bearclaw/status` telemetry, and expose dispatch, distinct BearClaw working-memory and federated OneNote KB health, compact scheduled-job warning/error health for Systems, user-keyed 7/28/90 step trends, personal records for seven-day kiosk rotations, and 48-hour Carlo/Stacey bowling highlights for Kitchen Show. Authenticated human Telegram stays on the appliance's single agent-first Luna path; Home Assistant is not its semantic router. Full Joanna reports add directional route comparisons, charging strategy, trip-aware morning context, and a private final route map. Retired QMD telemetry is not imported. Mini App monitoring independently checks local and public serving paths every minute, tolerates three minutes of failure, and creates a persistent Repair plus a mobile alert. Recovery clears the incident after one healthy minute. A restoring incident latch deduplicates alerts across restarts; Joanna receives a diagnosis request when its main service is reachable. Private probe settings stay in local secrets. These checks verify serving and the unsigned authentication boundary, not signed user actions. Road-trip companion: [![Watch on YouTube](https://img.shields.io/badge/Watch-YouTube-FF0000?logo=youtube&logoColor=white)](https://youtu.be/9-9T6v17NEw) [![vCloudInfo Blog Post](https://img.shields.io/static/v1?label=vCloudInfo&message=Blog%20Post&color=21759B&logo=wordpress&logoColor=white)](https://www.vcloudinfo.com/2026/07/teslamate-road-trip-reports-codex.html). | `rest_command.bearclaw_*`, `event: tesla_road_trip_active`, `sensor.bearclaw_status_telemetry`, `sensor.bearclaw_scheduled_job_health`, `sensor.joanna_onenote_kb_health`, `sensor.joanna_*`, `binary_sensor.joanna_*`, `automation.bearclaw_*`, `script.notify_engine`, `script.send_to_logbook` | | [![YAML source: telegram_bot](https://img.shields.io/static/v1?label=YAML&message=telegram_bot&color=lightgrey&logo=github&logoColor=181717)](telegram_bot.yaml) | Legacy Telegram transport marker for BearClaw; the shared `joanna_send_telegram` helper now forwards through the codex_appliance direct Telegram API. | `rest_command.bearclaw_telegram_send`, `script.joanna_send_telegram` | diff --git a/config/packages/tugtainer_updates.yaml b/config/packages/tugtainer_updates.yaml index 7c60f9db..0c015e28 100644 --- a/config/packages/tugtainer_updates.yaml +++ b/config/packages/tugtainer_updates.yaml @@ -11,7 +11,8 @@ # Notes: Available-only reports stay silent in HA; Updated, Failed, and Rolled-back reports remain visible. # Notes: Fires update events independently when a report has both Available and Updated sections. # Notes: Fires `tugtainer_home_assistant_core_updated` when `### Updated:` includes Home Assistant. -# Notes: Every Available report dispatches Joanna; Tugtainer owns report scheduling and de-duplication. +# Notes: Every Available report dispatches Joanna; include still-pending available(notified) items in the Tugtainer notification template. +# Notes: A daily 07:30 check dispatches Joanna on the appliance to verify rollout completion and protected control-plane updates. # Notes: Home Assistant changelog dispatch uses core-YYYY.M URL format from parsed/fallback version. # Notes: Home Assistant is auto-update eligible after its Tugtainer pre-update config check; no per-release approval is required. # Notes: Portainer image-update entities are auto-disabled because Tugtainer/Compose own container rollouts. @@ -208,44 +209,78 @@ automation: - alias: "Tugtainer - Dispatch Joanna For Available Updates" id: tugtainer_dispatch_joanna_for_available_updates - description: "Dispatch Joanna for every Tugtainer Available report." + description: "Dispatch appliance Joanna for Available reports and daily post-rollout verification." mode: queued trigger: - platform: event event_type: tugtainer_available_detected + id: available_report + - platform: time + at: "07:30:00" + id: post_rollout variables: - title: "{{ trigger.event.data.title | default('Tugtainer update') }}" - message: "{{ trigger.event.data.message | default('Update event received') }}" - event_type: "{{ trigger.event.data.event_type | default('info') }}" + post_rollout: "{{ trigger.id | default('available_report') == 'post_rollout' }}" + title: "{{ 'Tugtainer post-rollout verification' if post_rollout else trigger.event.data.title | default('Tugtainer update') }}" + message: >- + {{ 'Verify the completed 07:00 rollout and every still-pending protected updater component from live inventory.' + if post_rollout else trigger.event.data.message | default('Update event received') }} + event_type: "{{ 'scheduled_verification' if post_rollout else trigger.event.data.event_type | default('info') }}" trigger_context: "HA automation tugtainer_dispatch_joanna_for_available_updates (Tugtainer - Dispatch Joanna For Available Updates)" action: - service: script.send_to_logbook data: topic: "DOCKER" message: >- - Tugtainer reported Available container updates. Joanna dispatch requested. + {{ 'Tugtainer daily post-rollout verification' if post_rollout else 'Tugtainer reported Available container updates' }}. + Appliance Joanna dispatch requested. - service: script.joanna_dispatch data: trigger_context: "{{ trigger_context }}" source: "home_assistant_automation.tugtainer_dispatch_joanna_for_available_updates" - summary: "Tugtainer reported container updates scheduled for automatic rollout" + summary: >- + {{ 'Verify Tugtainer rollout completion and protected updater maintenance' + if post_rollout else 'Tugtainer reported container updates scheduled for automatic rollout' }} entity_ids: - "input_datetime.tugtainer_last_update" diagnostics: >- title={{ title }}, event_type={{ event_type }}, + post_rollout={{ post_rollout }}, message={{ message }} request: >- - Review the Tugtainer report for containers listed under Available. + Review the report and the complete live Tugtainer inventory, including + pending items omitted from the report because they were already announced. + Reconcile registered enabled hosts against the current environment's + live Docker hosts, including docker_13; identify missing agents or + untracked containers rather than checking registered hosts alone. Eligible containers, including Home Assistant core, are scheduled for Tugtainer's 07:00 automatic rollout and do not require per-release approval. When Home Assistant is listed, confirm its pre-update config check, HTTP healthcheck, and rollback safeguards remain configured; do not race the scheduled rollout with an early manual recreation. - Protected Tugtainer control-plane containers cannot self-update; update - those one host at a time through their owning Compose definitions when - low-risk. Verify the scheduled and manual updates, reconcile Tugtainer's - inventory, and report any failures or rollbacks. + Protected Tugtainer control-plane containers cannot self-update. + Inspect the manager, every agent, and socket proxies independently of + report contents; update pending components one host at a time through + their owning Compose definitions when low-risk. Retain prior images and + verify compatible manager/agent APIs and application health afterward. + {% if post_rollout %} + This is the daily post-rollout check. Verify the actual terminal update + jobs, database readiness, backup exporters, and affected application + endpoints. Investigate every remaining pending item; distinguish an + intentional opt-out or delay from a failed, rolled-back, or missed + rollout. Healthy updates first detected after today's rollout belong + to the next scheduled window; do not treat them as missed updates. + Recover eligible failed services through their owning Compose + definitions when safe, without racing any active update job. Preserve + database major versions, volumes, local-build opt-outs, and Home + Assistant's config-check, HTTP healthcheck, and rollback gate. + {% endif %} + Refresh Tugtainer's real inventory after manual updates. If an + authenticated refresh API is unavailable, use its native check worker + after loading the complete model registry and SettingsStorage, without + resending notifications and retriggering this dispatch; do not + silently leave stale update flags. Report newly repaired failures and + verified blockers; keep unchanged healthy results internal. - alias: "Tugtainer - Dispatch Joanna For Home Assistant Core Digest" id: tugtainer_dispatch_joanna_for_home_assistant_core_digest