The migration's already-present branch called prefs.remove(key) and threw
the prefs copy away when drift already held the key. A build that writes to
SharedPreferences (a non-drift build, or any in-between test build) collects
new messages there, so the next drift run silently gapped that history out.
It cost 574 real channel messages, recovered from backups.
Union the prefs copy into drift by element identity (messageId, else
publicKey, else canonical JSON), keeping drift's live copy on a collision and
appending prefs-only elements. Verify the merged write before removing the
source. New `merged` counter in the report.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up to #348 (now merged): dev has drift, but release-signed.yml
had no build_runner step, so the next v* release would fail its
android/windows/linux builds on the missing offband_database.g.dart.
Adds `dart run build_runner build --delete-conflicting-outputs` after
`flutter pub get` in the three jobs that build (android, windows,
linux). gate and release don't compile Dart, so no codegen there.
Matches the step #348 added to build.yml/flutter_dart/deploy-web. This
one couldn't ride #348 because release-signed.yml had diverged on dev
(#342/#346) and editing the stale copy caused a merge conflict — see
#348 discussion.
Part of epic #312. Agent: SapphireCompass (session 8d755b5e)
First live signing run (#345) failed at APK cert verification. The APK
signed correctly, but the label parse (grep "Signer #1 certificate
SHA-256 digest" | awk) returned empty on the runner's apksigner, whose
output labels that line differently than the local build-tools. Empty
!= expected produced a false mismatch. The release correctly did NOT
publish (fail-safe held).
Now extracts by shape: the cert SHA-256 is the only 64-hex string in
the output (SHA-1 is 40, MD5 is 32), so grep -ioE '[0-9a-f]{64}' is
label-independent. Verified against a real signed APK (e7da8cd5…).
Echoes the raw apksigner + keytool output for diagnosability, and
hardens the AAB parse the same way (it was skipped last run, so its
runner format is still unconfirmed — the echo will show it).
Part of epic #312. Agent: SapphireCompass (session 8d755b5e)
My previous commit added the codegen step to this branch's OLD
android-signed release-signed.yml. But dev has since replaced that file
with a 5-job pipeline (#342/#346), so editing the stale copy made both
sides diverge -> PR #348 went CONFLICTING and CI stopped running.
release-signed.yml only runs on v* tags, never on this PR, so it does
not belong in this PR at all. Restored to the merge-base version so it
merges cleanly (dev's version wins). The codegen for the release
pipeline will be added to DEV's release-signed.yml in a follow-up, AFTER
this merges (build_runner isn't a dep on dev until then).
The PR-relevant codegen (build.yml, flutter_dart.yml, deploy-web.yml) +
the .g.dart untracking are unchanged and still green the PR.
For #335 / PR #348. Agent: SapphireCompass (session 8d755b5e)
Replaces the stopgap (53f87f5) that committed
lib/storage/drift/offband_database.g.dart to green CI. Restores the
repo's "generated files are not committed" convention:
- dropped the .gitignore negation; the .g.dart is gitignored + untracked
(git rm --cached, working copy kept)
- added `dart run build_runner build --delete-conflicting-outputs`
immediately after `flutter pub get` in every Dart job: flutter_dart
(analyze), build.yml (all 6), release-signed (android-signed),
deploy-web (before build_pipe)
- keeps the drift_dev 2.34.0 / drift 2.34.2 / build_runner ^2.15.1 pins
from the stopgap for deterministic generation
Verified locally on the bench toolchain (3.44.1): deleted the working
.g.dart, ran build_runner from scratch (regenerated cleanly), confirmed
the generated file passes `dart format --set-exit-if-changed` and full
`flutter analyze --fatal-infos --fatal-warnings` is clean with the file
generated (not tracked).
For #335 / PR #348. Agent: SapphireCompass (session 8d755b5e)
CI failed on every job: offband_database.g.dart (drift's build_runner output)
was gitignored and no workflow runs build_runner, so analyze and all six
platform builds hit "Target of URI hasn't been generated". drift is the first
code in this repo needing codegen, which is why dev's CI never had to.
The workflows are not matrixed - six separate build jobs plus analyze, web
deploy and release-signed, each with its own `flutter pub get`. Adding a
codegen step to all of them is wide and easy to miss one. Instead the generated
file is committed (drift supports this): one file, works across every job and
workflow with no CI edits, and reproducible.
drift_dev is pinned (2.34.0) alongside the already-pinned drift, so a future
regeneration produces the same file rather than drifting from the committed
copy. Regenerated against the pinned versions before committing.
Follow-up worth having: a CI check that regenerates and diffs, so a schema
change without regen fails loudly rather than shipping a stale .g.dart.
Adversarial review (standards#145) of the storage/migration surface found four
real defects, all data-integrity. All confirmed against the code and fixed; no
false positives.
BLOCKER - concurrent save race. saveChannelMessages / saveMessages are
read-modify-write with an await gap, so two saves to the same key raced and the
second clobbered the first, silently losing messages (e.g. a message and its
delivery ack arriving together). BlobStore now provides synchronized(key, ...),
a per-key operation chain; every RMW - merge-save, remove, and the load-path
legacy migration - runs through it. Different keys stay concurrent. New test
fires two concurrent saves and asserts the union survives.
BLOCKER - non-atomic DB relocation. The documents->support move copied and
deleted each file (.sqlite/-wal/-shm) in turn, so a failure after the main file
moved stranded the DB across two locations and corrupted it. Now copies all
files, verifies each by size, and only then deletes the sources; on any failure
it rolls back the destination and leaves the original intact.
MAJOR - load-path legacy migration raced save. The #194 index->PSK adoption did
a blind write to the PSK key that could clobber a save that landed first. It now
runs under the key lock and MERGES (union) instead of overwriting, so both the
adopted history and any fresh message survive.
MINOR - channel merge key lacked a sender. Two senders posting identical text at
the same timestamp without a messageId would collide and lose one. The key now
includes the sender, matching MessageStore.
flutter analyze clean, dart format clean, 509 tests pass. Full Gemini log in
docs/llm-consultations/.
The gate introduced in this PR correctly FAILED on its own PR in 8s:
pubspec is rc.3, which had a CHANGELOG section but no notes files. That
is the gate working exactly as intended, demonstrated live.
Fix is the work we needed regardless: author rc.3's three per-audience
notes. This also seeds dev so every future dev PR passes the gate (the
current version now has all four files), and the rc.3 backfill branch
will carry these to build the actual release.
DRAFTS for owner review — the prose is mine, the judgement is Ben's.
Play blurb is 403/500 chars.
Part of epic #312, plan #321. Agent: SapphireCompass (session 8d755b5e)
Both findings from the adversarial review, verified before applying.
1. Draft-then-publish (was: create-then-upload). Creating the release
first made it publicly visible with zero assets until uploads
finished; a stalled upload or dead runner would strand a partial
public release — the exact failure this pipeline exists to prevent.
Now: create --draft → upload → edit --draft=false, which
`gh release create --help` documents as the correct pattern. Both
the create and retry branches end published, so a run retried after
a mid-way death can't get stuck as a draft. gh flags verified via
--help.
2. Verify the AAB certificate, not just the APK. The APK check was a
proxy; the AAB is what Play actually receives. apksigner can't read
an AAB, so this uses keytool -printcert -jarfile and pins the same
e7da8cd5… fingerprint. Verified locally: built a debug AAB, confirmed
the extraction command yields 765fb469… matching the known debug
fingerprint, so the check would correctly FAIL a debug-signed AAB.
Part of epic #312, plan #321. Agent: SapphireCompass (session 8d755b5e)
Today a tag builds and signs an APK/AAB but creates no release and
attaches nothing. That is the same partial-release failure mode as
rc.2/rc.3 (Play-only, no changelog). This closes it.
release-signed.yml becomes a complete release pipeline off a v* tag:
gate -> release-notes gate (backstop; fails in seconds)
android -> signed APK + AAB, cert-verified (environment-gated)
windows -> zipped Release build
linux -> tarred bundle
release -> create GitHub release, body = release notes, attach all four
Security invariant preserved: only the android job references the
release-signing Environment, so only it receives the keystore. The
windows/linux/release jobs never see it. Still no pull_request trigger.
The release job runs ONLY after every build passes, so a signing
failure yields no release rather than an empty one. Published directly
because the notes were reviewed in the cut PR.
Four-file notes gate (.github/scripts/release-gate.sh), run on every PR
(release-gate.yml) and as job one of the tag workflow:
CHANGELOG.md section for the pubspec version
release-notes/<v>.md non-empty -> GitHub release body
play/<v>.txt non-empty, <=500 chars (Play limit)
discord/<v>.md non-empty
play and discord are human-pasted; CI validates, never sends. Any
missing/oversize file fails the build, so a noteless release (rc.2/rc.3)
is unmergeable.
Gate script lives in .github/scripts/ because /scripts is in
.git/info/exclude (would not commit). Verified locally: fail surfaces
all four problems at once, pass is clean, 500 passes, 501 fails.
Backfills the combined rc.2/rc.3 CHANGELOG entry the two versions
shipped without.
release-notes/, play/, discord/ seeded with README conventions.
No ref input needed: the rc.3 backfill branch will carry these workflow
files, so tagging that branch runs them at that commit.
Part of epic #312, plan #321. Agent: SapphireCompass (session 8d755b5e)
The persisted store was being overwritten with the windowed in-memory list, so
any channel or DM with more than _messageWindowSize (200) messages lost
everything older than the newest 200 on the first save after load. Observed
live: Public went 232 -> 201 in one session on a build that already had the
#333 fix, so this was not the race - it was windowing truncating the store.
This is the slow-erosion cause behind the whole 566 -> 231 -> 206 -> 201
history.
saveChannelMessages / saveMessages now MERGE into the persisted set instead of
overwriting: upsert by message identity so older persisted messages are kept,
new ones added, and the in-memory copy wins for edits/reactions/status. If the
existing history fails to decode, the save aborts loudly rather than merging
into an empty base and truncating (SAFELANE 6).
Deletion is now an explicit path - removeChannelMessage / removeMessage - since
the app deletes individual messages. Routing delete through the merging save
would resurrect them; the connector's deleteChannelMessage / deleteMessage now
call the explicit remove.
Windowing stays for display and memory; it no longer dictates what is stored.
Tests: 250-message history survives a 200-window save; new message appends;
edit is captured not duplicated; delete does not resurrect. 508 pass, analyze
and format clean.
Cost: a save now re-reads and re-encodes the channel/contact history. Cheap
with drift's per-key writes (#335); a future append-only schema removes even
that.
drift_flutter's native default is getApplicationDocumentsDirectory(), which on
Windows resolves to the user's Documents folder - redirected into OneDrive on
most machines (verified: drift_flutter 0.3.1 connect.dart). A live SQLite file
syncing to OneDrive risks lock contention and corruption, and it is simply the
wrong place for app data.
The DB now opens in getApplicationSupportDirectory() (%APPDATA% on Windows),
the same place SharedPreferences already lives, so all app data sits together.
Existing installs already have a DB in the old location, so opening a fresh one
there would abandon their migrated history. _appSupportDatabaseDirectory()
relocates it once on first run: it moves offband_store.sqlite and its -wal/-shm
sidecars from the documents dir to the support dir before drift opens, and
never deletes a source without a successful copy. If relocation fails it is
logged loudly and a fresh DB is created, with the migration re-running from
SharedPreferences rather than silently losing anything.
Web is unaffected: path_provider has no web backend and drift ignores
databaseDirectory there, using OPFS/IndexedDB.
Adds `path` as a direct dependency (was transitive).
flutter analyze clean, tests pass.
The guard read pubspec.lock with a newline-sensitive regex, so it passed on
LF (CI) and failed on CRLF (Windows) for the same, correct assets. A guard
that is itself platform-fragile is worse than none. Normalise CRLF->LF before
matching.
Surfaced by the 290+306+335 integration merge, where the lockfile came through
with CRLF endings.
Rebased onto dev after #331 landed. Brings the signing workflow in line
with the rest of the matrix:
- Pinned to Flutter 3.44.1, matching the bench and every other
workflow. A release artifact in particular must not be built by
whatever `stable` happens to point at that day, which was the whole
argument for #331.
- Generates dart_defines.json and passes --dart-define-from-file, so
the signed release does not ship with a dead GIF picker.
- Cleanup step now removes dart_defines.json alongside the keystore
material, since it holds the Giphy key.
Verified that a job pinned to an environment still receives repository
secrets (precedence is org < repo < environment, environment winning on
a name collision), so the repo-scoped GIPHY_API_KEY reaches this job
while the keystore secrets stay environment-scoped behind the reviewer
gate.
Part of epic #312, plan #321 (Phase 5).
Rejecting only CN=Android Debug catches the #111 fallback but not a
DIFFERENT key being substituted, which breaks cross-channel updates
against Play just as badly. Now asserts the exact fingerprint.
Established from artifacts users have already installed, with no access
to the keystore password (reading a certificate off a signed APK needs
none):
release b55 (2026-07-06) e7da8cd5...
release b58 (2026-07-10) e7da8cd5...
local release build e7da8cd5...
Three independent sources agree, and the owner confirmed the same
strycher-personal.jks is enrolled for Play App Signing, so this is the
fingerprint that must hold for a GitHub download to update over a Play
install.
Found while establishing it: published release b59 (2026-07-20) is
DEBUG-SIGNED (765fb469, CN=Android Debug), unlike b55 and b58. That is
#111 occurring in production. Filed separately.
Part of epic #312, plan #321 (Phase 5).
CI has only ever produced debug-signed Android artifacts, because
build.gradle.kts:63-70 falls back to signingConfigs "debug" when
key.properties is absent, which is exactly the CI condition (#111).
Debug-signed builds cannot be installed over a Play install or over a
properly signed GitHub release, so users hit "App not installed".
Adds a separate workflow that signs with the real keystore.
Security posture, given this repo is PUBLIC and the keystore is the one
credential that cannot be replaced if leaked:
- No pull_request trigger. A PR, including from a fork, must never run
a job that can read these secrets.
- Pinned to the release-signing Environment, which carries a required
reviewer and only accepts main or v* tags. The signing secrets are
scoped to that environment, NOT to the repo, so a workflow that
omits the environment key cannot read them at all.
- Keystore decoded to disk only for the build, removed in a step with
if: always() so a failed build leaves nothing behind.
- Secrets passed via env: and printf'd into a file, never placed on a
command line where they would appear in process listings, and never
echoed.
Includes an apksigner check that fails the job if the built APK carries
CN=Android Debug. A green build alone does not prove release signing,
because the gradle fallback is silent; that check is what makes this
verifiable rather than assumed.
The keystore is PKCS12 despite its .jks extension, and PKCS12 cannot
carry a key password distinct from the store password, so one secret
correctly populates both fields.
Part of epic #312, plan #321 (Phase 5).
Publishes the Flutter web client. offband.org stays the Hugo marketing
site in OffbandMesh/offband-site, a separate Pages project, untouched.
push to main -> --branch=main -> production -> offband.app
push to dev -> --branch=dev -> dev tier -> dev.offband.app
Direct upload only. Cloudflare's dashboard "Connect to Git" must not be
enabled for this project; it would create a second git-integrated
project racing this workflow. Matches how offband-site deploys.
Because direct-upload projects cannot configure production branch
controls in the Cloudflare dashboard, the "never auto-deploy arbitrary
branches to prod" requirement from #122 is enforced by the workflow's
trigger list instead. That is the safety mechanism, and it is better
placed here than in dashboard state: reviewable and version controlled.
Builds via build_pipe rather than a plain `flutter build web`, because
build_pipe appends ?v=<version> to the bootstrap and manifest files.
Without it, browsers serve stale bundles after a deploy.
Two fixes folded in:
- pubspec.yaml build_pipe build_command did not pass
--dart-define-from-file, so a deployed site would have shipped a
dead GIF picker even though #331 wired the key into build.yml. The
deploy path and the CI path used different build commands.
- Retired the inherited zjs81 Workers deploy: removed deploy.yml and
wrangler.toml (which targeted THEIR account's "meshcore" project)
and dropped the now-dead `deploy` script from package.json. A root
wrangler.toml carrying Workers config can also interfere with
`wrangler pages deploy`.
Verified locally by running the real build with the bench toolchain:
build_pipe accepts the new flag, cache busting is applied
(flutter_bootstrap.js?v=e2019703...), output is 46 files with a 10.2 MB
largest asset (Pages limits are 25 MiB per asset and 20000 files), and
the Giphy key is confirmed compiled into main.dart.js.
Not verifiable locally: whether the repo's CLOUDFLARE_API_TOKEN carries
Pages:Edit scope. The workstation token is DNS-only and 403s on the
Pages API. A 403 on first deploy would indicate wrong scope; it fails
safely.
Part of epic #312, plan #321 (Phase 4).
Wires the migration into startup and points the four bulk stores at drift.
SharedPreferences keeps the settings, which is what it is for.
Startup runs the migration after prefs are up and BEFORE any store reads, and
awaits it. Letting stores race a half-finished migration is precisely the
shape of #333, where a storage path silently chose the wrong key and 566 real
messages read as empty.
Converted: message_store, channel_message_store, contact_store,
contact_discovery_store. All four now have ZERO prefs get/set for bulk data.
Three safety properties, deliberately built in:
1. readWithPrefsFallback - if a key is somehow not in drift, the prefs copy is
still served rather than reading as empty. It logs a WARNING when it fires,
because a fallback during normal operation means the migration is
incomplete and someone needs to know. Silence here is what made #333 look
like data loss.
2. deleteEverywhere / keysWithPrefix span BOTH backends. A clear that only
removed the drift row would leave a pre-migration prefs copy to reappear
through the fallback, resurrecting deleted history.
3. The legacy-key migrations inside the stores now check both backends and
only touch prefs when a legacy key actually exists. This also carries the
#306 fix into this branch: the unconditional prefs.remove was still present
here, since this branched from dev rather than from the #306 work.
Verified: analyze clean, format clean, 504 tests pass, Windows and web both
build. Migration rehearsed earlier against a copy of a real 7 MB store: 69
keys, 5.21 MB, zero failures.
NOT yet run against live data - that happens on first launch of this build,
and the owner should have a prefs backup before that.
Moves message history, contacts and discovered contacts out of the settings
store. Settings stay in SharedPreferences, which is what it is for.
Ordering is the whole safety argument: WRITE, VERIFY BY READING BACK, and only
then remove the source. #333 was a storage path that chose a key silently and
made 566 real messages read as empty; deleting before verifying would make
that class of mistake permanent instead of cosmetic. On any failure the source
is left intact and the error is logged - never a silent drop (SAFELANE 6).
Idempotent by construction: a key already present in drift is not overwritten,
so re-running is a no-op. If an older build re-writes a migrated key into
prefs, the migrated copy wins and the stale prefs copy is discarded rather
than promoted.
Rehearsed against a COPY of a real 7 MB store, as the plan required before
touching live data:
REHEARSAL: 69 migrated, 0 failed, 5.21 MB, 69 bulk keys expected
Every key checked for exact length, confirmed removed from prefs, and every
settings key confirmed untouched. The live store was never opened.
Two things the tests caught that review would not have:
1. getString THROWS on a non-string value rather than returning null, so a
non-string under a bulk prefix was counted as a migration FAILURE. It now
type-checks with prefs.get() and skips. Alarming falsely is its own bug.
2. The `contacts` prefix was checked against the real store rather than
assumed: it matches only the 6 bulk contact blobs, and correctly does NOT
match contact_unread_count*.
Not yet wired into app startup - that is the switchover, and it is deliberately
a separate commit so this can be reviewed on its own.
flutter analyze clean, dart format clean, 504 tests pass.
Owner asked whether the web assets must be re-downloaded on every drift bump.
They do not - the version can be pinned - but doing that safely needed three
changes, because the failure mode is silent.
1. drift and drift_flutter are pinned EXACTLY (2.34.2 / 0.3.1), no caret. The
committed web assets are built for a specific drift release, so a caret
would let pub resolve a newer drift than the shipped binaries and break the
web build with no signal at compile time.
2. pubspec.lock is now committed (removed from .gitignore). It was ignored,
which for an application means resolution can differ between machines and
CI. That undermines the pin: the assets are matched against the RESOLVED
version, so resolution has to be reproducible. Owner-authorized, and it is
a repo-wide change rather than a storage-only one.
3. New test asserting the shipped assets match the resolved drift version,
plus that both files exist and the wasm really starts with the WebAssembly
magic bytes. Verified in both directions: it passes when matched and fails
loudly with re-download instructions when skewed.
Without (3) a stale asset compiles, deploys, and only fails in the user's
browser - the exact silent-failure class SAFELANE 6 forbids and that produced
#333 earlier.
Upgrading drift is now a deliberate step: bump the pin, re-download both
assets, update web/drift_assets.version. The test fails until you do.
flutter analyze clean, tests pass.
Step 1 gate, web half. Building is not evidence, so this was run in a browser.
Assets: sqlite3.wasm (748 KB) and drift_worker.js (355 KB), both taken from
the SAME drift-2.34.2 release so they are built against each other. Validated
before use - the wasm has correct WebAssembly magic (0061 736d) and carries
SQLite 3.53.3; the worker is genuine compiled Dart JS.
Note the filename: the release ships `drift_worker.js`, not the
`drift_worker.dart.js` the docs name. The database URI was corrected to match,
which would otherwise have been a silent 404 at runtime.
Browser result, served over HTTP with COOP/COEP and Content-Type:
application/wasm:
Using WasmStorageImplementation.opfsLocks
PROBE-OK open+write+read
PROBE-OK 6MiB round-trip (localStorage cap is 5MiB)
drift selected OPFS, the top storage tier, and round-tripped 6 MiB - data the
current localStorage-backed web build physically cannot hold. No exceptions.
Adds web/_headers for Cloudflare Pages: forces application/wasm on the wasm
asset and sets COOP/COEP for cross-origin isolation. Verified to ship into
build/web. Without COOP/COEP drift still works, falling back to IndexedDB,
which is slower but still unbounded next to localStorage - a performance tier,
not a correctness requirement.
web_probe/ is a standalone entrypoint that exercises ONLY the drift stack, so
a pass or fail is unambiguous without BLE and the rest of the app in the way.
It is not part of the app build.
Still no user data touched and no store switched over. The migration is the
next step.
flutter analyze clean, dart format clean, 497 tests pass.
Step 1 of #335, deliberately before any data migration: prove the store works
before trusting it with message history.
Adds drift + drift_flutter, a key/value StoredBlobs table mapping 1:1 onto the
existing SharedPreferences keys, and verifyReadWrite() as a runtime probe.
Key/value rather than relational tables on purpose: it maps directly onto the
current store interfaces, so callers do not change and the migration can be
verified row-for-row against the old keys. Relational schemas come later, once
this is proven and querying is actually wanted.
Verified here:
- 3 tests pass, including a 6 MiB round-trip - larger than the 5 MiB
localStorage cap that makes the web build impossible today
- flutter analyze clean
- Windows release build succeeds
- Web release build compiles
Two findings from the gate that a plan-on-paper would have missed:
1. sqlite3_flutter_libs 0.6.0+eol is a DEPRECATED no-op. From sqlite3 v3.x it
is unnecessary and drift_flutter already covers it, so it is not a direct
dependency here.
2. The web build COMPILES BUT WOULD FAIL AT RUNTIME: drift needs sqlite3.wasm
and drift_worker.dart.js shipped in web/, and neither is present. Per drift
docs they are downloaded from GitHub releases, version-matched to
pubspec.lock (drift 2.34.2, sqlite3 3.5.0), and the server must serve .wasm
as Content-Type: application/wasm. Not done here - downloading files needs
explicit human approval.
Also noted: the drift_dev CLI does not compile at these versions
(allSchemaEntities missing from the drift3_preview GeneratedDatabase), so its
asset tooling is unavailable. build_runner code generation is unaffected.
No user data is touched. No store is switched over. The migration is the next
step and remains gated on the web assets question.
Measured, not guessed. Cumulative per-code frame timing named the owner:
frame handler cumulative (2002ms total sync): code=3 2002ms/105calls
code=3 is RESP_CODE_CONTACT, and it accounted for ALL of the synchronous
frame time at ~19ms per contact. That is why a per-call 100ms threshold never
fired: no single call is slow, the volume is.
The 19ms is _handleContact calling notifyListeners() for every contact. Each
notification is a synchronous full-tree rebuild, and each rebuild walks the
contact list, so the cost grows with the address book. The channel handler
directly alongside already guards its notify with _isLoadingChannels; the
contact handler had no equivalent.
Notifications are now throttled to one per 250ms while _isLoadingContacts is
set, and remain immediate outside a pull so single adverts still update live.
Throttled rather than suppressed so the sync progress bar keeps advancing. The
existing notifyListeners() when the pull completes still fires, so the final
state cannot be stale.
Scope of what this accounts for, stated honestly: measured synchronous frame
work was ~2s per report window and the pull showed ~4s total, against a ~30s
observed freeze. This removes the largest measured contributor. If the stall
persists, the remainder is outside the frame handlers and the cumulative
counters will show frame time has dropped, which narrows it further.
flutter analyze clean, dart format clean, 494 tests pass.
The conversation-load metric was misread, and nearly caused a fix to the wrong
thing. It reported store=30138ms across 300 contacts, which looked like
loadMessages costing ~100ms each. It is not:
- The device-wide fallback key `messages_<dev>` does not exist on the
reporting machine, and only 11 per-contact message keys exist in total, so
loadMessages does three in-memory map lookups and returns empty.
- storeWatch spans an `await`, so it measures WALL time including event-loop
queueing, not work. merge, which is synchronous and has no await, measures
0ms - consistent with the isolate being busy elsewhere while those awaits
wait their turn.
So the 30s is ~300 awaits each waiting for a busy isolate, not 30s of storage
work. The metric label now says so, rather than inviting the same misreading.
The per-call 100ms threshold cannot see the actual shape here: a handler that
takes 40ms and runs 234 times owns ~9s of isolate time and never trips it.
Frame handlers now accumulate synchronous time per frame code and report the
top codes by total, which names the owner regardless of per-call cost.
Diagnostics only, no behaviour change.
flutter analyze clean, dart format clean, 494 tests pass.
The stores violated the no-silent-failures rule, and that is why tonight's
diagnosis took as long as it did. Two classes:
1. Ten catch blocks swallowed decode errors and returned an empty collection.
`catch (_) { return []; }` means a corrupt or unreadable store presents to
the caller as "no data", so the user sees no contacts, no channels or no
history with nothing anywhere explaining why. Indistinguishable from data
loss. Every one now logs what failed, with the store named, before
returning.
2. ChannelMessageStore._storageKey fell back from the PSK-identity key to the
legacy slot-index key without a word. That fallback is correct before a
channel list exists, but once history has been migrated to the PSK key it
reads a DIFFERENT key and returns empty. That is exactly #333: a user's 566
Public messages were intact on disk while the app showed nothing, with no
error, no parse failure and no log line to follow.
It now warns when it falls back while a resolver is installed, since that
combination specifically means the channel list was not loaded first.
Had either of these been loud, #333 would have been a one-line log read
instead of a multi-hour investigation that looked like data loss.
flutter analyze clean, dart format clean, 494 tests pass.
The startup loads are now clean (all steps ~0ms, down from 37792ms), but the
contact pull still stalls in multi-second bursts: 8.0s and 7.7s at +25s..+52s
on the last run, matching the reported freeze from ~15s to ~49s.
No frame handler exceeds 100ms, because the work is not in one.
_loadMessagesForContact is fired unawaited from the contact handler, so it runs
after the handler returns and escapes that timing entirely.
This times it directly and reports cumulative store time versus merge+notify
time every 25 contacts, so the next run says which half is responsible rather
than requiring another guess. Two candidates it will discriminate:
- store: prefs read + jsonDecode per contact
- merge+notify: the per-contact notifyListeners, i.e. one full widget tree
rebuild per contact, each of which walks the contact list
Diagnostics only, no behaviour change.
flutter analyze clean, dart format clean.
The previous commit fixed loadSmazEnabled and killed the post-connect stall
(startup-load channelSettings: 37792ms -> 0ms, measured twice). The contact
pull still choked, in multi-second bursts, each landing immediately after an
"Added new contact" line.
No frame handler exceeded the 100ms threshold, because the work is not in a
handler: the contact handler fires _loadMessagesForContact unawaited, so it
runs after the handler returns and escapes that timing.
message_store.loadMessages had the same unconditional prefs.remove(oldKey),
and it runs ONCE PER CONTACT during a pull. On Windows every prefs mutation
rewrites the entire file, so a 234-contact pull meant 234 full multi-MB writes
for legacy keys that do not exist.
A sweep found the identical pattern in six more stores: contact_store,
unread_store, channel_store, community_store, contact_group_store and
channel_order_store. Those run once per load rather than per item, so they are
far cheaper, but it is the same bug and they are fixed the same way.
In every case the removal now happens only when a legacy key actually exists,
paired with the write that migrates it, so the cleanup still occurs on a real
migration.
flutter analyze clean, dart format clean, 494 tests pass.
Back at the root was calling SystemNavigator.pop(). That is not "background
the app": on Android it calls finish() on the activity, which tears down the
Flutter engine and drops the radio connection. Backing out and reopening
therefore landed on a disconnected radio needing a fresh connect. Wrong call
for the job, and worse than the behaviour it replaced.
Backgrounding without finishing needs moveTaskToBack, which has no Flutter
equivalent, so it goes over a method channel:
- MainActivity exposes meshcore_open/app_lifecycle with a moveTaskToBack
method, alongside the existing USB channel.
- AppBackgrounder wraps it and is a no-op off Android, where programmatic
backgrounding either does not apply (desktop windows close by their own
chrome) or is forbidden (iOS). On those platforms back at the root stays
unhandled rather than doing something destructive.
- AppShell awaits it instead of calling SystemNavigator.pop().
The activity stays alive, so the connection survives and reopening returns to
where the user was.
flutter analyze clean, dart format clean, 469 tests pass.
The scanner's auto-entry was gated on activeTransport == bluetooth, so the
dead end fixed in the previous commit still applied to USB and TCP: a user
landing on the scanner with a live serial or network connection stayed stuck,
and pressing Connect did nothing, since there was nothing left to connect.
Both are real deployment paths, so the gate is removed and the check is now
transport-agnostic.
The first-connect path for USB and TCP is unaffected. Those screens are pushed
on top of the scanner and navigate themselves with pushReplacement; while they
are connecting they are the current route, so the scanner's isCurrentRoute
guard is false and it does not also push. That guard, not the transport check,
is what prevents a double navigation.
flutter analyze clean, dart format clean, 469 tests pass.
Two bugs, reported together from an S25 FE test.
**Back popped to the scanner.** The previous commit made back pop whenever the
route could be popped. The primary views sit on top of the scanner, so from
the channel list back landed on the radio-connect screen. That is a regression
I introduced: the `PopScope(canPop: !isConnected)` I removed existed to
prevent exactly this, and I removed it without establishing why it was there.
Back now distinguishes the two cases. A primary view (one carrying the bottom
bar) hands back to the OS and backgrounds the app. A detail screen (a channel
chat, which has no selectedIndex and is genuinely pushed) pops to its list.
Reaching the scanner is what Disconnect is for, not what Back is for.
**The scanner was a dead end while connected.** It only routed into the app on
a connection transition, behind a one-shot flag that never reset while
connected. So once showing with a live connection it stayed there, and
pressing Connect did nothing, since there was nothing left to connect. It now
routes in whenever it is the visible route and a connection is live, guarded
against double-push by a dedicated flag rather than reusing
_changedNavigation, which separately gates the disconnect-on-dispose cleanup.
Known limitation, pre-existing and unchanged: this auto-entry is bluetooth
only. USB and TCP connect from their own screens, which navigate themselves.
flutter analyze clean, dart format clean, 469 tests pass.
Owner hardware test on an S25 FE and Windows turned up five issues. Three of
them shared one root cause.
**Map: always show names.** New mapAlwaysShowNames setting, toggled from the
map panel. Names were shown only past zoom 14; the override wins at any zoom.
Evaluated at build rather than only on camera movement, so flipping the switch
repaints immediately instead of waiting for the next pan.
**Panel footer on every screen.** Disconnect and Settings now also appear
inside a channel. The footer is app-level, so it belongs anywhere the panel
is, not just the primary views.
**Back button (three reported symptoms, one cause).** Channels, Contacts and
Map each wrapped themselves in `PopScope(canPop: !isConnected)`, so while
connected the system back press was swallowed whole. That explains all three:
an open drawer would not close, Android would not background the app, and on
Windows the pinned layout has no Scaffold drawer, so the app bar auto-implied
a back arrow whose press PopScope then discarded - a button that visibly did
nothing.
AppShell now owns back handling for every screen that uses it, in order:
close an open drawer, else pop the route (a pushed chat returns to its list),
else hand back to the OS via SystemNavigator.pop() so Android returns to the
home screen or the previous app. The three per-screen PopScope wrappers are
removed. AppShell becomes stateful to hold the scaffold key this needs.
flutter analyze clean, dart format clean, 469 tests pass. Not yet re-tested on
hardware.
Epic D. The Map view has no list, so its nav panel was empty. It now carries
the map's own layer controls.
Those six toggles (chat nodes, repeaters, other nodes, discovery contacts,
guessed locations, overlaps) previously existed only in a modal behind the
floating button, so filtering the map meant covering the map with a dialog to
do it. In the panel, and especially pinned on a wide screen, they stay visible
and the map updates underneath as they are flipped.
Both surfaces read and write the same AppSettings, so a change in one is
reflected in the other. The floating-button dialog is deliberately left in
place: on a phone it is fewer taps than opening the drawer, and the panel is
the pinned-desktop path. If that duplication is unwanted, removing the dialog
is a one-line follow-up.
No new settings or storage; the existing setters were reused unchanged.
flutter analyze clean, dart format clean, 469 tests pass. Not yet run on
hardware.
Epic C, rescoped. The original plan was to eliminate the overflow menu and
move everything into the drawer. An inventory showed that premise was wrong:
the ellipsis is six different menus sharing an icon, and most of what they
hold is screen-level (force flood mode, path management, delete all
discovered). Those cannot live in a global nav panel, which has no notion of
which contact or channel is meant.
Only Disconnect and Settings are app-level, which is exactly why they were
duplicated across three screens. Those move; everything else stays put.
- AppShell gains optional onDisconnect / onSettings, rendered as a pinned
footer in the nav panel. Disconnect keeps the error colour it had as a red
menu entry, since it drops the radio connection.
- Channels, Contacts and Map pass both and drop those two menu entries.
- Map's overflow menu is removed entirely, having nothing left.
- Channels keeps its menu only for Manage Communities, which was already
conditional on having joined a community.
- Contacts keeps its menu for Discovered contacts.
- No detail screen menu is touched.
Pinned on a wide screen, Disconnect and Settings are now visible without
opening anything.
flutter analyze clean, dart format clean, 469 tests pass. Not yet run on
hardware.
The connect freeze is a legacy-key migration that writes even when there is
nothing to migrate.
loadSmazEnabled called prefs.remove(oldKey) unconditionally whenever the
scoped key was absent, which is the normal case for a channel that has never
had the setting changed. On Windows shared_preferences re-serialises and
rewrites the ENTIRE prefs file on any mutation, so each of those no-op removes
cost a full multi-MB write. Repeated across every channel slot on every
connect, that is tens of full-file rewrites back to back.
Measured with the instrumentation from the previous commit, on the reporting
device (6.9 MB prefs, ~42 channel slots):
startup-load channelOrder 0ms
startup-load contactCache 12ms
startup-load channelSettings 37792ms <-- here
startup-load cachedChannels 0ms
startup-load channelMessages 21ms
startup-load unreadState 0ms
startup-load discoveredContacts 11ms
No frame handler exceeded 100ms, confirming the stall was local storage work
and not radio traffic.
The remove now happens only when a legacy key actually exists, alongside the
write that migrates it. contact_settings_store had the identical pattern per
contact and is fixed the same way.
This also explains the platform gap in the original report: Android's backend
batches into native storage, so the same code costs almost nothing there,
while every no-op remove on Windows is a full file rewrite.
flutter analyze clean, dart format clean, 469 tests pass.
Two reported freezes: one shortly after connect, one during channel sync,
with the sync progress bar never repainting and then completing all at once.
That is the UI isolate being blocked, and the log already shows the
signature - a long silence followed by many frames sharing one timestamp,
which is queued input flushing after the loop frees up.
The previous fix (batching 600 per-contact notifications) moved the stall from
43.6s to 40.1s, which is noise. It was the wrong culprit, so this replaces
guessing with measurement:
- The post-SELF_INFO cache load is split into named steps, each timed and
logged as `startup-load <step> took Nms`. The step order is unchanged, and
cachedChannels still runs before channelMessages so PSK-keyed history
resolves.
- Every RX frame handler is timed, and any that occupies the isolate for more
than 100ms logs `slow frame handler: code=N blocked UI for Nms`. This names
the handler wherever it is, rather than requiring a guess about which one.
No behaviour change. Diagnostics only, kept as Perf-tagged logging because
this class of stall is worth catching again.
flutter analyze clean, dart format clean, 469 tests pass.
Diagnosed from the app's own log. After SELF_INFO the client went completely
silent for 43.7s, then flushed a burst of radio frames all sharing the
identical timestamp - queued while the event loop was blocked, not a slow
radio.
Cause: loadContactCache() looped over every cached contact calling
_ensureContactSmazSettingLoaded and _ensureContactCyr2LatSettingLoaded, and
each of those notified on completion. With 300 contacts that is 600
notifyListeners() calls in a burst, each rebuilding the whole widget tree, and
each rebuild itself walks the contact list. O(contacts^2) on the UI isolate.
Both loaders are now awaitable and take notify: false for bulk use.
loadContactCache awaits them all and notifies once.
Measured on the reporting device (radio 4f6585264e, TCP): 300 contacts, 1090
discovered contacts, 12 channels holding 1841 messages, 6.9 MB prefs file.
Log evidence: 43.7s stall between "Pulled battery" and the first channel
response, followed by same-millisecond frame delivery.
This is the Windows-visible freeze in #306. The platform asymmetry is
consistent with contact-list size rather than anything desktop-specific, so
whether Android is genuinely faster or simply has a smaller address book on
its paired radio is still open - #306 stays open pending a like-for-like
re-measure.
flutter analyze clean, dart format clean, 469 tests pass.
Channel history is keyed by the channel's PSK (#194), and the resolver set on
the store reads that PSK out of _channels. After SELF_INFO, loadCachedChannels()
and loadAllChannelMessages() were both fired without awaiting, so the messages
could load while _channels was still empty. The resolver then returned null,
_storageKey() silently fell back to the legacy slot-index key, and channels
read as empty even though their history was intact under the PSK key.
Silent because the fallback is a legitimate code path: there is no error, no
parse failure and no log line, just an empty list. It is indistinguishable
from data loss to the user.
Observed on a live radio: Public held 566 messages under
channel_messages_<dev>psk_8b3387e9..., and the app displayed nothing.
loadAllChannelMessages() now runs only after loadCachedChannels() completes.
The startup call in main.dart already awaits in the right order; it runs before
any radio is connected, which is the source of the "Public key hex is not set"
warnings at launch, and is harmless once this post-connect load is correct.
Was previously committed against #291 on the nav-redesign branch; split out
here under its own issue.
Owner added GIPHY_API_KEY as a repo secret, resolving the open decision
flagged on this issue.
The app reads the key at COMPILE time via
String.fromEnvironment('GIPHY_API_KEY') (lib/widgets/gif_picker.dart:25),
so CI builds produced without it ship a non-functional GIF picker that
displays a "needs a Giphy API key" hint. That was harmless while CI
output was discarded; it stopped being harmless in #322 when artifacts
became real deliverables.
Generates dart_defines.json from the secret and passes
--dart-define-from-file, matching how the project is built locally
rather than inventing a second mechanism.
Wired into the four jobs that produce artifacts users run: android,
linux, web, windows. NOT wired into ios and macos, which are compile
checks producing nothing installable, and where the key would have no
effect on whether compilation succeeds.
Uses jq on the Linux runners rather than hand-escaped printf, so the
value is JSON-escaped correctly whatever it contains. An earlier printf
attempt silently mangled an escape and broke the YAML, which is the
argument for using the tool instead of string juggling. jq is verified
preinstalled on the runner images (Ubuntu 22.04 has 1.6, 24.04 has
1.7.1). The windows runner uses PowerShell ConvertTo-Json, since jq is
not guaranteed there.
Note: the key is compiled into the binary and is therefore extractable
from any published APK. Putting it in CI does not widen exposure beyond
what shipping the app already does.
Part of epic #312, plan #321 (Phase 3).
Three different Flutter versions were in play and none was the bench:
build.yml channel: stable (floating, resolving to 3.44.6)
flutter_dart.yml channel: stable (floating)
deploy.yml pinned 3.41.2, with a comment claiming it matched
local development, which had not been true for months
bench 3.44.1 / Dart 3.12.1
A floating toolchain is tolerable for a compile check and not tolerable
once CI produces signed release artifacts (#330), so this lands first.
Pinned all 8 flutter-action usages to 3.44.1, matching the bench exactly,
so "works locally" and "works in CI" mean the same thing. Verified against
Flutter's release index that 3.44.1 is a published stable release
(2026-06-01, Dart 3.12.1) rather than assuming a locally-installed
version was publicly fetchable.
Corrected deploy.yml's misleading local-parity comment rather than
leaving it to mislead the next reader.
Note: CI's green history is on 3.44.6 or later, so pinning DOWN may
surface a break the newer toolchain masked. That is the intent. If it
breaks, fix the break or move the bench up deliberately; do not re-float
the pin.
GIPHY_API_KEY as a CI secret was the third item of Phase 3 in #321. It
is deliberately NOT included here, is not dropped, and is flagged on
#331 for the owner's decision, since it is a credential only they can
create.
Part of epic #312, plan #321 (Phase 3).
formatNotificationText used GifHelper.parseGif (Giphy only) while the chat
renders via resolveGifUrl (Giphy + allowlisted Tenor CDN). An allowlisted
Tenor GIF therefore rendered inline but still dumped a raw URL into the
notification tray.
Point both at resolveGifUrl so the tray summarises exactly the set the chat
displays. Off-allowlist URLs still pass through as plain text, pinned by test.
This is the same duplicate-detection drift that caused #284; closing it at the
source rather than shipping a known inconsistency.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pasted GIF links rendered as plain text because parseGif only matched the
compact forms. Adds GifHelper.resolveGifUrl, which maps any supported payload
to the URL that renders it, and widens coverage to the links people actually
paste.
Allowlist is deliberately narrow, Giphy and Tenor only. Off-list URLs return
null and stay plain tap-to-open links, never auto-fetched: auto-loading a
stranger's URL leaks the viewer's IP and enables tracking-pixel abuse, and
inline-rendering unmoderated hosts is a malware and inappropriate-content
vector. Tests pin the rejections, including a lookalike host
(giphy.com.evil.example) and a general image host (imgur).
Newly rendered:
- i.giphy.com/<id>.gif and .webp, the form a browser copy produces
- media.tenor.com and c.tenor.com direct assets
A tenor.com/view/<slug>-<id> PAGE url is deliberately NOT matched. The modern
Tenor CDN path uses an opaque hash that cannot be derived from the page id, so
resolving one requires the Tenor API and a key. Such a link stays plain rather
than silently rendering the wrong image. Supporting it is a separate decision.
Also removes the hardcoded 'https://media.giphy.com/media/$gifId/giphy.gif'
literal duplicated across all four render call sites. That duplication is the
same drift that caused the #284 notification regression, where a second
hand-rolled copy of GIF detection went stale.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Notifications summarised a GIF as "Sent a GIF" using their own hand-rolled
`^g:[A-Za-z0-9_-]+$` regex rather than GifHelper. #282 changed the payload to
`https://giphy.com/gifs/<id>`, which that regex does not match, so GIF
notifications regressed to dumping the raw URL into the tray.
Detection now goes through `GifHelper.parseGif`, matching how the adjacent
reaction case already delegates to ReactionHelper. That restores the summary
and covers every form the app can send, including replies (`@[Name] ` +
payload), which the old regex never matched even for legacy `g:<id>`.
Verified no other hand-rolled `^g:`/`^m:`/`^s:`/`^r:` payload regexes remain
outside their owning helpers, so this class of drift is closed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A capability-gated control that doesn't appear is indistinguishable from
a broken one. Diagnosing "the FEM LNA toggle is missing" on real hardware
required a rebuild, because the caps byte was never surfaced and the app
debug log records nothing in a release build unless logging is enabled.
Adds an "Offband capabilities" row to Device Info showing the raw caps
byte, firmware version code, and which gated features it grants, e.g.
"0x02 (v16) - block". That turned an unexplained missing toggle into a
one-look answer: firmware v16 running, FEM LNA bit clear, client correct.
Also logs the same values at device-info parse for anyone who does have
logging on.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes the client half against the firmware as-built (#298):
- Device-info offset 83 carries the FEM LNA state on v16+, appended
unconditionally (reads 0 on non-capable boards). parseFemLnaState is
length-guarded and returns null pre-v16. Byte presence signals firmware
version, not capability — the cap BIT gates the UI.
- 0xC3 reply handler adopts the reported value verbatim: firmware returns
post-apply hardware state, not an echo, so a refused write surfaces as
truth rather than a lie.
- setFemLna / requestFemLnaState no-op unless the capability bit is set,
so a non-capable radio never sees 0xC3 traffic.
- Radio Settings toggle rendered only when the bit is set, driven by
connector state via ListenableBuilder rather than local optimistic
state, so it always shows what the radio reports.
- errCodeUnsupportedCmd documented: a mis-gated request draws [0x01][0x01],
not [0x01][0x06]. Neither is 0xC3-prefixed, so no new error path.
Gating is on the bit alone, never model or version: firmware derives it
from a runtime FEM probe, so two Heltec V4s can legitimately disagree and
rak3401 reports false by design (its SKY66122 gates LNA and PA together).
13 tests. Not hardware-validated — firmware 0xC3 is still on a branch.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the client half of the Heltec V4 FEM LNA control surface at the
protocol level: 0xC3 CMD_OFFBAND_FEM_LNA (0x01 SET [value], 0x02 GET),
a 3-byte [0xC3][sub][value] reply parser, and the
OFFBAND_CAP_FEM_LNA = 0x04 capability gate.
Deliberately a fork-private command rather than a sixth byte on the
stock CMD_SET_OTHER_PARAMS (38): that frame is shared with upstream
MeshCore and is sent to every radio regardless of fork, so widening it
would perturb stock firmware. Nothing is emitted unless the capability
bit is set.
The gate checks the bit only, never model or version — firmware derives
it at runtime from the auto-detected FEM chip, so it is a per-unit
answer and two Heltec V4s can legitimately disagree.
PROVISIONAL: firmware owns the caps byte and has not yet confirmed 0x04
is free, and the 0xC3 setter is not built yet. Spec sent to
OffbandMesh/meshcore-firmware#298. Do not merge before firmware
confirms the shape as built.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Gemini adversarial review finding (minor, accepted). _pathDiag derived the
expected byte count from the connector's global _pathHashByteWidth, which is
the CONNECTED device's width, while each Contact now carries the width its
path was actually captured at.
Failure case: a contact discovered on a 1-byte net (3 hops, 3 bytes) viewed
while connected to a 2-byte device computes expected = 3 * 2 = 6 and logs
"bytes=3/6 TRUNCATED" for a complete, correct path. A false TRUNCATED in the
very diagnostic added to investigate truncation would actively mislead.
_pathDiag now takes the per-path width; all seven call sites pass
contact.pathHashWidth.
Refs #298, #309
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Strycher
