Two of five findings were real.
Log the downgrade refusal BEFORE the path lookup and the throw. onUpgrade runs
inside drift's beforeOpen, so it fires on whichever isolate issued the first
query, and whoever catches the exception decides what the user sees. A
downgrade that only stopped background work, with nothing in the log, would be
the quiet failure this branch exists to replace. The log line now lands first,
so the record exists however the exception is handled.
Strengthen the recovery bench. It asserted row COUNTS, so a converter
producing the right number of rows with wrong contents would have passed.
It now also checks that every identified blob record has a row carrying its
identity, and that no payload is empty, both derived from the raw blob rather
than from the converter under test so the check is not circular. Records with
no messageId are not identity-checked, because reproducing the `x:` fallback
key here would make it circular again; the count assertion still covers them.
Verified on a copy of the owner's database with the trap reconstructed (the
6,308-message channel removed from the marker plus one planted row the blob
does not contain): the key migrates, and all 6,308 identities are present.
Then verified the new assertion can FAIL, by planting an empty payload the
migration does not overwrite:
Expected: empty
Actual: WhereIterable<String>:['{}']
Three findings were rejected, with reasons in the PR body: the duplicate
merge-key collapse is pre-existing and unchanged by this diff, the claimed
migration/blob-save race cannot occur because of the re-check inside the
per-key lock, and the path_provider call is bounded and already proven.
Task: meshcore-open-lkp
Agent Mail: TealGlen
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Owner decision, 2026-10-22: downgrades stay unsupported, and must fail
deliberately rather than silently.
Before this, onUpgrade guarded on `from < 2`, so a database from a FUTURE
schema fell straight through it. The open succeeded, and drift then stamped
user_version back down to this build's 2, because it writes schemaVersion
whenever the versions differed. The database's only record of having been on a
newer build was destroyed, and a later run of that newer build would re-run
its own upgrade against tables already in their new shape.
onUpgrade now throws OffbandSchemaDowngrade when `from > to`, naming both
schemas, saying the upgrade is one-way, and giving the database path when it
can be resolved (path_provider has no web implementation, so the path is
best-effort and the message stands without it).
Not made to work, deliberately. Once a key is on rows its blob is frozen, so
an older build would read stale history and write into the blob while this one
reads rows, orphaning everything it received. Supporting a downgrade would
mean dual-writing the blob on every save, which is the cost #917 exists to
remove.
The two tests that documented the old silent behaviour now assert the refusal,
including that user_version is left at 3 on a file-backed database, which is
the whole point of throwing. Three more guard the upgrade direction: an equal
version takes neither branch, a fresh database still lands on 2, and a real
1 to 2 upgrade still carries its blobs across, since that is the path every
existing install takes.
Release-note disclosure and rollout staging are tracked separately.
Task: meshcore-open-3e1
Agent Mail: TealGlen
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four of the owner's store keys could never migrate to rows. Each held rows
whose merge_key is absent from its blob (1, 3, 7 and 5 of them), and the
readback gate compared the key's TOTAL row count against the number of
distinct records in the blob. Those two numbers can never be brought level
again: nothing removes the extra rows, and the frozen blob can never come to
contain them. Every attempt refused, for good.
His two largest channels were among them, 6,313 and 5,648 messages, so they
kept re-encoding 3.77 MB and 3.55 MB of history on every save. That is the
cost #920 removed and they were not getting it. No data loss: the blob stays
complete and authoritative for a refused key.
The gate asserted the wrong invariant. What it exists to catch is a record
from the blob failing to reach a row, which is a SUBSET question. Equality
became wrong the moment rows could legitimately hold messages the frozen blob
does not, which is true for every migrated key the instant a new message
arrives.
- messageMergeKeys(storeKey) reads back the stored identities, and the gate
now requires every deduped blob key to be among them. Rows beyond that are
logged and accepted rather than treated as corruption.
- The refusal log names how many records are missing and samples three, which
is more use than the two totals it printed before.
- The four keys need no healing migration: they migrate normally at next
access, pinned by "a key stuck under the old gate recovers on next access".
The step can still fail, and both halves of the loop are proved: a BlobStore
whose insert silently drops a row is refused and stays on the blob, and a
lossy converter is still refused before anything is written.
Renamed the test file, since it now guards the gate rather than describing the
trap it was written to demonstrate.
Task: meshcore-open-lkp
Agent Mail: TealGlen
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#943 says drift "refuses to open a database whose user_version is newer than
the binary's schemaVersion". That is wrong, and the real mechanism points
somewhere else.
drift has no newer-schema refusal. It compares the stored user_version against
schemaVersion for INEQUALITY and routes both directions into onUpgrade
("Schema version upgrades and downgrades will both be run here"). The
pre-#917 build declared schema 1 and no migration getter at all, so it got
drift's default onUpgrade, which throws. That is why the owner's old install
would not start: not a version check, but a missing strategy, so it never
reached the blobs it would have found.
The consequence matters for the decision. This build does NOT throw on a
downgrade: the guard is `from < 2`, so a database from a future schema opens
silently, and drift then stamps user_version back down to 2, erasing the only
record that it has been through a newer build. The next schema bump inherits a
silent failure rather than #943's loud one.
- schema_downgrade_test.dart, six tests: a newer version reports as hadUpgrade
rather than being refused; a bare strategy throws (the #943 failure); this
build opens a schema-3 database without complaint; and it rewrites the
version backwards. Two more pin that an equal version takes neither branch
and that a fresh database still lands on 2, so the downgrade cases are
genuinely testing the downgrade path.
- Corrects the stale claim in offband_database.dart, which still told readers a
downgrade finds its history. The merged plan was corrected in 4b8f5731; this
is the same correction in the code it describes.
Also lands migration_refusal_trap_test.dart for #952, found while gathering
this evidence: four of the owner's store keys hold rows the blob does not, so
the readback gate's equality check refuses them on every attempt, for good.
Five tests, including the clean-key control and a check that no message is
lost. Fix deliberately not included; #952 carries the decision.
The three decisions in #943 are untouched and remain the owner's.
Task: meshcore-open-3e1
Agent Mail: TealGlen
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Epic D of Feature #853. A different defect class from Epic B: these stores
ARE correctly per-radio and bled anyway, through timing.
**D2, #158: the window was an unexpected disconnect.** A manual
`disconnect()` clears `_selfPublicKey`; `_handleDisconnection()` did not. So
after a drop, every per-radio store, and the one `channels_screen` builds
from `selfPublicKeyHex`, still held the previous radio's prefix. Connect a
DIFFERENT radio and any read before its SELF_INFO landed served the old
radio's history. That matches #158 being seen on desktop and not Android:
it is a race, not a logic error.
`_clearStorageScopeKeys()` now runs on an unexpected drop. It clears
`_selfPublicKey` as well as the eleven connector-owned stores and
StorageService, because `channels_screen` does not share those instances and
derives its key from the getter, so clearing only the stores would have left
that path reading stale.
Verified before relying on it: every one of the thirteen per-radio stores
guards on an empty key, in BOTH directions. I checked rather than assumed,
because if any had used a bare prefix instead, clearing the key would have
created a NEW unscoped global key, which is the exact bug class this Feature
exists to remove. The names and deviceId stay, so the UI can still say which
radio it is reconnecting to; only the data-scoping identity goes.
**D1, #190: its named root cause is addressed, proven not argued.** #190
blamed slot-index keying with no PSK binding. Two mechanisms landed since:
PSK-identity keying (#194) and the reuse-clear in `setChannel` (#193). Rather
than assert that, there are tests reproducing #190's own scenario, a slot
reassigned to a different channel:
- keyed by PSK, the newly added channel reads a different key and sees
nothing of the old occupant, which is #190's reproduction
- the old occupant keeps its own history, because this is isolation by
identity rather than deletion
- the index fallback still works and still warns, pinning the remaining
surface
What remains of #190 is the index fallback firing before the channel list has
loaded, which is #333 and a timing problem, so it belongs to this Epic rather
than to #190. Recommendation for the owner: close#190 as superseded by #193,
#194 and this Epic. Not closing it myself; it is his call and D1 was scoped
as a verdict with evidence, not a closure.
Suite 1,425 pass, analyze clean, format clean.
Citadel: meshcore-open-pdx, meshcore-open-cwd
Refs #939, #158, #190, #333, #853
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Real stock 1.18.0 reports its version as "v1.18.0-81dbae5" (semver +
git-describe hash), which classifyVersionString reads as the Offband
dual form. On a stock-shape (82-byte) frame the classifier treated that
as an inverse conflict and failed closed to stockLegacy, so the Device
CLI console (#883/#884) was hidden on every real stock 1.18.0 device.
Found by E2.2 bench verification (#869) against a live device.
An 82-byte device-info frame is definitively stock: the Offband fork
always appends at least one caps byte (>=83 bytes). So a stock-shape
frame is now classified by ver code alone, regardless of the version
string. The fork-shape branch is unchanged - it still requires
dual-version corroboration before emitting fork frames, the direction
where being wrong is dangerous.
Regression test uses the real hardware string v1.18.0-81dbae5. Full
suite green (1394); Gemini 2.5-pro review found no defects.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Epic C of Feature #853, carrying #507. Much smaller than the plan assumed,
because `BlockedView` already existed and was already wired into Settings.
What it did not have was the distinction #507 asked for, or any tests.
**The gap.** With no radio connected the pane rendered "No blocked users",
which is indistinguishable from a connected radio with an empty list. Since
#471 made blocks per radio, that wording invited exactly the wrong
conclusion: that the blocks had been lost. Now a disconnected pane says "No
radio connected" and explains why ("Blocks belong to the radio that holds
them"), because the reason is what stops it reading as data loss.
English-only string additions per the repo convention; `untranslated.json`
tracks the other 17 locales.
**Tests: 5, and the first one has teeth.** I proved it by disabling the new
branch and watching it fail:
Expected: exactly one matching candidate
Actual: Found 0 widgets with text "No radio connected"
Then reverted. A test nobody has watched fail is not evidence, and this
session has already produced three harnesses that reported success while
measuring nothing.
Covered, all through the widget rather than the service:
- no radio says so, and gives the reason
- a connected radio with nothing blocked says THAT
- a block made on radio A is absent on radio B
- switching back restores A's list, because isolation is not deletion
- an unblock survives a switch and back, which is #507's clear-sticks case
and the re-clear that #471 was about
#507 also lists "migration drops the global keys". That already exists:
`BlockService.loadForDevice` calls `_store.dropLegacyGlobal()` on every
device load. Noted rather than re-implemented.
Suite 1,419 pass, analyze clean, format clean.
Still outstanding for this Epic: the T1000e hardware check #507 asks for,
which is an owner step and the only part I cannot do.
Citadel: meshcore-open-uu9, meshcore-open-9ft
Refs #938, #507, #471, #853
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both scoped by the device-key prefix, and the legacy untagged data deleted
per the owner's decision 1 rather than orphaned.
**#854 path history: per radio AND out of SharedPreferences.** Two changes in
one migration, because doing them separately means two migrations over the
same data. The key was prefix + contact, so a route measured by a 1 W
handheld was applied to a repeater and the reverse. It now lives in drift as
prefix + device key + contact.
The move out of prefs is not gold-plating: that one store held 2,092 keys and
86% of a 4.28 MB file, and every prefs write on Windows re-encodes the whole
file (#923). This resolves#924 and makes every unrelated settings write
cheaper in proportion.
**#856 delivery observations: per radio, staying in prefs.** A repeater's
45-second deliveries were training the ACK-timeout model a 1 W handheld then
used. One small key, 3,091 bytes and 14 entries on the owner's install, so
unlike path history there is no cost argument for moving it and a move would
be churn.
**The legacy purge is chunked, and that is measured rather than guessed.**
Each prefs `remove` re-encodes and rewrites the whole file, so I measured the
real sequence against the owner's live prefs before designing it: 2,092 keys,
12.7 s of total work, mean 6.1 ms, but 32 ms for the first removal and under
1 ms by the last, because the file collapses from 4.28 MB to 45 KB as it
goes. I had assumed ~57 s and was wrong; it is tractable.
One unbroken pass would still block for 12.7 s, well past the 5 s watchdog,
which is the exact failure #917 just fixed. So it yields every 20 removals.
It runs after the startup loads so it never delays first paint, and it is
idempotent: no writes once there is nothing left, so running it on every
connect is safe.
Legacy and new cannot be confused, which is what makes the purge safe: the
new scheme writes to drift and the legacy keys are in prefs, so no length or
shape heuristic is needed to tell them apart. That is decision 2 making
decision 1 unambiguous.
**Wiring:** StorageService is shared with PathHistoryService and
TimeoutPredictionService, so the connector cannot own it like the other
per-radio stores. It is injected from main() and fed the device key on
SELF_INFO alongside them. Unset, both surfaces load nothing and never write,
so a read before the handshake cannot serve another radio's data (#754
precedent).
**Tests: Epic A's bleed tests inverted, 11 of them.** Every expectation that
demanded the bleed now demands isolation, and the before/after is the git
history of one file. Plus the purges: prefix-only, idempotent, and the
scoped data surviving the purge of the unscoped.
**Not done, and flagged rather than dropped:** the plan's B2 also said "bound
what persists to what PathHistoryService actually caches". That bound existed
to stop the prefs file growing. In drift a write costs one row and growth
costs nothing per write, so the bound would now delete regenerable route
history for no benefit. Skipping it is a judgement, not an oversight, and it
is called out here and in the PR.
The inventory's known-holes set is empty for the first time: {854, 855, 856}
at Epic B's start, then {854, 856} after B1, now none. An Epic A sanity
assertion that holes must exist failed on that and was corrected, because the
invariant that matters is conditional: if a hole is declared it must name an
issue, and empty satisfies it.
Citadel: meshcore-open-0z3, meshcore-open-z8t
Refs #937, #854, #856, #924, #853
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#855 was filed as "pending messages are stored globally, not per radio".
It is not a scoping bug. `loadPendingMessages`, `savePendingMessages` and
`clearPendingMessages` had no callers anywhere in lib/ or test/, and the key
was absent from the owner's live prefs. The `_pendingMessages` field in
message_retry_service.dart is an unrelated in-memory list that happens to
share the name.
So the surface is DELETED, not scoped: three methods and the key constant.
Scoping it per radio would have been three careful changes and a migration
decision, for an API nothing calls.
This is the finding that justified consolidating #854, #855 and #856 into
one Epic instead of landing three PRs. Nobody reading #855 on its own would
have gone looking for callers, because the issue already told them what the
fix was.
Removed with it:
- the inventory row, replaced by a comment saying the surface is GONE rather
than reclassified, so the history is readable
- A2's round-trip test, whose job was to prove the thing being deleted
actually worked. It served that and then became a test of nothing
- 855 from the known-holes expectation, which now reads {854, 856}; that set
shrinking to empty is what closing Epic B looks like
What still guards it: the inventory's "declared but vanished from source"
check fires if `pending_messages` ever comes back undeclared.
Citadel: meshcore-open-li1
Refs #937, #855, #853
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Feature #853's acceptance bar is that every store has "a verified scoping
verdict and no hole remains". Until now, verifying that meant a human
re-reading an audit in an issue body, which is how three separate bleeds
(#754, #854, #856) survived earlier rounds of bleed fixes.
`scoping_inventory.dart` is that audit in code: every persistence surface
with perRadio / globalOnPurpose / knownHole, a mandatory reason, and the file
that owns it. A reason is required because an unexplained verdict is how a
settled surface gets re-opened as a hole later.
The test makes it a gate. It scans lib/ for the shape every key in this repo
uses, a `const String` with a lower snake_case value, and fails when one is
neither a declared surface nor an explicitly justified non-key. It also
checks the reverse: a surface declared here but gone from the source. That
one will fire when Epic B deletes `pending_messages`, which is intended.
**It caught four things on its first two runs**, which is the argument for
building it before Epic B rather than after:
- `channel_cyr2lat_` and `contact_cyr2lat_` were never audited. #853 listed
channel_settings_store as one surface; it owns two keys.
- `los_error_elevation_unavailable`, a third error sentinel I had not found
by hand.
- `__msgrows_migrated`, declared by me and then reported missing, because my
pattern required a letter first and internal markers start with
underscores. The gate caught my own declaration bug.
Scope of the scan, stated so nobody mistakes it for total: it would miss a
key built inline with no constant, and a key that is not snake_case. Scanning
every string literal in lib/ was tried and rejected, because it fires on
asset paths and log tags, and a check that cries wolf gets worked around.
That is the #927 lesson, which is also why there are three tests that prove
this gate can fail rather than assuming it.
Citadel: meshcore-open-z36
Refs #936, #853
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
On connect, the block reconcile unioned the client's block list with the
radio's and pushed the difference, so a fresh/erased radio (empty list) had
every client block offloaded onto it and came up pre-blocked (#471, owner
verdict ERROR). Owner decision: Option A. A radio that reports an EMPTY block
list is treated as blank and is NOT auto-offloaded; the client stays the source
of truth and blocks reach a radio only by explicit user action. A radio that
already holds blocks still unions normally. Client-side only, no wire change.
Added a setOffbandCapsForTest seam so the reconcile is testable.
Part of radio-profile isolation Feature #853. The per-radio Blocked view +
provenance (#507) is the follow-on.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Discovered contacts (nodes heard but not added) were stored under one global
'discovered_contacts' blob shared by every radio, so one radio's discoveries
showed up while another was connected (#754, Feature #853). Scope the store by
the connected device-key prefix like contact_store and the others; the connect
block sets the key and the existing startup load clears + reloads the list for
that radio. An unset key loads nothing and never writes, so it can't read or
clobber the legacy shared blob. No migration: the legacy blob is orphaned and
each radio's list self-heals as it re-hears adverts.
Part of radio-profile isolation Feature #853.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1. existingFiles() walked hardcoded names with per-name exists(), so the periodic flush could rotate mid-walk and return a name it had just confirmed and rotation then renamed. Handing the share sheet a path that no longer resolves fails the WHOLE export, which is the exact failure the rotated-log fix existed to prevent. Now one directory listing, filtered and ordered by name. flushAndGetAllFiles() degrades to the active file if listing throws, because the logger cannot report its own listing failure anywhere, and one log beats none. shareLogs() also re-checks existence at the last moment, since a trace can be pruned between collection and share.
2. forceDartIsolateBlock used DateTime.now() for its deadline. The wall clock can move backwards, which would push the deadline away and leave the app frozen past the window or indefinitely, needing a force-kill on a test build. Now a Stopwatch, which is monotonic, matching what the Kotlin block already did with nanoTime.
analyze clean, 1307 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The existing trigger hangs the Android main thread, so its trace is a Java stack. That proved the capture works and that our Kotlin frames resolve via the R8 mapping, but it cannot say whether our own DART frames survive into a trace, which is the question #906 was filed to answer.
This one blocks the Dart UI isolate the way the real ANRs do rather than with an artificial spin: it repeatedly JSON-encodes a large nested structure shaped like the stores #334 is about. The Play clusters show the real ANRs inside the JSON stringifier and under allocation pressure, so a trace from this block should match their shape, and whether any of our Dart frames appear above those is then readable.
It spins deliberately. Any await in the loop would yield to the event loop, the isolate would keep answering input, and no ANR would fire. If the process survives the full duration it logs why, so the tester is not left guessing.
Gating is unchanged and still doubled: the bool.fromEnvironment constant keeps both rows out of any shipped build, and the owner-only Experimental section hides them again in a test build. Both rows are relabelled native side and Dart side so they are distinguishable.
Tests: both triggers asserted to be immediate no-ops off Android, which matters because a missing platform guard would hang a desktop build instead of failing visibly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1. The export shipped only the active log, and FileLogWriter._rotated was private, so offband.1.log and .2.log were unreachable by anything. An exit-record pointer line written at startup rotates out after a couple of MiB of ordinary logging, handing over a trace file with its metadata gone. FileLogWriter now exposes the files that exist, keeping the rotation naming where it belongs, and the mobile export attaches all of them.
2. dispose() called ioExecutor.shutdownNow() while a read could be holding a MethodChannel.Result. Interrupting it left the Result neither succeeded nor errored, so the Dart future never completed. Now shutdown(), with a disposed flag and a RejectedExecutionException path that always replies.
3. The trace read was skipped for REASON_CRASH_NATIVE on the assumption that its stream is always a protobuf. AOSP says a trace from an earlier ANR stays attached even when the process later died for another reason, so gating on the reason loses real text traces. The read is now attempted on every record and a printable-bytes check decides whether the content is usable.
4. The test-only main-thread block used a single Thread.sleep, which an interrupt could cut short: no ANR, no record, and a wasted trip to the hardware. It now sleeps to a deadline and ignores interrupts, which guarantees the duration without a busy-wait's CPU burn.
Review also flagged that FileLogWriter can exceed maxBytes for a single oversized entry. That is pre-existing writer semantics, already pinned by a test in this branch, and deliberately not changed here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It said hanging the UI makes Android raise an ANR. That is not true on its own. Android's documented trigger is 'has not responded to an input event within 5 seconds', and it documents no time-based trigger for a blocked main thread, so a freeze with nothing pending just recovers: no ANR, no process death, and no exit record to read back.
The subtitle now says to keep tapping during the freeze, which is what actually queues the unanswered input the dispatcher measures. Without that the tester taps once, waits, sees nothing happen and reasonably concludes the feature is broken.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Exit records are Android-only, so on a desktop build the dart-define alone would leave a button that does nothing. The row now needs the flag and a supported platform.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Hanging the main thread is the only way to produce a real REASON_ANR record to read back. No adb command induces an ANR: am crash produces a crash, and kill -3 dumps traces without a REASON_ANR death. So the test needs a trigger inside the app.
Gated twice. The Experimental section ships in release behind a 7-tap unlock, which is not enough for a control whose job is to hang the app, so the row also sits behind a bool.fromEnvironment constant and is compiled out entirely unless a build passes OFFBAND_ANR_TEST. That flag is passed on the command line, never added to dart_defines.json, which feeds every build including Play.
It is an action rather than a persisted toggle, so there is nothing for disableExperimental() to clear and app_settings.dart is untouched. The label is deliberately not localized, since it never reaches a user; the Observer rows in the same file set that precedent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
shareLogs() exported exactly one file, the active offband.log, and the rotated .1.log and .2.log were never exported at all. So a trace written anywhere other than inline would never have reached anyone, and a trace written inline would later rotate out of the only file that ships.
SharePlus already takes a list of files, so this attaches the trace files beside the log with no new UI and no new localized strings. Mobile only: the records are Android-only and the desktop path saves a single chosen file.
Without this the Epic would ship and deliver nothing, which is why it is part of the build rather than a follow-up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reads the records over the channel, logs one line per record whatever its reason (the owner's scope ruling on #908), and writes any ANR trace to a file beside offband.log.
Dedup is a single persisted integer, the highest exit timestamp already logged. Records come back newest first, so a pass takes everything strictly newer than the watermark and then advances it to the highest seen; two records sharing a millisecond are both taken in that same pass, so neither is lost. A missing watermark is treated as a capped backfill and labelled as such, so a months-old ANR is not misread as fresh.
Scheduled in a post-frame callback and deliberately not awaited, so startup time is unchanged. Every failure path writes a line and none is silent; a failed query leaves the watermark untouched so the next launch retries. An explicit flush follows the writes because FileLogService batches on a 2 second timer, and without it a startup line can be lost, leaving a trace file with nothing pointing at it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A save is now one upsert per message in the list being saved, and costs the
same with 50,000 messages of history as with 50. That is the Feature's
"done when", and it is the part #919 could not deliver.
Also gone: a delete is a single-row DELETE instead of rewriting the whole
history, and a load reads only its own key's rows, ordered and pageable in
SQLite.
Load order matters and is deliberate. Rows are consulted first, but ONLY
when rows actually exist. A key marked migrated with no rows still falls
through to the legacy slot-index and unscoped-key adoption (#194, #306),
because those run on blobs; short-circuiting on the marker alone would strand
history stored under an older key. Adoption then hands its result to the
migration, so the key ends up on rows either way.
The blob path stays as the fallback for a key whose migration was refused.
Refused keys keep working exactly as they did after #919: off the UI isolate,
merge-not-overwrite, failing closed on an undecodable blob. Tested, including
that a refused key does not accumulate half a history in rows.
One unreadable row now costs one message. On the blob a single bad byte took
the whole conversation with it.
Clear paths had to learn about rows: a cleared channel or conversation that
only dropped its blob would have stayed fully readable. clearAll also sweeps
row-only keys, which no longer appear in a blob key listing.
`encodeHistoryForTest` is new on both stores. The #919 tests built their
blob fixtures by saving through the store and reading the blob back, which
cannot work once saves write rows, and hand-written fixtures are what rotted
the first time. They now call the stores' own encoder directly.
Tests: 24 added, through the public store API only, because that is what an
existing install hits on its first launch after this ships. Covered: blob
adoption on first load and on first save, the windowed-save truncation case
(#343), per-radio and per-contact scoping (#472), delete, clear, clearAll on
row-only keys, a refused migration still serving reads and writes, and the
payload being the stores' own encoding.
Suite: 1,344 pass, analyze clean, format clean.
Citadel: meshcore-open-tfq
Refs #920, #917, #334
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Per key, once: count the blob's records, convert it to rows off the UI
isolate, insert them in one transaction, read the stored count back, and
only then mark the key as reading from rows.
**The blob is never deleted**, not even on a verified match. Rows become the
read and write path; the blob is left untouched. Reclaiming that space is a
separate owner-gated step, because the moment the delete runs is the moment
a mistake stops being recoverable, and a downgrade to an older build still
finds its history where it looks for it.
Every failure leaves the blob authoritative and says so loudly: an
undecodable blob, a converter that throws, a count mismatch, a failed
insert. None of them retries, because a count that did not match once will
not match on a loop.
The migrated set is its own marker key, not "this key has rows". Those are
different statements: a key can have rows and an incomplete migration at the
same time, and treating rows as proof of completeness is how half a history
would read as a whole one. An unreadable marker treats every key as
unmigrated, which is the safe direction because the blobs are all still
there.
The first version of the readback gate was a check that could not fail. It
compared the stored row count against the number of rows the converter
produced, so a converter that silently dropped records matched itself and
passed. Caught by a test built to drop a record. The blob's element count is
now measured by a separate decode that does nothing but count, so the
converter cannot report its own homework: `rows.length` must equal that
count, and the stored count must equal the distinct identities. Both can
fail independently, and together they close the loop from blob element to
stored row.
Duplicate identities in a source blob are deduped rather than refused. The
old merge should not have produced any, but the legacy slot-index adoption
path (#194) could have, and that must not read as data loss.
Tests: 15, nine of them failure paths. The lossy-converter case is the one
that caught the broken gate.
Citadel: meshcore-open-x9q
Refs #920, #917, #334
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Schema 2 adds one row per message beside StoredBlobs, which stays exactly as
it is. A save becomes an upsert of one row, so its cost stops depending on
how much history is already stored. That is what #919 could not do: it moved
the whole-history re-encode to another isolate, it did not remove it.
Shape, and why it is this shape:
store_key the EXISTING blob key, byte-identical
merge_key the stores' existing _mergeKey identity
timestamp_ms INTEGER, so ORDER BY and LIMIT run in SQLite
payload the per-message JSON the stores already emit
PK (store_key, merge_key), index (store_key, timestamp_ms)
Reusing the blob key means every per-radio and per-channel scoping rule
(#194, #472) carries over untouched, and the migration becomes a per-key
operation with an exact record count to verify against.
The payload is JSON rather than a column per field. A Message has 25 fields
including three maps, two byte arrays and the translation set; hand-mapping
them would mean trusting a new field-by-field converter against 41 MB of
real history on its first run, and #355 cost 574 messages the last time a
storage migration was hurried. With a payload column the migration is a pure
re-key, the stores' existing encoders are reused verbatim, and a new message
field needs no second migration. The trade is that message fields are not
SQL-queryable; nothing asks for that today, and columns can be added beside
the payload later without another data migration. Flagged for the owner on
the issue before building it.
The row API lives on BlobStore rather than in a second class because both
APIs must share one OffbandDatabase: opening it twice is an error on the web
backends, and a second singleton would be one mistake away from doing that.
`limit` with no `offset` returns the NEWEST n, still oldest-first, because
that is what a chat screen opens with.
Tests: 18. The row set covers store-key scoping (a leak there would be a
history bleed between radios), merge-key upsert, paging, per-row delete, and
rows coexisting with a retained blob. The schema set builds a database that
looks like one an older build left behind, v1 table with real data and
user_version 1, and proves the upgrade runs, creates a usable table, leaves
the blobs byte-identical and lands the index on both an upgraded and a fresh
database.
Citadel: meshcore-open-g6j
Refs #920, #917, #334
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Deleting one message rewrites the entire blob, so on the owner's largest
channel a delete blocked the UI for as long as a save did, right after the
user tapped something. Both delete paths now go through
`mergeOffThread`.
Two call shapes, one helper. `mergeOffThread`'s second argument is renamed
`incomingJson` to `payload`, because the callback decides what that string
means: the incoming batch when saving, the target merge key when deleting.
The method only carries it across the boundary.
`dropFrom{Channel,Dm}HistoryJson` return null when the target is not in the
history, so nothing is written at all. Re-encoding 5 MB to change nothing is
exactly the cost this Epic exists to stop paying, and the old code did it on
every miss.
`MessageStore.removeMessage` also gains per-key serialisation it never had:
it was a read-modify-write outside `BlobStore.synchronized`, so a save
landing after a delete could resurrect the deleted message. There is a test
for that race.
Tests: 8 added in their own file, including both stores' public delete API,
an absent target, an absent blob and a corrupt blob left byte-identical.
Citadel: meshcore-open-z7c
Refs #919, #917, #334
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`MessageStore.saveMessages` had the same read-modify-write as the channel
store: decode the whole DM history, rebuild every record, re-encode it, all
on the UI isolate, for every single message that arrives.
Applies A1's helper. Only the incoming batch is encoded on the UI isolate;
the decode, merge and re-encode go to another isolate via
`mergeDmHistoryJson`, a top-level function so `compute` can send it. The
five JSON helpers it needs are now static for the same reason.
`mergeDmHistoryJson` throws on undecodable stored JSON rather than
swallowing it, which `mergeOffThread` turns into a refused write. That is
the same refusal the inline code made, and it is what stops a 5 MB history
being replaced by the one message that happened to arrive.
Tests: 6 added. Fixtures encode through the store's own encoder, never a
hand-written map, because a hand-rolled fixture rots the moment a non-null
field joins the schema. Includes a save round trip proving the second save
merges rather than overwrites (#343).
Citadel: meshcore-open-8sg
Refs #919, #917, #334
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
BlobStore.mergeOffThread runs the whole read-modify-write through compute(): stored JSON and incoming JSON in, merged JSON out, so only strings cross the isolate boundary and the object graph never does. Keeps the per-key serialisation that stops concurrent saves clobbering each other.
channel_message_store.saveChannelMessages now encodes only the INCOMING messages on the UI isolate (normally one message, measured 0.002 ms) and hands the 5.38 MB decode-merge-encode to another isolate, where it measured 134 ms per save and crossed Android's 5s input-dispatch watchdog in a burst (#334).
Fails closed: a stored blob that will not decode throws out of the merge, is logged, and the key is NOT written, which is the same refusal the inline code made. compute runs inline on web, which has no isolates; that is documented behaviour, not a silent fallback, and #920 removes the cost for every platform.
_messageToJson, _messageFromJson, _mergeKey, _repeatToJson and _repeatFromJson became static so the top-level merge can call them; none touched instance state.
9 new tests cover merge equivalence, timestamp ordering, empty/absent blobs, the throw on corrupt JSON, fail-closed leaving the blob byte-identical, and three concurrent saves all surviving. Fixtures go through the store's own encoder rather than hand-written maps, after a hand-rolled fixture rotted on a non-null field.
Refs #334, #917
Agent: TealGlen (session c61ce804)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(#881): frame 66/29 CLI codec with cc| correlation
Pure codec per the 1.18 contract: ASCII fail-closed builds, lenient reply decode, truncation bound judged on the raw payload. Gemini findings 1+2 fixed, 4 documented.
Agent: GentleGlacier (session f0c92a36)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(#882): local CLI service + shared cc| correlation helpers
PrefixAllocator and splitCorrelated extracted; the repeater service delegates unchanged in behavior. LocalCliService runs correlated frame-66 round trips; the connector routes RESP 29 to listeners.
Agent: GentleGlacier (session f0c92a36)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(#883): device CLI console screen
The local-device counterpart of the repeater CLI screen (interop plan
F2/E2.1): type a command, it goes to the connected radio over frame
66/29 via LocalCliService, and the correlated reply renders with a
per-reply latency chip. No path/flood selection and no retries - one
send, one frame-level timeout window surfaced as its own row.
Built to the owner-approved behaviors: masked secret replies
(long-press to copy, cleartext never in the widget tree), the 165-byte
truncation badge, string-matched error styling, and command history.
Entry point and population gating are #884, deliberately not here.
Security fix surfaced by the #883 Gemini review: LocalCliService logged
the cleartext of an unmatched (late) reply, so a secret reply arriving
after its command timed out could leak to the app log despite the
screen's masking. The log line now withholds the reply text (token +
byte length only); a test pins it.
Also replaces em-dash characters in the sibling E2.1 files flagged by
the #464 CI guard.
Tests: 12 widget tests for the screen and a service test for the log
redaction. Full suite green (1207 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor(#902): declutter Settings - user-ops to Actions, reorder Debug
Owner-directed Settings hygiene, split from the #884 entry point so the
Device CLI row lands in a clean Debug pane.
Moved to the Actions pane (they are user operations, not diagnostics):
config Export/Import and the three GPX exports (the whole export card),
plus "Sync queued messages now".
Debug pane reordered to App Debug Logging, Companion Debug, App Debug,
Serial Capture, Mesh Topology (which stays - useful diagnosing with a
user). The Device CLI row at the top is #884, not here.
Behavior preserved: every row keeps its handler, icon and copy; only
their pane and order changed.
Tests: a new widget test pins the spec - the moved rows present under
Actions and absent from Debug, and the full Debug order. A Gemini review
finding of a Debug-ordering defect was refuted (the order matches the
spec; the review misread the toggle for the viewer); its secondary point
was taken and the test now pins the full order. Full suite green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(#884): Device CLI entry point + capability gate
Adds the entry point for the console (screen landed in #883) to
Settings > Debug. A "Device CLI" row appears at the TOP of the Debug
pane only when the classifier places the connected radio at stock v14+
(classification.allowsCliFrame): stock 1.18+ today, Offband at the
upstream-1.18 absorb via the D1 seam. The row is absent on stock-legacy,
Offband, and when disconnected (classification null). Tapping it opens
DeviceCliScreen.
The gate reads the capability through context.select, so only that
boolean flipping rebuilds the pane, not every connector update (#81).
Depends on the classifier from dev (merged into this branch).
Tests: positive (stock 1.18 shows at top, opens the console) and three
negatives (disconnected, stock-legacy ver 13, Offband all hidden). Per
the Gemini review the test's frame helper now asserts the version fits.
Full suite green (1244 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(#881): dart format the codec test (CI format gate)
test/cli_command_codec_test.dart (landed in #881) was not formatted, so
the branch's first run of the "analyze" job (flutter analyze
--fatal-infos + dart format --set-exit-if-changed .) failed on it.
No logic change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(#885): mirror set-name character rejection + CLI l10n
The firmware's `set name` rejects the characters [ ] \ : , ? * and
bounces a bad name as a cryptic ERROR. Mirror that client-side so the
name is caught before it is sent, in both places a name is set:
- Settings node-name dialog: live inline error, Save blocked on an
invalid name (nothing sent).
- Device CLI console: a `set name <value>` with a forbidden char is
echoed with a rejection row and never sent; the device stays the
authority for everything else.
Shared validator in lib/helpers/device_name_validator.dart
(forbiddenCharsIn is distinct + canonical-order). Also moves the
"Sync queued messages now" row strings and the new validation messages
onto app l10n (EN copy unchanged).
Tests: validator unit (the exact seven chars, order-stable, deduped),
console pos/neg (+ rejected command still recallable from history, per
the Gemini review), and the Settings dialog pos/neg. Gemini review
otherwise clean; its one finding (rejected command not in history) was
refuted - history derives from the transcript, which the rejection path
appends to - and pinned with a test. Full suite green (1255 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: diffwireauto <330573775+diffwireauto@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Channel notify modes lived in one global AppSettings map keyed by PSK or a
bare slot index, so one radio's settings bled into another and set-and-back
could not clear a stale alias (#839). Scope them by the connected device-key
prefix like every other per-radio store (prefix -> pskHex -> mode); resolve
All when the radio or PSK is unknown instead of falling back to the slot
index; store a single entry per channel so All truly resets. One-time
migration moves the legacy flat map and name-keyed mute set into the first
radio synced after upgrade; later radios start clean. Removed the now-unused
legacy muteChannel/unmuteChannel/isChannelMuted methods so mutedChannels is no
longer a second, cross-radio source of truth (pre-PR Gemini review finding).
Buzzer Inherit (#544/#545) reads the same path and inherits the fix.
Epic #844. Build #848.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment-only; the analyze job's em-dash guard scans source files. Part of epic 866 (container not named in the scope per the push gate).
Agent: GentleGlacier (session f0c92a36)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Seven comparison sites dispositioned: pre-split floors (>=8, >=13) stay bare by design; fork floors are caps-anchored and now covered by a stock-ver-22 regression test. Channel-data bound is vacuous: the client never sends CMD_SEND_CHANNEL_DATA.
Agent: GentleGlacier (session f0c92a36)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Device-info handling classifies the radio before reading any Offband byte; a conflict nulls caps/caps2/FEM and logs why. Capability bytes now also die on disconnect, like the ver code.
Agent: GentleGlacier (session f0c92a36)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Classifies the connected radio (stockLegacy / stockModern / offband) from device-info frame shape + ver code + version string; fails closed on conflicts and malformed input per the plan's can't-tell table. Gemini findings 1A/2 fixed (uncorroborated fork shape and inverse conflict both fail closed); finding 3 declined with justification on the issue.
Agent: GentleGlacier (session f0c92a36)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
D1: first Discover attempt sometimes drops (normal LoRa packet loss); the retry gets through. Per owner (Option A), discoverRegions now makes up to 2 attempts (_discoverRegionsOnce), each using the firmware est_timeout; an unreachable repeater still ends in a timeout after both. Test: first attempt times out, second succeeds, two sends.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
D1: the picker excluded direct repeaters. hasPath required a non-empty path, but a direct 0-hop repeater has pathLength 0 and an empty path, so the owner's own adjacent repeater was greyed as 'No stored path'. Direct is a reachable (best-case) route; hasPath is now pathLength >= 0, excluding only pathLength -1 (flood/unknown). The reply-path already sends zero-hop for direct, which is correct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The picker lists only repeaters with a stored path (the only ones that can answer), with a header note explaining that. Search surfaces pathless matches too, but greyed out and non-selectable with a 'No stored path' label, so the user sees the repeater exists and why it is blocked. Default selection skips pathless repeaters; Discover is disabled for them. Widget test covers hidden-by-default + greyed-on-search.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Discovery no longer caps the reply to zero-hop. discoverRegions now sends the repeater contact's stored path as the reply path (the firmware routes the reply back along it as-is, Mesh.cpp:176-177), so a multi-hop repeater can answer. The discovery timeout re-arms to the firmware's own est_timeout from RESP_CODE_SENT (scales with hops) instead of a fixed 10s. The repeater list is never filtered on path/hops, since stored paths go stale as the companion moves (owner guidance). An unknown/stale path still surfaces as a plain timeout, not a silent drop. Test covers the path being used as the reply path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
D1 found Discover Regions got no reply from the owner's repeater. Root cause (firmware BaseChatMesh.sendAnonReq): the firmware prepends its own 4-byte tag, but buildAnonRegionsRequestFrame also prepended a timestamp, shifting the request-type byte so the repeater dispatched it as a login, never regions. And discoverRegions correlated on a client-invented tag, not the firmware tag the 0x8C echoes. Fix: request payload is [0x01][reply_path_len] (no timestamp); discoverRegions learns the tag from the RESP_CODE_SENT ack via the frame stream and matches the 0x8C by it. Removed the dead tag map + 0x8C switch case. Tests corrected to the real contract (they had encoded the wrong one).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
D1 hardware finding: the repeater picker was an unsorted, unsearchable dropdown, unusable with many repeater contacts. Replaced with a case-insensitive alpha-sorted, searchable list; discovered regions are alpha-sorted too. Widget test asserts sort order and search filtering.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pre-PR Gemini review (Epic #815) flagged a scope leak: setChannelSendScope was outside the try, so a lost ack after the firmware applied the scope skipped the clear, and the next unscoped channel send would ride the stale scope. Moved the set inside the try so the finally always clears. Regression test asserts a clear is attempted when the set throws.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A firmware-gated channel overflow action opens a picker that discovers regions from a chosen repeater (reusing RegionDiscoveryService), lets the user select a region or None, and persists it via connector.setChannelRegionScope (C1 store). The dialog is decoupled (data + callbacks + injected service) for widget testing. l10n added.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sendChannelMessage routes both the text and reaction sends through _sendChannelFrameScoped: when the channel has a region scope (C1 store), it sets the flood-scope key (CMD 54 sub-0, SHA256 of the region) before the send and clears to node default after. Fail-closed: a scoped channel on firmware without region support is blocked, never sent unscoped. Tested via a seam: unscoped passthrough, scoped set/send/clear with the right key, and the blocked case.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ChannelRegionScopeStore persists a channel-index -> region-name map per device (first 10 hex of the device key, matching the other channel stores). set/clear/scopeFor, null = unscoped. Tests cover round-trip, per-device and per-channel isolation, persistence, and no-device-key guard.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pre-PR Gemini review (Epic #814) flagged a use-after-dispose: if the screen pops while a discovery is in flight, the completing future would call notifyListeners on the disposed service and throw. Added a _disposed flag + _safeNotify guard + dispose override, with a regression test that disposes mid-discovery.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A firmware-gated repeater-hub tile opens a Discover Regions screen that runs RegionDiscoveryService against the selected repeater and renders each state distinctly (loading / region list / no-regions / timeout / error, with retry). Shows region name + source repeater; the wire reply has no per-region SNR, so none is shown (the B2 spec's SNR is not available). Widget tests cover all four states via an injected fake discoverer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RegionDiscoveryService (ChangeNotifier) drives one discovery run via an injected discoverer function and exposes a distinct state for each outcome: idle/loading/success/empty/timeout/error. Regions deduped by Region value-equality. Concurrent starts ignored. Injected seam keeps it unit-testable without a live connector.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pre-PR Gemini review (Epic #813 T4) flagged that RESP 28 carries no correlation tag, so two concurrent getDefaultFloodScope calls could orphan the first completer. Coalesce the second caller onto the in-flight request, matching the single-in-flight guard on exportPrivateKey. (Gemini's second finding, reserveTransportCode(0xFFFF), was rejected: firmware does code-- to 0xFFFE, which the code already matches.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
discoverRegions() correlates the 0x8C reply by the echoed tag (mirrors the 0xC6 pkt-hash pattern); setChannelSendScope/clearSendScope/setSendUnscoped and setDefaultFloodScope/clearDefaultFloodScope/getDefaultFloodScope wrap CMD 54/63/64 with RESP 28 handling. All gated fail-closed on supportsRegionScope. Two new dispatch cases (0x8C, RESP 28). Tested via the connector test seams.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the stock flood-scope companion frames (CMD 54/63/64, RESP 28), the regions-discovery anon request (CMD 57 type 0x01) and its 0x8C CSV reply parser, and firmwareSupportsRegionScope (FIRMWARE_VER_CODE >= 13). Wire layouts verified in firmware source; frame build/parse round-trip tested incl. empty and non-matching frames.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Client-side region transport-key derivation matching firmware RegionMap/TransportKeyStore: key = first 16 bytes of SHA256("#"+name), a Region value model, and the HMAC on-air code. Known-answer tests use independent vectors. No firmware change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>