Wires the migration into startup and points the four bulk stores at drift.
SharedPreferences keeps the settings, which is what it is for.
Startup runs the migration after prefs are up and BEFORE any store reads, and
awaits it. Letting stores race a half-finished migration is precisely the
shape of #333, where a storage path silently chose the wrong key and 566 real
messages read as empty.
Converted: message_store, channel_message_store, contact_store,
contact_discovery_store. All four now have ZERO prefs get/set for bulk data.
Three safety properties, deliberately built in:
1. readWithPrefsFallback - if a key is somehow not in drift, the prefs copy is
still served rather than reading as empty. It logs a WARNING when it fires,
because a fallback during normal operation means the migration is
incomplete and someone needs to know. Silence here is what made #333 look
like data loss.
2. deleteEverywhere / keysWithPrefix span BOTH backends. A clear that only
removed the drift row would leave a pre-migration prefs copy to reappear
through the fallback, resurrecting deleted history.
3. The legacy-key migrations inside the stores now check both backends and
only touch prefs when a legacy key actually exists. This also carries the
#306 fix into this branch: the unconditional prefs.remove was still present
here, since this branched from dev rather than from the #306 work.
Verified: analyze clean, format clean, 504 tests pass, Windows and web both
build. Migration rehearsed earlier against a copy of a real 7 MB store: 69
keys, 5.21 MB, zero failures.
NOT yet run against live data - that happens on first launch of this build,
and the owner should have a prefs backup before that.
Moves message history, contacts and discovered contacts out of the settings
store. Settings stay in SharedPreferences, which is what it is for.
Ordering is the whole safety argument: WRITE, VERIFY BY READING BACK, and only
then remove the source. #333 was a storage path that chose a key silently and
made 566 real messages read as empty; deleting before verifying would make
that class of mistake permanent instead of cosmetic. On any failure the source
is left intact and the error is logged - never a silent drop (SAFELANE 6).
Idempotent by construction: a key already present in drift is not overwritten,
so re-running is a no-op. If an older build re-writes a migrated key into
prefs, the migrated copy wins and the stale prefs copy is discarded rather
than promoted.
Rehearsed against a COPY of a real 7 MB store, as the plan required before
touching live data:
REHEARSAL: 69 migrated, 0 failed, 5.21 MB, 69 bulk keys expected
Every key checked for exact length, confirmed removed from prefs, and every
settings key confirmed untouched. The live store was never opened.
Two things the tests caught that review would not have:
1. getString THROWS on a non-string value rather than returning null, so a
non-string under a bulk prefix was counted as a migration FAILURE. It now
type-checks with prefs.get() and skips. Alarming falsely is its own bug.
2. The `contacts` prefix was checked against the real store rather than
assumed: it matches only the 6 bulk contact blobs, and correctly does NOT
match contact_unread_count*.
Not yet wired into app startup - that is the switchover, and it is deliberately
a separate commit so this can be reviewed on its own.
flutter analyze clean, dart format clean, 504 tests pass.
Step 1 gate, web half. Building is not evidence, so this was run in a browser.
Assets: sqlite3.wasm (748 KB) and drift_worker.js (355 KB), both taken from
the SAME drift-2.34.2 release so they are built against each other. Validated
before use - the wasm has correct WebAssembly magic (0061 736d) and carries
SQLite 3.53.3; the worker is genuine compiled Dart JS.
Note the filename: the release ships `drift_worker.js`, not the
`drift_worker.dart.js` the docs name. The database URI was corrected to match,
which would otherwise have been a silent 404 at runtime.
Browser result, served over HTTP with COOP/COEP and Content-Type:
application/wasm:
Using WasmStorageImplementation.opfsLocks
PROBE-OK open+write+read
PROBE-OK 6MiB round-trip (localStorage cap is 5MiB)
drift selected OPFS, the top storage tier, and round-tripped 6 MiB - data the
current localStorage-backed web build physically cannot hold. No exceptions.
Adds web/_headers for Cloudflare Pages: forces application/wasm on the wasm
asset and sets COOP/COEP for cross-origin isolation. Verified to ship into
build/web. Without COOP/COEP drift still works, falling back to IndexedDB,
which is slower but still unbounded next to localStorage - a performance tier,
not a correctness requirement.
web_probe/ is a standalone entrypoint that exercises ONLY the drift stack, so
a pass or fail is unambiguous without BLE and the rest of the app in the way.
It is not part of the app build.
Still no user data touched and no store switched over. The migration is the
next step.
flutter analyze clean, dart format clean, 497 tests pass.
Step 1 of #335, deliberately before any data migration: prove the store works
before trusting it with message history.
Adds drift + drift_flutter, a key/value StoredBlobs table mapping 1:1 onto the
existing SharedPreferences keys, and verifyReadWrite() as a runtime probe.
Key/value rather than relational tables on purpose: it maps directly onto the
current store interfaces, so callers do not change and the migration can be
verified row-for-row against the old keys. Relational schemas come later, once
this is proven and querying is actually wanted.
Verified here:
- 3 tests pass, including a 6 MiB round-trip - larger than the 5 MiB
localStorage cap that makes the web build impossible today
- flutter analyze clean
- Windows release build succeeds
- Web release build compiles
Two findings from the gate that a plan-on-paper would have missed:
1. sqlite3_flutter_libs 0.6.0+eol is a DEPRECATED no-op. From sqlite3 v3.x it
is unnecessary and drift_flutter already covers it, so it is not a direct
dependency here.
2. The web build COMPILES BUT WOULD FAIL AT RUNTIME: drift needs sqlite3.wasm
and drift_worker.dart.js shipped in web/, and neither is present. Per drift
docs they are downloaded from GitHub releases, version-matched to
pubspec.lock (drift 2.34.2, sqlite3 3.5.0), and the server must serve .wasm
as Content-Type: application/wasm. Not done here - downloading files needs
explicit human approval.
Also noted: the drift_dev CLI does not compile at these versions
(allSchemaEntities missing from the drift3_preview GeneratedDatabase), so its
asset tooling is unavailable. build_runner code generation is unaffected.
No user data is touched. No store is switched over. The migration is the next
step and remains gated on the web assets question.
The stores violated the no-silent-failures rule, and that is why tonight's
diagnosis took as long as it did. Two classes:
1. Ten catch blocks swallowed decode errors and returned an empty collection.
`catch (_) { return []; }` means a corrupt or unreadable store presents to
the caller as "no data", so the user sees no contacts, no channels or no
history with nothing anywhere explaining why. Indistinguishable from data
loss. Every one now logs what failed, with the store named, before
returning.
2. ChannelMessageStore._storageKey fell back from the PSK-identity key to the
legacy slot-index key without a word. That fallback is correct before a
channel list exists, but once history has been migrated to the PSK key it
reads a DIFFERENT key and returns empty. That is exactly #333: a user's 566
Public messages were intact on disk while the app showed nothing, with no
error, no parse failure and no log line to follow.
It now warns when it falls back while a resolver is installed, since that
combination specifically means the channel list was not loaded first.
Had either of these been loud, #333 would have been a one-line log read
instead of a multi-hour investigation that looked like data loss.
flutter analyze clean, dart format clean, 494 tests pass.
The previous commit fixed loadSmazEnabled and killed the post-connect stall
(startup-load channelSettings: 37792ms -> 0ms, measured twice). The contact
pull still choked, in multi-second bursts, each landing immediately after an
"Added new contact" line.
No frame handler exceeded the 100ms threshold, because the work is not in a
handler: the contact handler fires _loadMessagesForContact unawaited, so it
runs after the handler returns and escapes that timing.
message_store.loadMessages had the same unconditional prefs.remove(oldKey),
and it runs ONCE PER CONTACT during a pull. On Windows every prefs mutation
rewrites the entire file, so a 234-contact pull meant 234 full multi-MB writes
for legacy keys that do not exist.
A sweep found the identical pattern in six more stores: contact_store,
unread_store, channel_store, community_store, contact_group_store and
channel_order_store. Those run once per load rather than per item, so they are
far cheaper, but it is the same bug and they are fixed the same way.
In every case the removal now happens only when a legacy key actually exists,
paired with the write that migrates it, so the cleanup still occurs on a real
migration.
flutter analyze clean, dart format clean, 494 tests pass.
The connect freeze is a legacy-key migration that writes even when there is
nothing to migrate.
loadSmazEnabled called prefs.remove(oldKey) unconditionally whenever the
scoped key was absent, which is the normal case for a channel that has never
had the setting changed. On Windows shared_preferences re-serialises and
rewrites the ENTIRE prefs file on any mutation, so each of those no-op removes
cost a full multi-MB write. Repeated across every channel slot on every
connect, that is tens of full-file rewrites back to back.
Measured with the instrumentation from the previous commit, on the reporting
device (6.9 MB prefs, ~42 channel slots):
startup-load channelOrder 0ms
startup-load contactCache 12ms
startup-load channelSettings 37792ms <-- here
startup-load cachedChannels 0ms
startup-load channelMessages 21ms
startup-load unreadState 0ms
startup-load discoveredContacts 11ms
No frame handler exceeded 100ms, confirming the stall was local storage work
and not radio traffic.
The remove now happens only when a legacy key actually exists, alongside the
write that migrates it. contact_settings_store had the identical pattern per
contact and is fixed the same way.
This also explains the platform gap in the original report: Android's backend
batches into native storage, so the same code costs almost nothing there,
while every no-op remove on Windows is a full file rewrite.
flutter analyze clean, dart format clean, 469 tests pass.
The firmware path-len byte is packed and self-describing (src/Packet.h:79-84):
high 2 bits = hash size - 1, low 6 bits = hash COUNT (hops), byte length =
count * size. The app treated the low 6 bits as a byte count and ignored the
width, on the strength of a comment asserting the high bits were "not reliably
populated". Both halves of that comment were wrong.
Receive: Contact.fromFrame read `count` bytes where firmware meant `count`
hops, so at 2-byte width it kept HALF of every path and discarded the rest.
DAYTON VA's 2-hop route became a 2-byte fragment, which is why every repeater
login in Bandit's capture timed out.
Send: buildUpdateContactPathFrame wrote the count raw, leaving mode bits 00.
That tells the radio "1-byte hashes" while handing it 2-byte hash data, so it
routed to nodes never on the route. This is the likelier cause of the observed
"routes via c6 and 5c separately" behaviour.
Also fixes a third unit inconsistency: setContactPath wrote customPath.length
(bytes) into pathLength (hops) after every path set.
Contact now carries pathHashWidth per path, so a stored path can never be
re-sliced at a width it was not captured at. realHopCount() is removed rather
than adjusted: it divided an already-correct hop count, halving it at 2-byte
width, and every call site was compensating for a bug that no longer exists.
Migration (Ben's call, option A): records written before this change have no
pathHashWidth and hold truncated bytes that cannot be recovered by
reinterpretation, so their path is dropped and the contact reverts to flood
until the radio re-supplies it. Logged, not silent.
Tests asserted the old semantics (expect(realHopCount(2, 2), 1)) and were
rewritten against firmware truth rather than adjusted to keep passing. Added
send-side encoding coverage, which did not exist. 456 tests pass.
Refs #240, #279, #299
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every received DM/channel message now logs arrival wall-clock vs the
sender-claimed payload timestamp (delta + TIME ANOMALY marker beyond
24h), routed directly to AppDebugLogService so the rotating on-disk
trail is unconditional. Models persist a nullable rxTime set at every
live ingest construction site; legacy stored records stay null.
Diagnostic child of #280.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
App-global (NOT per-device) block list, source of truth for the client:
- BlockStore persists blockedKeys (pubkey hex) + blockedNames (name->first-seen
epoch, for A7 promote-and-prune) via PrefsManager, global keys.
- BlockService (ChangeNotifier): isBlocked/isNameBlocked/block/unblock/
blockName/unblockName + unmodifiable accessors.
- Wired into main.dart MultiProvider.
Epic A #165 / A1 #168. Design: docs/architecture/block-contract-as-built.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Gemini review flagged setPublicKeyHex using `value.length > 10`, which would
store an empty scope for an exactly-10-char device key (unreachable today —
MeshCore public keys are 64 hex — but inconsistent with ChannelStore, which
uses >=10). Align to >=10 for consistency and to remove the latent off-by-one.
Channel message history was persisted keyed by the reusable slot index, so a
channel added to a slot that once held another channel inherited its messages.
Storage now keys by the channel's PSK identity via a resolver on the store
(index -> channel PSK), leaving all call sites unchanged. A one-time migration
adopts pre-existing slot-index history into the PSK key on first load (only when
the PSK is known); clearChannelMessages now wipes both the PSK key and the raw
slot-index key so Build-B's reuse-clear can't be re-migrated onto a new occupant.
Build-A (durable) of epic #190; supersedes Build-B's (#193) guard. Also removes
the dead device-wide fallback read in loadChannelMessages.
this adds the actual last modified timestamp when present, before we used
last advert time as last modified in error
also sets _pendingInitialContactsSync to true on first connect over BLE
- Introduced translation functionality in chat screen, allowing users to translate messages before sending.
- Added MessageTranslationButton to the input bar for enabling/disabling translation.
- Implemented translation service to handle incoming and outgoing text translations using llama models.
- Enhanced message storage to include original and translated text, language codes, and translation status.
- Created UI components for displaying translated messages and managing translation options.
- Added translation model management, including downloading and storing models locally.
- Updated app settings to manage translation preferences and model selections.
- Added storageUsedKb and storageTotalKb properties to MeshCoreConnector.
- Updated battery and storage frame parsing with improved error handling.
- Refactored log RX data handling to use BufferReader for better readability and error management.
- Enhanced message parsing in ChannelMessage and Message classes to utilize BufferReader.
- Introduced new text type for signed messages in meshcore_protocol.dart.
- Updated BLE debug log screen to use BufferReader for payload parsing.
- Refactored message retry service to handle ACK hashes as integers instead of Uint8List.
- Improved message storage serialization and deserialization to accommodate new expectedAckHash type.
- Added wasPulled property to Contact model for better state management.
* Refactor contact handling: replace DiscoveryContact with Contact, update related methods and settings
* Enhance contact handling: include latitude, longitude, and last modified timestamp in contact updates; refactor path handling to accommodate discovered contacts across multiple screens
* Enhance SNRIndicator: include discovered contacts in name resolution for repeaters
* Refactor path handling: replace addReturnPath with buildPath to improve path construction logic and handle target contact types
* Update lib/screens/map_screen.dart
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Add localization for "Show Discovery Contacts" in multiple languages and refactor location plausibility check in map screen
* Enhance contact management: update discovered contacts' active status and improve contact handling with flags and raw packet data
* Refactor ChannelsScreen: pass ChannelMessageStore to buildExpandedContent and ensure messages are cleared after channel creation
* Update MapScreen: adjust label zoom threshold and refactor guessed marker building to include labels
* Refactor ChannelsScreen: change channelMessageStore to a private getter and update its usage in buildExpandedContent calls
* Enhance location plausibility check: add latitude and longitude bounds to ensure valid coordinates
* Update lib/connector/meshcore_connector.dart
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Refactor MeshCoreConnector and related stores: update discovered contacts handling, migrate legacy keys, and set public key in community store
* Refactor MeshCoreConnector and ChannelsScreen: update discovered contacts handling and set public key in community store; enhance location plausibility check in MapScreen
* Update CMD_ADD_UPDATE_CONTACT frame format to include optional latitude and longitude fields
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- Translated contact settings and related strings in Slovenian, Swedish, Ukrainian, Chinese, Dutch, Polish, Portuguese, Russian, and Slovak.
- Added new strings for discovered contacts actions such as adding, copying, and deleting contacts.
- Enhanced the DiscoveryContact model to include a rawPacket field for better data handling.
- Updated the contacts screen to support new actions in the context menu for discovered contacts.
- Improved the contact discovery store to handle the serialization of the new rawPacket field.
- Implemented contact settings in localization files for Swedish, Ukrainian, and Chinese.
- Added new DiscoveryContact model to handle discovered contacts.
- Created DiscoveryScreen to display discovered contacts with filtering and sorting options.
- Integrated contact discovery storage to persist discovered contacts.
- Updated settings screen to include options for automatic contact addition.
- Enhanced app bar and list filter widgets for better user experience.
- Fixed variable naming inconsistencies in contact model.
formats all dart files using `dart format .` from the root project dir
this makes the code style repeatable by new contributors and makes PR review easier
- Implement Community model for managing community data, including secret handling and PSK derivation.
- Create CommunityQrScannerScreen for scanning and joining communities via QR codes.
- Develop CommunityStore for persisting community data using SharedPreferences.
- Introduce QrCodeDisplay widget for displaying QR codes with customizable options.
- Add QrScannerWidget for reusable QR code scanning functionality with validation and controls.
Core Features
Unread Message Tracking: Added persistent unread counts for contacts and channels with visual badges
Message Deletion: Users can now long-press to delete individual messages in chats and channels
SMAZ Compression: Added per-contact compression settings (previously only channels)
UTF-8 Length Limiting: Text inputs now enforce protocol byte limits correctly
Channel Message Paths: New screen to visualize packet routing through repeater network with map view
Protocol Updates
Added maxContactMessageBytes() and maxChannelMessageBytes() helpers for message length validation
Changed channel PSK format from Base64 to Hexadecimal (breaking change)
Added app version field to connection handshake frame
UI Improvements
Unread badges on all contact and channel list items
Enhanced message bubbles with path visualization for channel messages
Character count displays in message input fields
Improved repeater CLI screen functionality
New Files
lib/storage/unread_store.dart - Unread tracking persistence
lib/storage/contact_settings_store.dart - Per-contact SMAZ settings
lib/widgets/unread_badge.dart - Unread count indicator
lib/helpers/utf8_length_limiter.dart - Byte-aware text input formatter
lib/screens/channel_message_path_screen.dart - Packet path visualization
Open-source Flutter client for MeshCore LoRa mesh networking devices.
Features:
- BLE device scanning and connection
- Nordic UART Service (NUS) integration
- Material 3 design with system theme support
- Provider-based state management
- Placeholder screens for chat, contacts, and settings
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Strycher
HDDen
Enot (ded) Skelly
zjs81
Winston Lowe
ericz
zjs81
446564