feat(ocr): diversity-preserving per-letter retention #18
No reviewers
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
daniel/obsidian-remarkable-round-trip!18
Loading…
Reference in a new issue
No description provided.
Delete branch "agent/ocr-letter-diversity"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Part of #17 (OCR Direction 2 — richer persisted store), Part of #13.
Context
!12 (merged) already persists per-letter $P templates in
SampleStore.letters(learnLetters,src/ocr/samples.ts:199) and wires them into reads (lettersTemplates→ocrTemplates(),src/main.ts:1918). So issue #17 part 1's "persist letter templates + wire intoocrTemplates()" is already done. What !12 does not do is retain them well:learnLetterskeeps the newestPER_LETTER=3samples per character via.slice(-PER_LETTER)(src/ocr/samples.ts:215). That collapses the owner's letter set onto however they most recently formed a letter — exactly the failure !16 fixed for word samples (keepDiverse,src/ocr/samples.ts:158). This MR closes that gap for letters (issue #17 part 1 remainder + part 4 retention policy for the letter field).What to change (all in
src/ocr/samples.ts)keepDiverseto take a cap. It is currently hard-coded toPER_WORD(while (live.length > PER_WORD), :162). Change the signature to e.g.keepDiverse(samples, cap)and update its one caller inaddSample(:146) to passPER_WORD. No behaviour change for words.keepDiversemeasures redundancy with the cheap $P cloud distance (cloudOf+matchDistance). That is the right space for letters too — they are the same stroke-cloud representation. (Context: the mr-16 codebase note records that retention deliberately uses the $P proxy, not the reader's DTW; keep that.)learnLetters(:199). Replacekept[chars[i]] = [...(kept[chars[i]] ?? []), norm].slice(-PER_LETTER)(:215) with: append the new normalised sample, then if the array exceedsPER_LETTERcallkeepDiverse(kept[chars[i]], PER_LETTER).keepDiversepreserves newest-last order and breaks ties by age, so recency still wins ties — matching the word path.PER_LETTERlike the other caps (PER_WORD/REMARKS_KEPT/CORRECTIONS_KEPT, :43–51). It is currently a bareconst PER_LETTER = 3(:45) with a now-wrong comment ("the oldest goes first"). Rewrite the comment to describe the diversity-preserving per-character cap, and choose the value deliberately: read-time cost is bounded byclassify's metric branch-and-bound pluscapPerChar/TEMPLATE_CAP = 6per char inrecognize.ts(see the mr-15 codebase note). KeepPER_LETTER ≤ TEMPLATE_CAPso no learned letter is silently dropped at read time.6aligns the store cap with the read cap;3stays conservative. Justify whichever you pick in the comment.Scope boundary
migrate()seam — that is the sibling MR onagent/ocr-store-migrate. This change makes no shape change (stillformat: 1, samelettersfield), so it needs no migration.WordReadersynthesis — that is child issue #17's split (join work).lettersTemplatesor theocrTemplates()wiring — only which letter samples survive the cap.Validation contract
npm installthennpx vitest run src/__tests__→ PASS (noflake.nix).Add a test in
src/__tests__/samples.test.tsmirroring the existing word case "drops a redundant near-duplicate, not a distinct sample, when over cap" (:158):learnLetters(vialearnCorrection, the real path) more thanPER_LETTERcorrections producing one character, where one sample is shape-distinct and the rest are near-duplicates of each other, with the distinct one fed early (so newest-wins would drop it). Assert the distinct sample survives instore.letters[ch].main): onmain,.slice(-PER_LETTER)keeps only the newestPER_LETTER, dropping the early distinct sample — so the survival assertion fails before this change. State in the MR that you watched it fail onmainfirst.src/__tests__/letterlearning.test.ts(held-out generalisation) andsrc/__tests__/scaling.test.ts(read-latency budget) green — the retention change must not regress either. No new latency assertion is needed: per-char template count is bounded byPER_LETTER ≤ TEMPLATE_CAP, so read work is unchanged-to-smaller.✅ Agent run completed
6m25s · 33 turns · $2.66 · pushed
agent/ocr-letter-diversity· session saved — new comments resume this runclaude referenced this pull request2026-10-20 17:13:37 +02:00
claude referenced this pull request2026-10-20 17:14:35 +02:00
claude referenced this pull request2026-10-20 17:14:50 +02:00
claude referenced this pull request2026-10-20 17:14:50 +02:00
What changed
keepDiverse(samples, cap)insrc/ocr/samples.ts(was hard-coded toPER_WORD);addSamplenow passesPER_WORD— no behaviour change for words.learnLettersnow appends the new normalised sample and, when over cap, callskeepDiverse(…, PER_LETTER)instead of.slice(-PER_LETTER), so the per-letter store keeps the owner's range of a letter rather than collapsing onto the newest stroke — the same $P-cloud diversity cull !16 gave word samples.PER_LETTERlike the other caps and raised it3 → 6(=TEMPLATE_CAP):capPerCharkeeps the owner's lower-weight learned letters ahead of bundled fonts, so atPER_LETTER ≤ TEMPLATE_CAPevery retained letter sample survives read-time capping — store cap and read cap line up. No shape change (format: 1, samelettersfield), so no migration.Verify
npm installthennpx vitest run src/__tests__→ 678 pass / 0 fail, including the newsamples.test.tscase "keeps a distinct letter shape fed early over later near-duplicates" and the still-greenletterlearning.test.ts(held-out generalisation) andscaling.test.ts(read-latency budget). Red gate watched fail first: I temporarily restored.slice(-PER_LETTER)and the new test failed (the early distinct loop sample was dropped —expected [ …(6) ] to include [distinct]), then restoredkeepDiverse.tsc -noEmitandeslinton the changed files are clean. (No pipeline has run for the head SHA yet; the local vitest run is the validation contract.)Follow-ups
None. Scope-excluded as intended: versioned
migrate()seam (sibling MR onagent/ocr-store-migrate), join/ligature templates (child issue #17), and thelettersTemplates/ocrTemplates()wiring (untouched).claude referenced this pull request2026-10-20 18:51:57 +02:00
claude referenced this pull request2026-10-20 18:52:05 +02:00
claude referenced this pull request2026-10-20 18:52:06 +02:00
claude referenced this pull request2026-10-20 18:59:13 +02:00
claude referenced this pull request2026-10-20 18:59:13 +02:00
claude referenced this pull request2026-10-20 19:00:17 +02:00