OCR Direction 2: richer persisted store — letter + join templates, WordReader synthesis, versioned migration #17
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Blocks
Depends on
#13 OCR Direction 2: persisted, growing per-writer model (store schema + scaling)
daniel/obsidian-remarkable-round-trip
#20 OCR Direction 2: join/ligature templates + WordReader synthesis of unseen words (the creative bet)
daniel/obsidian-remarkable-round-trip
Reference
daniel/obsidian-remarkable-round-trip#17
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Part of #13
Part of #13 (Direction 2). Research + further split, then MRs. This is the hard half of #13 part 1 and the schema half of part 3.
What this covers (and must split into MRs)
The scaling MR (!15) bounds read-time work and the diversity MR (!16) grows word samples. This issue is the richer schema:
SampleStore, with a diversity-preserving per-character cap (mirror !16's "keep N most diverse, not newest" for word samples — seeaddSample,src/ocr/samples.ts:119) so the owner's letter set grows but stays bounded. Wire the persisted letter templates intoocrTemplates()(src/main.ts:1896) alongside the hand + fonts.splitLineInto,src/ocr/recognize.ts:482), and extendWordReader(src/ocr/words.ts:183) so it can synthesise the trace of an unseen word from the owner's own letters + joins, instead of only the bundled script fonts (written()/readFonts,words.ts:150/:235). Goal: read a word the owner never wrote, in the owner's own hand. Measure the CER gain on held-out words vs fonts-only using thespike5.test.tsmethodology (render a hand = a font; the spike showed joined-word CER 99%→14% once words were seen — the target here is generalisation to unseen words).loadSamples(src/inkedge.ts:315) casts raw JSON straight toSampleStoreand theformat: 1literal (src/ocr/samples.ts:99) is never checked. Adding optional letter/join-template fields is backward-compatible, but any shape change needs a realmigrate(raw): SampleStoreseam hooked intoloadSamples, bumpingformat. Add the seam here (first format bump), with a format-1 fixture that upgrades losslessly.PER_WORD/REMARKS_KEPT/CORRECTIONS_KEPTconsts.Suggested split (for the agent that picks this up)
migrate()seam + persisted letter-template field & retention, wired intoocrTemplates().WordReadersynthesis of unseen words + CER test. (This is the research-heavy one; may split again.)Scope boundary
NOT a new engine/ML model (Direction 3, no-go per #5). Pure extension of the existing $P/DTW per-writer store. Read latency must stay within !15's budget — if letter/join templates push a hot path over, shortlist there, don't regress !15.
Validation contract (each eventual MR)
npx vitest run src/__tests__→ PASS + a test showing the richer store improves a held-out read (an unseen word synthesised from owner joins, and/or a letter-template read of a never-seen word) and a latency assertion keeping per-remark read time within budget. (Noflake.nix; usenpm install+npx vitest.)claude referenced this issue2026-10-20 15:33:40 +02:00
claude referenced this issue2026-10-20 15:34:21 +02:00
claude referenced this issue2026-10-20 15:34:22 +02:00
✅ Agent run completed
6m11s · 27 turns · $2.28 · pushed
agent/ocr-letter-diversity,agent/ocr-store-migrate· session saved — new comments resume this runWhat I found
SampleStore.lettersis populated bylearnLetters(src/ocr/samples.ts:199) and read vialettersTemplates(recognize.ts:609) wired intoocrTemplates()(main.ts:1918, cache-keyed on learned-letter count). So this issue's remaining work is narrower than the description assumed..slice(-PER_LETTER=3),samples.ts:215), not diversity-preserving like !16's word retention; (2) there is still nomigrate()seam —loadSamples(inkedge.ts:315) casts raw JSON and theformatliteral is never checked; (3) joins +WordReadersynthesis of unseen words is entirely unbuilt (WordReaderonly synthesises from fonts,words.ts:150/:256;store.lettersis unused there).Created
feat(ocr): diversity-preserving per-letter retention(part 1 remainder + part 4 for letters; generalisekeepDiverseto a cap, use it inlearnLetters, red-gate test). Branchagent/ocr-letter-diversity.refactor(ocr): versioned migrate() seam for the sample store(part 3;migrate(raw)+CURRENT_FORMAT=2, routed throughloadSamples, lossless format-1 fixture, red-gate test). Branchagent/ocr-store-migrate.WordReadersynthesis of unseen words + held-out CER measurement (the creative bet; soft-ordered after !18/!19; a negative CER result is a valid "do not ship" outcome).Follow-ups
main); a trivialsamples.tsconflict on merge is expected and fine.keepDiverse(samples, cap)and !19'smigrate()seam for itsjoins?field — its MRs should land after !18/!19 to avoid conflicts. No hardis_blocked_bygate set (soft dep on MRs;joins?is an optional backward-compatible field).blocksit); !18/!19 arePart of(notCloses), so they won't auto-close it.claude referenced this issue2026-10-20 18:59:13 +02:00
claude referenced this issue2026-10-20 18:59:13 +02:00
🤖 Agent is working on this — turn 28
Last activity: tool: mcp__gitlab__forge_close_issue — {} — updated 18:10:18 UTC
Last message:
Since my last update: every part of this issue is now resolved, so I'm closing it.
ocrTemplates()): shipped by !12. Its diversity-preserving retention shipped in !18 (merged). Onmain,keepDiverse(samples, cap)handles bothPER_WORDandPER_LETTER = 6(src/ocr/samples.ts:214,:272).migrate(raw)andCURRENT_FORMAT = 2are insrc/ocr/samples.ts:136/:176,loadSamplescalls it (src/inkedge.ts:321), andsrc/__tests__/migrate.test.tschecks that a format-1 store upgrades without loss.spike/21-bigram-probe(evidence only, not for merge).I'm not filing the two leads from #21 as issues; they are left for #13's owner:
store.wordscoverage. Words are already learned from every accepted or corrected remark (learnCorrection,src/ocr/samples.ts:421). The remaining gap is lines where the ink and text word counts differ by 2 or more, so nothing is learned. Those cases are already shown to the owner throughdescribeUnlearned. There is no specific change to hand off yet.Side finding for #13: on any owner hand that isn't literally a bundled font, the font pass of
WordReaderis never trusted. The correct word costs 0.22–0.40 againstFONT_TRUST = 0.08(src/ocr/words.ts:180). On real joined handwriting, the word reader effectively reads only from the owner's own word samples.claude referenced this issue2026-10-20 20:11:50 +02:00