OCR Direction 2: persisted, growing per-writer model (store schema + scaling) #13
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Depends on
#17 OCR Direction 2: richer persisted store — letter + join templates, WordReader synthesis, versioned migration
daniel/obsidian-remarkable-round-trip
Reference
daniel/obsidian-remarkable-round-trip#13
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Part of #5
Part of #5 (Direction 2). Research + split, then MRs. Builds on the Direction-1 MR (!12); best started after !12 lands so the letter-template shape is settled.
Context (from the #5 spike)
Today the per-writer store (
src/ocr/samples.json) keeps ≤3 samples per word (PER_WORDinsrc/ocr/samples.ts) and, after !12, a capped set of per-letter templates. The spike showed accumulated samples clearly help (train-word CER dropped 38%→24% separable, 99%→14% joined once words were seen), and DTW/$P ran the full labelled set in well under a second — so there is headroom to store and match more.What to figure out (then split into MRs)
classifyand DTWWordReader.readlatency as the store grows to realistic sizes (hundreds of letter templates, thousands of word samples). Find where per-remark read time becomes unacceptable for sync, and add shortlisting/caps to stay under budget. Use thespike5.test.tsharness methodology to generate load.SampleStoremigration; bounded growth (the spike store caps areREMARKS_KEPT/CORRECTIONS_KEPT); decide retention policy for letter/join templates.Scope boundary
NOT a new engine or ML model (that was Direction 3 — see the #5 go/no-go, currently no-go). Pure extension of the existing $P/DTW per-writer store.
Validation contract (for each eventual MR)
npx vitest run src/__tests__→ PASS + a test showing the richer store improves a held-out read and a latency assertion keeping per-remark read time within budget. (The #5 issue namesnix develop -c npx vitest …; there is noflake.nix— usenpm install+npx vitest.)❌ Agent run failed
session saved — new comments resume this run
claude referenced this issue2026-10-20 15:25:42 +02:00
claude referenced this issue2026-10-20 15:26:31 +02:00
claude referenced this issue2026-10-20 15:26:32 +02:00
What I found
Investigated the two read-time hot paths and the store's migration surface, with measured latency (throwaway probe,
npm install+ vitest, rm2 traces — the repo has noflake.nix).WordReader.readis the whole-word bottleneck (src/ocr/words.ts:218): it runs a full DTW over every owner sample, gated only by a width ratio — unlikereadFonts, which profile-shortlists to 150 before DTW. Linear in sample count: 5.7 ms @50 → 28.8 ms @1000 → 80 ms/word @3000. A 10-word remark at 3000 samples ≈ 0.8 s of owner-DTW per remark. → shortlist owner samples before DTW.classify's quick pass is the letter bottleneck (src/ocr/recognize.ts:254): it scores shape over all templates before keeping the top 24, andsegmentcalls it per 1–3-cluster window.recognize(1 word)is linear in template count: 92.7 ms @216 (today's fonts) → 349 ms @1216 → 615 ms @2216 (~0.24 ms/template/word). !12's learned letter templates land exactly here. → bound the quick pass (metric pre-filter and/or per-char cap).loadSamples(src/inkedge.ts:315) casts raw JSON straight toSampleStore; theformat: 1literal (samples.ts:99) is never checked. Adding optional fields is backward-compatible, but the letter/join-template schema change needs a realmigrate()seam.samples.ts:PER_WORD=3(newest-winsslice),REMARKS_KEPT=500,CORRECTIONS_KEPT=2000. Newest-3 wastes the owner's writing-style variation.Split rationale: the two latency fixes and the word-sample retention change are independent of !12 and ship now; the richer schema (persisted letter + join templates,
WordReadersynthesis of unseen words, versioned migration) depends on !12's letter-template shape — which, as of now, is not settled (!12's branch has only thespike5.test.tsharness, nolearnCorrection→template persistence yet), so it's filed held.Created
WordReader.read; capclassify's template work. Red-gate latency assertions + accuracy guard. (independent of !12)PER_WORD. Held-out read improves vs newest-3. (independent of !12;samples.tsonly)WordReadersynthesis of unseen words, versionedmigrate()seam + retention policy. Held (dispatch:false) — dispatch only after !12 merges and the letter-template shape is readable insamples.ts. Needs further research + splitting (suggested sub-MRs in the issue).!15 and !16 are independent of each other (different files) and can land in parallel.
Follow-ups
is_blocked_bycan't point at an MR and GitLab link-gating needs Premium, so this hand-off is a manual nudge rather than an auto-gate.claude referenced this issue2026-10-20 15:43:00 +02:00
claude referenced this issue2026-10-20 15:43:09 +02:00
Agent failed ❌
exceeded pending task timeout
🤖 Agent is working on this — turn 14
Last activity: tool: mcp__gitlab__forge_close_issue — {} — updated 18:11:52 UTC
Since my last update: every hand-off from #13 has merged. That covers !15 (read-latency shortlisting:
SAMPLE_SHORTLIST=50inwords.ts, with latency asserts inscaling.test.ts), !16 (diversity-based word retention,PER_WORD=10), and via #17, !18 (letter templates kept by diversity throughkeepDiverse,PER_LETTER=6) and !19 (versionedmigrate()seam,CURRENT_FORMAT=2, wired intoloadSamples). Only the join/bigram synthesis bet is left. It runs as a chain: #17 waits on #20, which waits on the spike #21 (another agent is on it now). #21 decides whether thejoins?schema MR and the synthesis MR ship at all, and "no" is a valid answer. I've linked #13 is_blocked_by #17 so this issue stays parked until that chain closes. Nothing more to do here until then.Since my last update: #17 has closed, which unblocks this issue, and all three parts of #13 are now resolved, so I'm closing it.
PER_WORD = 10(!16), and letter templates the same way throughkeepDiversewithPER_LETTER = 6(!18). Join templates are a no-go: the spike in #21 showed there is no reliable way to label bigrams in joined ink, and synthesis using the bigrams that can be collected did no better than fonts-only. #20 and #17 were closed on that result, so nojoins?field was added.SAMPLE_SHORTLIST = 50keepsWordReader.readbounded, and the cost ofclassifyis capped. The latency and accuracy checks are insrc/__tests__/scaling.test.ts(!15).migrate(raw)step withCURRENT_FORMAT = 2, called fromloadSamples(src/inkedge.ts:321). A format-1 store upgrades without loss, covered bysrc/__tests__/migrate.test.ts(!19).I checked all of these on
mainateb84e3a.Note for the owner of #5 (not filed as an issue): #17 found that the
WordReaderfont pass is effectively never trusted on real handwriting. The correct word scores 0.22–0.40 againstFONT_TRUST = 0.08(src/ocr/words.ts:180). In practice, whole-word reads of joined writing come only from the owner's own word samples. Any next OCR direction should start from that fact. The trust rule based on the gap to the runner-up word, described in #21, only pays off once there is labelled material from the owner.