OCR calibration: stop the $P letter reader passing junk cursive above OCR_TRUST (+ labelled fixtures) #10
No reviewers
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
daniel/obsidian-remarkable-round-trip!10
Loading…
Reference in a new issue
No description provided.
Delete branch "agent/ocr-letter-calibration"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Part of #4 (which is Part of #2). This is work item 2 of 3. Recalibrate the $P per-letter reader (
src/ocr/recognize.ts) so mis-segmented joined-up writing fails safe to the image instead of leaking junk text above the trust threshold. Sibling MR !? handles the WritePad path; this MR is only the letter reader + fixtures.Measured evidence (I ran this during investigation — reproduce it, don't take it on faith)
Rendering text in bundled fonts as device strokes (the same
write(text, font, jitter)helper the existing tests use) and reading viarecognize(strokes, templates, lexicon):The leak: print scores 0.96–0.97, but one cursive case scores 0.669 ≥ 0.65 with total garbage (
"C%% @w mm %mmm", mostly symbol glyphs%/@). There is a wide clean gap between legitimate print (0.96+) and the junk leak (0.669), so raisingOCR_TRUSTplugs it with large margin and no risk to print.The existing test
recognize.test.ts:59only exercisesallurecursive (which scores 0.527, correctly below) — it never caught thehershey-scriptleak. That is the calibration gap.What to do
src/__tests__/fixtures/— a small JSON (or afixtures/handwriting.tsmodule) listing labelled samples:{ text, font, jitter, kind: "print" | "cursive" }. Seed it with the cases above plus a few more per class so the calibration is regression-tested against a set, not one point. (Strokes are generated deterministically from bundled fonts via thewritehelper — seerecognize.test.ts:7— so the fixture only needs the parameters, not raw stroke dumps. Keep it this way; it stays diffable.)recognize.test.tsthat renders thehershey-script"Check this again tomorrow" fixture and assertsrecognize(...).confidence < OCR_TRUSTand that it does not return the correct text. Watch it fail on today's code (0.669 ≥ 0.65) before changing anything — that proves the gate bites.OCR_TRUST(src/ocr/recognize.ts:431) to a value that sits in the gap(0.669, 0.961)— ~0.75 gives headroom on both sides. Re-run the whole labelled set and confirm: everyprintfixture stays above the new threshold (the existing "reads print letters" test atrecognize.test.ts:39assertsconfidence > OCR_TRUSTand must still pass), everycursivefixture lands below. Pick the number from the measured separation and state the justification in a code comment next to the constant (print min vs cursive max observed). Do not blind-bump.Scope boundary
src/ocr/recognize.ts+src/__tests__/fixtures/+recognize.test.ts. Do not touchwritepad.ts,main.ts, or settings (sibling MRs).OCR_TRUSTsends more cursive to the WritePad fallback, so the WritePad quality gate (sibling MR !?) is what ultimately catches what this MR lets fall through. They are validated together at the end of #4; each has its own unit red gate.Validation contract
nix develop -c npx vitest run src/__tests__/recognize.test.ts→ PASS, including the new red-gate test (which fails onmain).nix develop -c npm test→ PASS (no regressions).Note: this repo has no
flake.nix—nix developfails. Run directly:npm cithennpx vitest run src/__tests__/recognize.test.ts(Node 24 on PATH). Verified during investigation.Part of #4.
✅ Agent run completed
8m19s · 39 turns · $2.69 · pushed
agent/ocr-letter-calibration· session saved — new comments resume this runclaude referenced this pull request2026-10-20 15:22:09 +02:00
claude referenced this pull request2026-10-20 15:22:46 +02:00
claude referenced this pull request2026-10-20 15:22:46 +02:00
What changed
OCR_TRUST0.65 → 0.78 insrc/ocr/recognize.tsso mis-segmented cursive fails safe to the image instead of leaking junk text. Justified inline by the measured class separation.src/__tests__/fixtures/handwriting.ts(params only — strokes rendered deterministically via thewritehelper, so it stays diffable).recognize.test.ts: a red gate on the documented leak (hershey-script"Check this again tomorrow"), and a whole-set print-vs-cursive separation test.Calibration note (I calibrated against the set, not one point): the hand-off's single 0.669 leak point was optimistic. Sweeping a wider sample, the real cursive tail reaches 0.739 (
hershey-script"we should talk soon") and the print floor is 0.831 (hershey-sans"update the spreadsheet now"). So the clean gap is (0.739, 0.831) and 0.75 would be too tight (0.011 margin). 0.78 sits centred in the real gap (~0.05 below the print floor, ~0.04 above the cursive ceiling). A per-letter-cost ceiling would not catch the leak (its per-letter cost ≈0.20 is low, symbol glyphs match cheaply), so the threshold is the right guard — no secondary guard added.Verify
npx vitest run src/__tests__/recognize.test.ts→ 8 pass, incl. the red gate (verified it fails onmainat 0.669 ≥ 0.65 before the constant change, passes after).npm test→ 57 files / 655 tests pass, no regressions.tsc -noEmitandeslinton the 3 changed files clean.upstream-sync/releaseworkflows); the lone red run is pre-existing onmain, unrelated to this diff.Follow-ups
OCR_TRUSTroutes more cursive to the WritePad fallback, so the WritePad quality gate in the sibling MR is what ultimately catches what this MR lets fall through — validated together at the end of #4.claude referenced this pull request2026-10-20 16:34:09 +02:00
claude referenced this pull request2026-10-20 16:34:09 +02:00
claude referenced this pull request2026-10-20 18:57:33 +02:00
claude referenced this pull request2026-10-20 18:59:19 +02:00