OCR: stop the built-in reader emitting junk on joined-up writing — calibrate confidence & make readings diagnosable #4
Labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
daniel/obsidian-remarkable-round-trip#4
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Part of #2
Part of #2. Directly addresses the top complaint in #2: the built-in reader "is not very reliable and mostly detects junk." The upstream philosophy is correct — show the ink image unless the reading is trustworthy — so "junk" means wrong readings are leaking past the trust thresholds instead of falling back to the image. The fix is calibration + diagnosability, not a new engine.
Root cause (from #2 investigation)
main.tsrecognizeLocally()runs, inocrMode: "builtin":recognize()($P per-letter reader,src/ocr/recognize.ts) → accept ifconfidence >= OCR_TRUST(0.65).src/ocr/writepad.ts) → accept ifconfidence >= WRITEPAD_TRUST(0.7).null→ image only.The letter reader segments words into letters by horizontal gaps (
clusterStrokes/segmentinrecognize.ts) — joined-up cursive has no gaps, so it mis-segments and the confidence1 - perLetter/0.6can still land above 0.65 on a wrong reading; WritePad can likewise pass 0.7 with a junk reading nudged up by the vault-word bonus (VAULT_BONUS/VAULT_MARGINinwritepad.ts). The existing testsrc/__tests__/recognize.test.ts:59asserts cursive should score belowOCR_TRUST, so the leak is a calibration gap in the real-stroke regime, plus an invisible WritePad fallback nobody can see firing.What to do (needs investigation → split into MRs)
readBy/readScore/by/score(src/ocr/samples.ts). Surface, per remark, which reader produced the text and its confidence — in the import report (src/incoming/report.ts) and/or optionally in the callout — behind a debug/verbose toggle. Without this, neither the user nor a tuner can tell builtin-junk from WritePad-junk.OCR_TRUST/WRITEPAD_TRUST, add a per-letter-cost ceiling that forces image fallback on mis-segmented cursive, and/or gate WritePad behind its own quality check. Do not blind-bump a constant — justify each threshold against measured reads. A small labelled fixture set should be added undersrc/__tests__/fixtures/so the calibration is regression-tested.builtin. Decide (design question for the implementer) whether to make it a visible mode or a toggle, so "junk" can be attributed and turned off.Scope boundary
NOT the vision-endpoint UX (#3) and NOT the trainable-reader R&D (the other #2 child). This issue is purely: make the existing local readers fail safe to the image and be diagnosable.
Validation contract (for the eventual MRs)
nix develop -c npx vitest run src/__tests__/recognize.test.ts src/__tests__/writepad.test.ts→ PASS, including a new test proving a known-cursive fixture returnsnull/below-trust (red gate: that fixture currently returns junk text above threshold).builtin/writepad/endpoint+ confidence per remark in the import report.✅ Agent run completed
9m14s · 44 turns · $2.93 · pushed
agent/ocr-diagnostics,agent/ocr-letter-calibration,agent/ocr-writepad-gate· session saved — new comments resume this runclaude referenced this issue2026-10-20 15:13:52 +02:00
claude referenced this issue2026-10-20 15:20:08 +02:00
What I found
recognize(): print scores confidence 0.96–0.97, buthershey-scriptcursive "Check this again tomorrow" scores 0.669 ≥ OCR_TRUST (0.65) and returns garbage"C%% @w mm %mmm". The wide gap means raisingOCR_TRUST(~0.75) plugs it cleanly — print has huge headroom. The existingrecognize.test.ts:59only testsallurecursive (0.527, correctly below), so it never caught thehershey-scriptleak."meeting is an Monday"scores 0.682 (rejected) while partial-junk"the quick brown po,frjunql5"scores 0.720 ≥ WRITEPAD_TRUST (0.7), accepted. The fix must be a quality gate (known-word fraction), exactly as the issue anticipated — not a bumped constant. Note: raisingOCR_TRUSTpushes more cursive onto WritePad, so these two must fail-safe together.readBy/readScoreinsrc/ocr/samples.ts) — it just isn't surfaced anywhere. Diagnosability is a clean, independent first MR.Created
Three MR hand-offs, each self-contained with measured evidence, red gates, and a validation contract:
builtin/writepad/endpoint+ confidence per remark in the import report behind anocrDiagnosticstoggle. (the "do first" piece — unblocks tuning; threads reader/score out ofrecognizeLocally)src/__tests__/fixtures/, red-gate test (hershey-scriptjunk currently passes), raiseOCR_TRUSTinto the measured 0.669–0.961 gap with justification. (touches onlyrecognize.ts+ fixtures/tests — no conflicts)ocrUseWritePadtoggle so it's attributable/switchable, red-gate test.Recommended merge order: !9 → then !10 / !11. !9 and !11 both touch
main.ts:recognizeLocally+ settings; whichever lands second rebases (adjacent edits, no logic conflict). !10 is independent. MR-to-MR order can't be gated viais_blocked_by(issue-only), so this is advisory — noted in each MR.Follow-ups
OCR_TRUSTsending more cursive to a gated WritePad) is validated end-to-end once !10 + !11 both land — each carries its own unit red gate meanwhile.flake.nixdoes not exist in this repo, sonix develop -c …(the validation-contract command style) fails; tests run vianpm ci && npx vitest run …on Node 24. Flagged in every MR.claude referenced this issue2026-10-20 15:26:31 +02:00
claude referenced this issue2026-10-20 15:26:32 +02:00
claude referenced this issue2026-10-20 15:29:42 +02:00
claude referenced this issue2026-10-20 15:29:42 +02:00
claude referenced this issue2026-10-20 15:30:07 +02:00
claude referenced this issue2026-10-20 15:30:07 +02:00
claude referenced this issue2026-10-20 16:33:01 +02:00
claude referenced this issue2026-10-20 16:34:00 +02:00
claude referenced this issue2026-10-20 16:34:09 +02:00
🤖 Agent is working on this — turn 0
Last activity: session init — updated 16:57:10 UTC
🤖 Agent is working on this — turn 8
Last activity: tool: mcp__gitlab__forge_close_issue — {} — updated 16:59:23 UTC
Last message:
Since my last update: all three hand-offs (!9 diagnostics, !10 letter-reader calibration and fixtures, !11 WritePad quality gate and toggle) are merged to main (checked at
db6c9a7). I ran the validation suite on main:npx vitest run src/__tests__/recognize.test.ts src/__tests__/writepad.test.ts src/__tests__/report.test.tsgave 39 passed / 0 failed. The repo has noflake.nix, sonix develop -cdoesn't work and I rannpx vitestdirectly.That run includes these tests:
recognize.test.ts:69and:80.OCR_TRUSTis now 0.78. Measured scores were 0.96–0.97 for print and 0.669 for the leaking cursive.writepad.test.ts:68checks the known-word-fraction gate, which sits on top ofWRITEPAD_TRUST.report.test.ts:55checks that the report gets areaders: builtin N (avg x), writepad N (avg y), endpoint Nline whenocrDiagnosticsis on and none when it is off.The report shows a count and average confidence per reader for each imported file, not one line per remark. That is enough to tell which reader produced the junk. WritePad can be turned off with
ocrUseWritePad. The validation contract is met, so I'm closing this issue.claude referenced this issue2026-10-20 19:00:28 +02:00
claude referenced this issue2026-10-20 19:00:28 +02:00
claude referenced this issue2026-10-20 20:06:07 +02:00
claude referenced this issue2026-10-20 20:06:07 +02:00
claude referenced this issue2026-10-20 20:06:45 +02:00
claude referenced this issue2026-10-21 00:17:38 +02:00
claude referenced this issue2026-10-21 00:19:36 +02:00
🤖 Agent is working on this — turn 10
Last activity: tool: mcp__gitlab__forge_close_issue — {} — updated 04:46:04 UTC
Since my last update: this issue was reopened because the owner's first real handwriting capture (
handwriting-samples/test-sheets/2026-10-21-001026, joined cursive) still let built-in junk through. Two lines were accepted aboveOCR_TRUST: p2-05/IäI→✗rl…,%«%.Jl€at 0.784 and p2-06@:@+m«↑@&↑3]at 0.788. !35 fixes this and is the only work left on this issue. It addsbuiltinTrusted(), which keepsOCR_TRUSTand also rejects any reading that is more than 30% symbols. On this capture the junk readings are 59% and 83% symbols, and the most symbol-heavy of the 18 prompts is 21%. A rejected reading then falls through to WritePad and, failing that, to the image. I read the diff. It touches 3 files, the branch merges cleanly into main (git merge-tree), and its agent reportsrecognize.test.ts11/11 with a red gate seen failing first. No CI pipeline has run on the branch yet. I also checked the WritePad readings it accepted on the same capture (9 of 18, read from the capture JSON). Their error rate (CER) is 9–31%, and they are legible text with some wrong words, not symbol junk. The ink image stays next to them by default (ocrKeepImage: true). WritePad's confidence barely tracks accuracy here: p2-02 was accepted at 0.773 with 29.7% CER, while p2-01 was rejected at 0.678 with only 14.8%. So raisingWRITEPAD_TRUSTwould not help, and better WritePad accuracy on this hand belongs to #14, not here. Nothing new was created. This issue closes when a human merges !35.@claude all done here?
Since my last update: yes, all done. !35 is merged and its symbol-junk gate (
builtinTrusted) is inmain, used by bothrecognizeLocallyand the test-sheet capture.mainalso hase43961b("built-in reader declines joined-up writing instead of guessing"), which adds a regression test running the owner's real capture lines through the reader and expecting it to decline every one (src/__tests__/recognize.test.ts:146).All six MRs for this issue are merged: !9 (reader and confidence shown in the import report), !10 (calibration with labelled fixtures), !11 (WritePad quality gate and toggle) and !35, plus the two Ollama setup MRs !27 and !33. Nothing is left open here, so I'm closing the issue. Making WritePad read joined-up writing more accurately is tracked in #14.