feat(ocr): handwriting test capture records the endpoint model, per-line read time and failure reason #39
No reviewers
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
daniel/obsidian-remarkable-round-trip!39
Loading…
Reference in a new issue
No description provided.
Delete branch "agent/capture-endpoint-model-latency"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Part of #14
Why
The owner's second capture (
handwriting-samples/test-sheets/2026-10-21-004942/) is the first one with vision-endpoint readings. On the same 18 lines (the ink is byte-identical to2026-10-21-001026), mean CER per reader was:The capture can't say which model produced those numbers, or how long each read took. #14's revisit gate, condition 1, is "the local VLM is unacceptable (privacy, latency, setup friction)". Comparing models later (qwen2.5vl:7b vs 3b, or an LM Studio model) needs the model name in the file. On top of that,
saveTestSheetswallows endpoint errors (.catch(() => null),src/main.ts≈ line 2228). So the capture records a failed line asnullwith no reason, and aNO_TEXTanswer also comes out asnull, which counts as "failed".What to change
src/ocr/testcapture.tsCaptureReadinggets an optionalms?: number: the wall-clock read time of that reader for that line.HandwritingCapturegets an optionalendpoint?: { model: string }. Record the model name only, never the endpoint URL or API key: the capture gets attached to a public-ish forge issue.errors?: Partial<Record<ReaderName, string>>, or an equivalent you prefer. A failed read staysnullinreadings, sosummarize()'s "failed" count keeps its meaning, and the error message is kept next to it.version: 1. Every new field is optional, so old captures still parse, andparseCapturemust accept both the old and new shapes.captureMarkdownmsper reader, or—when absent.Endpoint model: qwen2.5vl:7b.src/main.tssaveTestSheetrecognize, bothwritePad.readcalls together, andthis.ink.ocr. Useperformance.now()orDate.now().capture.endpoint = { model: this.settings.ocrModel }whenocrMode === "endpoint"..catch(() => null)with a catch that recordstoTransportError(error).message(orString(error)) in the item's errors.nullfromink.ocrmeans NO_TEXT, which is a valid answer, so record it as{ text: "", accepted: false }and not as a failure.Constraints and decisions
Held out, as before: nothing is learned or remembered (see the doc comment on
saveTestSheet).No URL, host or API key in the capture.
Don't change the reading logic, the trust thresholds or the prompt. This MR is measurement only.
Out of scope:
*quick*on p1-02, invented quotes on p2-06);Both are possible follow-ups once this lands.
Validation contract
npx vitest run src/__tests__/testcapture.test.ts→ PASS. Add tests that:parseCapturewithms,endpoint.modeland an error message;captureMarkdownshows the mean time column, the model line and the error message;—for time. The committedhandwriting-samples/test-sheets/2026-10-21-004942/handwriting-test-2026-10-21-004942.jsonis a real old-shape fixture.maintoday, because there's no time column or model line.npx vitest run(full suite) → all pass.npx tsc --noEmit -skipLibCheckandnpx eslint src→ clean.flake.nix, so use plainnpm ci && npx …and notnix develop.🤖 Generated with Claude Code
🤖 Agent is working on this — turn 67
Last activity: assistant message — updated 23:09:06 UTC
Last message:
claude referenced this pull request2026-10-21 01:04:37 +02:00
What changed
maininto this branch (no conflicts in the files) to clear the conflict flag.src/ocr/testcapture.ts: three new optional fields, so old captures still parse.CaptureReading.msis the read time.HandwritingCapture.endpoint.modelis the model name only.CaptureItem.errorsholds the error message, while the reading staysnullso the "failed" count means the same.summarize()gainsmeanMs. The note now has a Mean time column (—when absent), anEndpoint model: …line and an Errors list below the readings.src/main.tssaveTestSheet: times each reader (recognize; bothwritePad.readcalls together;ink.ocr) and records the model when the mode is endpoint. Endpoint errors are now saved instead of dropped. ANO_TEXTanswer (null) is now saved as{ text: "", accepted: false }, not as a failure.toTransportError(...).messagecan include the endpoint address (Could not reach the OCR endpoint at http://…,ECONNREFUSED 192.168.x.y:11434). A new helper,captureError(), replaces the endpoint setting, its host/hostname, the API key and anyhttp(s)://URL with[endpoint]before the message is saved. It has its own test.nullwith the error "The line could not be drawn as a picture." (before, it wasnullwith no reason).Verify
The new tests in
testcapture.test.tsfailed before the change and pass now. They cover the round-trip withms, model and error; the note's time column, model line and error; the old-shape fixture2026-10-21-004942, which renders—; and the redaction. Full suite: 842 pass.tscandeslint srcare clean. No CI pipeline ran on this branch.saveTestSheetitself was only type-checked, not run on a device.Follow-ups
None. Both out-of-scope items in the description (VLM prompt artifacts, endpoint mode for the offline replay) remain possible follow-ups.