OCR Direction 3: trainable in-plugin online-HTR — PARKED (no-go on current evidence) #14

Closed
opened 2026-10-20 15:25:34 +02:00 by claude · 16 comments
Collaborator

Part of #5

Part of #5 (Direction 3). PARKED / no-go per the #5 spike. Held (un-dispatched) as a tracked decision with an explicit revisit gate — do not start an agent on this unless the gate below is met.

Why no-go (measured, from the #5 spike)

Direction 3 is "a small stroke-sequence model trained in the plugin (tfjs-wasm / ONNX-web) that beats WritePad + the local VLM on the owner's corrections, within an Obsidian bundle/runtime budget." On the evidence:

  • The bar on joined cursive is already low-CER and free. WritePad (already bundled, GPL, no training, no added dependency) scored 21.6% held-out CER on joined cursive in the spike; the local VLM (#3), not measurable in this environment, is expected to be the accuracy leader. A model would have to beat both.
  • Cost/risk is high. The runtime bundle today has zero ML dependencies (rmapi-js only). tfjs-wasm or onnxruntime-web add multiple MB + a WASM runtime to an Obsidian plugin, plus an in-plugin training loop. Research for #2 already found no off-the-shelf package that is both browser-ONNX/tfjs and on-device-trainable — this is build-not-adopt.
  • The integrated writer-adaptive win is cheaper elsewhere. Directions 1 (!12) and 2 deliver the "integrated, trains on corrections, no server" property the issue author wanted, at near-zero bundle cost, for the separable regime — without a neural model.

Expected marginal value low, cost/risk high → stop at Directions 1–2, as the #5 scope boundary explicitly permits.

Revisit gate (dispatch only if ALL hold)

  1. #3's local VLM proves unacceptable for the owner (privacy, latency, or setup friction they won't accept), AND
  2. WritePad proves insufficient on the owner's real cursive hand (measure via #4's per-reader CER surface on real remarks — not the synthetic corpus), AND
  3. Directions 1–2 do not close the gap on the owner's real data.

If revisited, the first step is still a feasibility spike (can a tiny CTC-over-stroke-features model trained on sheets + a few hundred corrections beat 21.6% CER within a ≤~a-few-MB bundle budget), not implementation.

Part of #5 Part of #5 (Direction 3). **PARKED / no-go** per the #5 spike. Held (un-dispatched) as a tracked decision with an explicit revisit gate — do not start an agent on this unless the gate below is met. ## Why no-go (measured, from the #5 spike) Direction 3 is "a small stroke-sequence model trained *in the plugin* (tfjs-wasm / ONNX-web) that beats WritePad + the local VLM on the owner's corrections, within an Obsidian bundle/runtime budget." On the evidence: - **The bar on joined cursive is already low-CER and free.** WritePad (already bundled, GPL, no training, no added dependency) scored **21.6% held-out CER** on joined cursive in the spike; the local VLM (#3), not measurable in this environment, is expected to be the accuracy leader. A model would have to beat *both*. - **Cost/risk is high.** The runtime bundle today has **zero ML dependencies** (`rmapi-js` only). tfjs-wasm or onnxruntime-web add multiple MB + a WASM runtime to an Obsidian plugin, plus an in-plugin training loop. Research for #2 already found **no off-the-shelf package that is both browser-ONNX/tfjs and on-device-trainable** — this is build-not-adopt. - **The integrated writer-adaptive win is cheaper elsewhere.** Directions 1 (!12) and 2 deliver the "integrated, trains on corrections, no server" property the issue author wanted, at near-zero bundle cost, for the *separable* regime — without a neural model. Expected marginal value **low**, cost/risk **high** → stop at Directions 1–2, as the #5 scope boundary explicitly permits. ## Revisit gate (dispatch only if ALL hold) 1. #3's local VLM proves **unacceptable** for the owner (privacy, latency, or setup friction they won't accept), AND 2. WritePad proves **insufficient** on the owner's *real* cursive hand (measure via #4's per-reader CER surface on real remarks — not the synthetic corpus), AND 3. Directions 1–2 do not close the gap on the owner's real data. If revisited, the first step is still a **feasibility spike** (can a tiny CTC-over-stroke-features model trained on sheets + a few hundred corrections beat 21.6% CER within a ≤~a-few-MB bundle budget), not implementation.
Author
Collaborator

✅ Agent run completed

7s · 2 turns · $0.94 · session saved — new comments resume this run

✅ **Agent run completed** _7s · 2 turns · $0.94 · session saved — new comments resume this run_
Author
Collaborator

What I found

An agent was dispatched on this parked issue, but the revisit gate is not met. None of the three conditions holds today:

  1. The local VLM is unacceptable to the owner? No evidence of that. #3 is closed with !6 and !7 merged. The one-click "Set up local reading for me" button (!27, #8) merged on main today (7a021f8), and #28 (v2: installer download and LM Studio) is in progress. @daniel's only feedback on that path was "calling ollama is acceptable" (note on #8). Nobody has said the VLM is rejected for privacy, latency or setup friction.
  2. WritePad falls short on the owner's real cursive? This hasn't been measured. The 21.6% CER still comes from the synthetic corpus in the #5 spike. The real-data measurement needs the owner's ocr/samples.json and #4's per-reader surface (!9) on real remarks. Agents can't access that data, and no owner report has come in.
  3. Directions 1–2 fail to close the gap? This can't be judged yet. Direction 1 (!12) and the store work (!15, !16, !18, !19) have merged. The cheap competitor to the neural model on joined cursive is still unresolved: #21 (the spike that decides it) is open, and #20 is queued behind it.

Created

None. No code, MR or child issue: the gate is not met, so per this issue's own terms there is nothing to build or spike.

  • I added a link: #14 is blocked by #20. Condition 3 can't be judged until the Direction 2 synthesis bet resolves, and the link stops the operator from dispatching this issue again before then.

Follow-ups

  • When #20 closes, the next run should check the gate again. Conditions 1 and 2 still need owner input: either "the local VLM isn't acceptable to me", or real-world error rates from the import report's per-reader confidence.
  • @daniel, you can override this at any time. Comment here if you want the neural reader explored regardless. Even then, the first step stays a feasibility spike: a CTC model over stroke features, a few MB in size, that has to beat 21.6% cursive CER before any production code is written.
## What I found An agent was dispatched on this parked issue, but the revisit gate is **not met**. None of the three conditions holds today: 1. **The local VLM is unacceptable to the owner?** No evidence of that. #3 is closed with !6 and !7 merged. The one-click "Set up local reading for me" button (!27, #8) merged on `main` today (7a021f8), and #28 (v2: installer download and LM Studio) is in progress. @daniel's only feedback on that path was *"calling ollama is acceptable"* (note on #8). Nobody has said the VLM is rejected for privacy, latency or setup friction. 2. **WritePad falls short on the owner's real cursive?** This hasn't been measured. The 21.6% CER still comes from the **synthetic** corpus in the #5 spike. The real-data measurement needs the owner's `ocr/samples.json` and #4's per-reader surface (!9) on real remarks. Agents can't access that data, and no owner report has come in. 3. **Directions 1–2 fail to close the gap?** This can't be judged yet. Direction 1 (!12) and the store work (!15, !16, !18, !19) have merged. The cheap competitor to the neural model on joined cursive is still unresolved: **#21** (the spike that decides it) is open, and **#20** is queued behind it. ## Created None. No code, MR or child issue: the gate is not met, so per this issue's own terms there is nothing to build or spike. - I added a link: **#14 is blocked by #20**. Condition 3 can't be judged until the Direction 2 synthesis bet resolves, and the link stops the operator from dispatching this issue again before then. ## Follow-ups - When #20 closes, the next run should check the gate again. Conditions 1 and 2 still need **owner input**: either "the local VLM isn't acceptable to me", or real-world error rates from the import report's per-reader confidence. - @daniel, you can override this at any time. Comment here if you want the neural reader explored regardless. Even then, the first step stays a feasibility spike: a CTC model over stroke features, a few MB in size, that has to beat 21.6% cursive CER before any production code is written.
Author
Collaborator

Since my last update: #20 has closed. It came back no-go: owner-hand synthesis from letters and joins did not beat fonts-only on held-out words. The measurement is in #21. This releases the blocked-by link, but the revisit gate is still not met, so I'm not starting a spike or opening any MR or child issue. Gate status now:

  1. The local VLM is unacceptable to the owner? No change. It is still not met: v1 of the one-click Ollama setup is on main, and #28 (v2) is in progress.
  2. WritePad falls short on your real cursive? No change. It has still not been measured: the 21.6% CER comes from the synthetic corpus only.
  3. Directions 1–2 fail to close the gap? This now leans toward met on synthetic data. Direction 1 does nothing for joined cursive by design. Direction 2's join-synthesis bet failed. #20 also found that on joined hands that aren't literally a bundled font, WordReader's font pass is never trusted (correct-word cost 0.22–0.40 against FONT_TRUST = 0.08), so it reads only from your stored word samples. It remains unmeasured on your real data.

So the only open gate conditions (1 and 2) depend on your judgment or your data, which no agent can supply. Since there's no automated blocker left, I'm handing this to you so the issue stays parked instead of waking agents.

To unpark it, comment here with either of these, and re-assign the agent:

  • the local VLM path isn't acceptable to you (privacy, latency or setup friction); or
  • real per-reader CER from the import report showing WritePad falls short on your cursive.

Or simply tell me to explore the neural reader regardless. Even then, the first step stays a feasibility spike: a tiny CTC model over stroke features, a bundle of a few MB at most, that has to beat 21.6% CER.

Since my last update: #20 has closed. It came back **no-go**: owner-hand synthesis from letters and joins did not beat fonts-only on held-out words. The measurement is in #21. This releases the blocked-by link, but the revisit gate is **still not met**, so I'm not starting a spike or opening any MR or child issue. Gate status now: 1. **The local VLM is unacceptable to the owner?** No change. It is still not met: v1 of the one-click Ollama setup is on `main`, and #28 (v2) is in progress. 2. **WritePad falls short on your real cursive?** No change. It has still not been measured: the 21.6% CER comes from the synthetic corpus only. 3. **Directions 1–2 fail to close the gap?** This now leans toward **met on synthetic data**. Direction 1 does nothing for joined cursive by design. Direction 2's join-synthesis bet failed. #20 also found that on joined hands that aren't literally a bundled font, `WordReader`'s font pass is never trusted (correct-word cost 0.22–0.40 against `FONT_TRUST = 0.08`), so it reads only from your stored word samples. It remains **unmeasured on your real data**. So the only open gate conditions (1 and 2) depend on your judgment or your data, which no agent can supply. Since there's no automated blocker left, I'm handing this to you so the issue stays parked instead of waking agents. **To unpark it**, comment here with either of these, and re-assign the agent: - the local VLM path isn't acceptable to you (privacy, latency or setup friction); or - real per-reader CER from the import report showing WritePad falls short on your cursive. Or simply tell me to explore the neural reader regardless. Even then, the first step stays a feasibility spike: a tiny CTC model over stroke features, a bundle of a few MB at most, that has to beat 21.6% CER.
claude removed their assignment 2026-10-20 20:11:41 +02:00
Author
Collaborator

Agent completed ✅

There's nothing new on #14 since my handover, so I posted nothing and changed nothing.

The newest note on the issue is still my update after #20 closed. Nobody has replied, and none of the three conditions for restarting the work has changed:

  1. Nobody has said the local Ollama reader is unacceptable.
  2. WritePad hasn't been measured on your real cursive, so the 21.6% error rate still comes only from the synthetic test set.
  3. On synthetic data, the two cheaper approaches don't close the gap. On your real data they haven't been tested.

The issue is assigned to you, not the agent, so it should stay parked. To restart it, comment on #14 with one of the three unpark options from my last note and reassign the agent.

**Agent completed** :white_check_mark: There's nothing new on #14 since my handover, so I posted nothing and changed nothing. The newest note on the issue is still my update after #20 closed. Nobody has replied, and none of the three conditions for restarting the work has changed: 1. Nobody has said the local Ollama reader is unacceptable. 2. WritePad hasn't been measured on your real cursive, so the 21.6% error rate still comes only from the synthetic test set. 3. On synthetic data, the two cheaper approaches don't close the gap. On your real data they haven't been tested. The issue is assigned to you, not the agent, so it should stay parked. To restart it, comment on #14 with one of the three unpark options from my last note and reassign the agent.
Owner

@claude yeah i feel what the plugin currently suggests for my handwriting is mostly garbage. but to better work and train/verify that can we maybe add some training templates where i can copy text you prepared? it shall be stored in the notes vault ina format that retains the ink strokes, so i can easily attach it in the forge issue

@claude yeah i feel what the plugin currently suggests for my handwriting is mostly garbage. but to better work and train/verify that can we maybe add some training templates where i can copy text you prepared? it shall be stored in the notes vault ina format that retains the ink strokes, so i can easily attach it in the forge issue
Author
Collaborator

✅ Agent run completed

3m19s · 28 turns · $1.08 · pushed agent/handwriting-test-sheet

✅ **Agent run completed** _3m19s · 28 turns · $1.08 · pushed `agent/handwriting-test-sheet`_
Author
Collaborator

What I found

  • The plugin already has two sheets: the letter sheet and the joined-up word sheet. It sends each one as a notebook, reads your ink back by comparing it against the strokes it wrote itself, and sorts each stroke into the box it was written in (src/ink/handsheet.ts). A test sheet of prepared sentences fits that same mechanism as a third kind.
  • Right now there's nothing to measure your real hand against. The 21.6% error rate is still from synthetic font strokes. Your stored remark ink has no ground-truth text unless you corrected it. A test sheet supplies the ground truth, and this is exactly the measurement #14's revisit gate (condition 2) asks for.
  • Decision: a saved test sheet is held-out. The plugin reads it with today's readers but learns nothing from it. Otherwise it would contaminate the word samples, and every later check would look better than it really is. Training stays with the letter sheet, the word sheet and your corrections.

Created

  • !34, a hand-off to an implementation agent. It adds:
    • Send handwriting test sheet to reMarkable: about 18 prepared lines in German and English, plus pangrams, dates, times, prices, punctuation and jargon. Each line is printed, with a guided line below it for you to copy onto.
    • Save handwriting test sheet to the vault: writes <handwriting folder>/test-sheets/<stamp>/ containing:
      • handwriting-test-<stamp>.json: your full strokes per line (x, y, width, pressure), the prompt text, the page size, your learned letters, and what each reader (built-in, WritePad, and the vision endpoint if you use it) read, even when the reading was too unsure to accept. Attach this one file to the issue.
      • a note with the ink images and a table of prompt vs. each reader's reading, with error rates.
    • An offline replay, so an agent can run the attached file against today's code: HANDWRITING_CAPTURE=<file> npx vitest run src/__tests__/testcapture.test.ts.
    • No vault words, note text or remarks go into the file. It holds only the prepared lines, your ink of them and the readings.

Follow-ups

  • Once !34 is merged: send the sheet, copy the lines the way you normally write, let it sync, save it, and attach the .json here. The next run on #14 can then measure real error rates per reader and check gate condition 2 properly.
  • Do you use the local Ollama reader, or only built-in/WritePad? Your answer settles gate condition 1. If Ollama is set up when you save, the file records its reading too.
  • #14 stays parked until a real capture is in.
## What I found - The plugin already has two sheets: the letter sheet and the joined-up word sheet. It sends each one as a notebook, reads your ink back by comparing it against the strokes it wrote itself, and sorts each stroke into the box it was written in (`src/ink/handsheet.ts`). A **test sheet** of prepared sentences fits that same mechanism as a third kind. - Right now there's nothing to measure your real hand against. The 21.6% error rate is still from synthetic font strokes. Your stored remark ink has no ground-truth text unless you corrected it. A test sheet supplies the ground truth, and this is exactly the measurement #14's revisit gate (condition 2) asks for. - Decision: a saved test sheet is **held-out**. The plugin reads it with today's readers but **learns nothing from it**. Otherwise it would contaminate the word samples, and every later check would look better than it really is. Training stays with the letter sheet, the word sheet and your corrections. ## Created - **!34**, a hand-off to an implementation agent. It adds: - **Send handwriting test sheet to reMarkable**: about 18 prepared lines in German and English, plus pangrams, dates, times, prices, punctuation and jargon. Each line is printed, with a guided line below it for you to copy onto. - **Save handwriting test sheet to the vault**: writes `<handwriting folder>/test-sheets/<stamp>/` containing: - `handwriting-test-<stamp>.json`: your full strokes per line (x, y, width, pressure), the prompt text, the page size, your learned letters, and what each reader (built-in, WritePad, and the vision endpoint if you use it) read, even when the reading was too unsure to accept. **Attach this one file to the issue.** - a note with the ink images and a table of prompt vs. each reader's reading, with error rates. - An offline replay, so an agent can run the attached file against today's code: `HANDWRITING_CAPTURE=<file> npx vitest run src/__tests__/testcapture.test.ts`. - No vault words, note text or remarks go into the file. It holds only the prepared lines, your ink of them and the readings. ## Follow-ups - Once !34 is merged: send the sheet, copy the lines the way you normally write, let it sync, save it, and attach the `.json` here. The next run on #14 can then measure real error rates per reader and check gate condition 2 properly. - Do you use the local Ollama reader, or only built-in/WritePad? Your answer settles gate condition 1. If Ollama is set up when you save, the file records its reading too. - #14 stays parked until a real capture is in.
daniel reopened this issue 2026-10-21 00:14:09 +02:00
Owner

@claude i pushed the handwriting samples to the repo, any other closed issuesworth repoeningnow?

@claude i pushed the handwriting samples to the repo, any other closed issuesworth repoeningnow?
Author
Collaborator

✅ Agent run completed

4m44s · 26 turns · $1.47 · pushed agent/builtin-symbol-junk-gate

✅ **Agent run completed** _4m44s · 26 turns · $1.47 · pushed `agent/builtin-symbol-junk-gate`_
Author
Collaborator

What I found

  • Your real capture (handwriting-samples/test-sheets/2026-10-21-001026/, 18 lines of joined cursive) confirms "mostly garbage":
    • WritePad: 29.6% mean CER. That's worse than the 21.6% from the synthetic corpus. Even the 9 lines it trusted average about 24%, so roughly one character in four is wrong.
    • Built-in reader: 94.8% CER. Worse, it accepted 2 lines of pure symbol junk: p2-05 /IäI→✗rl…,%«%.Jl€ at 0.784 confidence and p2-06 @:@+m«↑@&↑3] at 0.788. Both cleared OCR_TRUST = 0.78, which #4 calibrated on synthetic fonts only. An accepted built-in reading also means WritePad is never asked.
  • #14 restart conditions:
    • Condition 2 (WritePad falls short on your real cursive) is now met.
    • Condition 3 (Directions 1–2 don't close the gap) is met.
    • Condition 1 (the local VLM is unacceptable to you) is still unknown. Your capture has no vision-model reading, because the reading mode wasn't set to the Ollama endpoint.
  • Closed issues worth reopening:
    • #4: yes. Its brief said to measure on your real remarks, and the leak it targeted is still there on real ink. I can't reopen issues from here, so the fix is handed off as !35. Reopen #4 if you want !35 to close it.
    • #20/#21 (letter-join synthesis) and the margin-based trust idea from #13/#17: not yet. The test sheet is deliberately held out (nothing learns from it), it holds no word-sheet ink, and 18 lines are too few to harvest letter pairs. It is the right evaluation set once there is training material, but it can't revive those bets on its own.
    • #22: it left a promised follow-up unfiled. Mermaid diagrams inside callouts are still sent as code. I filed that as #36 rather than reopening #22.
    • #1, #3, #26, #29: no reason to reopen.

Created

  • !35: hand-off. The built-in reader rejects readings that are more than 30% symbols, so the line falls through to WritePad. On your capture, all 18 prompts score 0.21 or less, and the two leaked junk lines score 0.59 and 0.83.
  • #36: Mermaid fences inside callouts/quotes (unassigned).

Follow-ups

  • To settle condition 1 for #14: run "Set up local reading for me", set the reading mode to the endpoint, then save a test sheet again and push it. The capture then records the vision model's error rate next to WritePad's.
    • If the vision model reads your hand well and you're fine running Ollama, the local-VLM path from #3 is the answer and #14 stays parked.
    • If it doesn't, or you won't run Ollama, say so here and #14 unparks into the feasibility spike: a small model trained on your strokes that has to beat 29.6% CER.
  • The 59 other PNGs in handwriting-samples/ (e.g. "But Rendering Bad / Remark it") have no text labels, so nothing can be measured from them. If you add a <same-name>.txt next to some of them with the intended text, they become a second test set of real remarks, which is useful for the vision-model comparison too.
## What I found - **Your real capture** (`handwriting-samples/test-sheets/2026-10-21-001026/`, 18 lines of joined cursive) confirms "mostly garbage": - **WritePad: 29.6% mean CER.** That's worse than the 21.6% from the synthetic corpus. Even the 9 lines it trusted average about 24%, so roughly one character in four is wrong. - **Built-in reader: 94.8% CER.** Worse, it **accepted 2 lines of pure symbol junk**: p2-05 `/IäI→✗rl…,%«%.Jl€` at 0.784 confidence and p2-06 `@:@+m«↑@&↑3]` at 0.788. Both cleared `OCR_TRUST = 0.78`, which #4 calibrated on synthetic fonts only. An accepted built-in reading also means WritePad is never asked. - **#14 restart conditions:** - **Condition 2 (WritePad falls short on your real cursive) is now met.** - **Condition 3 (Directions 1–2 don't close the gap) is met.** - **Condition 1 (the local VLM is unacceptable to you) is still unknown.** Your capture has no vision-model reading, because the reading mode wasn't set to the Ollama endpoint. - **Closed issues worth reopening:** - **#4: yes.** Its brief said to measure on your real remarks, and the leak it targeted is still there on real ink. I can't reopen issues from here, so the fix is handed off as !35. Reopen #4 if you want !35 to close it. - **#20/#21 (letter-join synthesis) and the margin-based trust idea from #13/#17: not yet.** The test sheet is deliberately held out (nothing learns from it), it holds no word-sheet ink, and 18 lines are too few to harvest letter pairs. It is the right evaluation set once there is training material, but it can't revive those bets on its own. - **#22:** it left a promised follow-up unfiled. Mermaid diagrams inside callouts are still sent as code. I filed that as #36 rather than reopening #22. - **#1, #3, #26, #29:** no reason to reopen. ## Created - **!35**: hand-off. The built-in reader rejects readings that are more than 30% symbols, so the line falls through to WritePad. On your capture, all 18 prompts score 0.21 or less, and the two leaked junk lines score 0.59 and 0.83. - **#36**: Mermaid fences inside callouts/quotes (unassigned). ## Follow-ups - **To settle condition 1 for #14:** run "Set up local reading for me", set the reading mode to the endpoint, then save a test sheet again and push it. The capture then records the vision model's error rate next to WritePad's. - If the vision model reads your hand well and you're fine running Ollama, the local-VLM path from #3 is the answer and #14 stays parked. - If it doesn't, or you won't run Ollama, say so here and #14 unparks into the feasibility spike: a small model trained on your strokes that has to beat 29.6% CER. - The 59 other PNGs in `handwriting-samples/` (e.g. "But Rendering Bad / Remark it") have no text labels, so nothing can be measured from them. If you add a `<same-name>.txt` next to some of them with the intended text, they become a second test set of real remarks, which is useful for the vision-model comparison too.
Owner

@claude tried the setup ollama for me, it downloaded, then i clicked test endpoint and it showed an error 400 from the api, something about invalid args or something

@claude tried the setup ollama for me, it downloaded, then i clicked test endpoint and it showed an error 400 from the api, something about invalid args or something
Author
Collaborator

❌ Agent run failed

6m35s

❌ **Agent run failed** _6m35s_
Author
Collaborator

Agent failed ❌

agent job failed: container agent OOMKilled; container agent OOMKilled

**Agent failed** :x: agent job failed: container agent OOMKilled; container agent OOMKilled
Owner

@claude updated the samples in the repo with ollama results

@claude updated the samples in the repo with ollama results
Author
Collaborator

✅ Agent run completed

7m45s · 65 turns · $1.97 · pushed agent/fix-ocr-probe-png, agent/capture-endpoint-model-latency

✅ **Agent run completed** _7m45s · 65 turns · $1.97 · pushed `agent/fix-ocr-probe-png`, `agent/capture-endpoint-model-latency`_
Author
Collaborator

What I found

  • Ollama reads your hand about 3× better than WritePad. Same 18 lines; the ink in 2026-10-21-004942 is byte-identical to the first capture:

    Reader Mean CER Lines read perfectly
    Ollama (endpoint) 10.5% 7 of 18
    WritePad 30.7% none
    Built-in 94.8% none

    Its weak spots:

    • Fluent wrong words, accepted with no confidence signal. p1-07 "aber nicht dringend" came back as "dann nicht belanglos", which changes the meaning. Others: "ob der Termin passt" → "ob den Termin plant", Jörg → Jürgen, Taxi → Tal.
    • Proper nouns and symbols. "Oberidian … llamad, lgit Sale"; ✓ became √; "12,50 €" became "€12.50".
    • Formatting it adds itself. Markdown emphasis (*quick*) and quotes you didn't write (p2-06).
  • The 2 junk lines the built-in reader "used" (p2-05, p2-06) come from your installed build, which predates !35. On current main both are rejected (symbol share 0.59 and 0.83, limit 0.30). Rebuild and the leak is gone.

  • Your "Test connection → 400" is a real bug, now fixed. The tiny probe image the button sends was a corrupt PNG: one wrong base64 character, so the image data failed its CRC check, and Ollama rejected it. Imports send real images, which is why reading works.

  • #14 gate: conditions 2 and 3 are met. Condition 1 is not: the local VLM is clearly the accuracy leader. An in-plugin model trained on a few hundred samples would have to beat 10.5% CER (not 21.6% or 29.6%), which is very unlikely. #14 stays parked.

Created

  • !38: trivial fix, a valid 64×64 probe PNG plus a regression test that checks it parses. The test failed before the fix. Full suite: 839 pass. Not tried against a live Ollama.
  • !39: hand-off. Test-sheet captures will record the endpoint's model name, per-line read time for each reader, and why a line failed, instead of swallowing the error. You'll be able to compare models and see latency, which is the other half of gate condition 1.

Follow-ups

  • @daniel: is Ollama's speed acceptable to you while importing? If yes, #14 can be closed as won't-do. Ollama is the answer, and the remaining work is making its output safer, not training a model.
  • Possible next steps on the Ollama path, once !39 lands:
    • Tell the model to answer in plain text, without Markdown or added quotes.
    • Let the offline replay call a local endpoint, so prompts and models can be compared on your captured ink without writing a new sheet.
    • Feed your letter/word-sheet ink to the model as worked examples. Today's test reads deliberately send none; corrections from real remarks already go along at import.
## What I found - **Ollama reads your hand about 3× better than WritePad.** Same 18 lines; the ink in `2026-10-21-004942` is byte-identical to the first capture: | Reader | Mean CER | Lines read perfectly | | --- | --- | --- | | **Ollama (endpoint)** | **10.5%** | 7 of 18 | | WritePad | 30.7% | none | | Built-in | 94.8% | none | Its weak spots: - **Fluent wrong words, accepted with no confidence signal.** p1-07 *"aber nicht dringend"* came back as *"dann nicht belanglos"*, which changes the meaning. Others: *"ob der Termin passt"* → *"ob den Termin plant"*, Jörg → Jürgen, Taxi → Tal. - **Proper nouns and symbols.** *"Oberidian … llamad, lgit Sale"*; ✓ became √; *"12,50 €"* became *"€12.50"*. - **Formatting it adds itself.** Markdown emphasis (`*quick*`) and quotes you didn't write (p2-06). - **The 2 junk lines the built-in reader "used" (p2-05, p2-06) come from your installed build, which predates !35.** On current `main` both are rejected (symbol share 0.59 and 0.83, limit 0.30). Rebuild and the leak is gone. - **Your "Test connection → 400" is a real bug, now fixed.** The tiny probe image the button sends was a corrupt PNG: one wrong base64 character, so the image data failed its CRC check, and Ollama rejected it. Imports send real images, which is why reading works. - **#14 gate:** conditions 2 and 3 are met. **Condition 1 is not: the local VLM is clearly the accuracy leader.** An in-plugin model trained on a few hundred samples would have to beat **10.5%** CER (not 21.6% or 29.6%), which is very unlikely. **#14 stays parked.** ## Created - **!38**: trivial fix, a valid 64×64 probe PNG plus a regression test that checks it parses. The test failed before the fix. Full suite: 839 pass. Not tried against a live Ollama. - **!39**: hand-off. Test-sheet captures will record the endpoint's **model name**, **per-line read time** for each reader, and **why** a line failed, instead of swallowing the error. You'll be able to compare models and see latency, which is the other half of gate condition 1. ## Follow-ups - **@daniel:** is Ollama's speed acceptable to you while importing? If yes, #14 can be closed as won't-do. Ollama is the answer, and the remaining work is making its output safer, not training a model. - Possible next steps on the Ollama path, once !39 lands: - Tell the model to answer in plain text, without Markdown or added quotes. - Let the offline replay call a local endpoint, so prompts and models can be compared on your captured ink without writing a new sheet. - Feed your letter/word-sheet ink to the model as worked examples. Today's test reads deliberately send none; corrections from real remarks already go along at import.
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
daniel/obsidian-remarkable-round-trip#14
No description provided.