feat(ocr): handwriting test sheet — copy prepared text on the tablet, save the ink to the vault for evaluation #34

Merged
daniel merged 5 commits from agent/handwriting-test-sheet into main 2026-10-20 23:53:51 +02:00
Collaborator

Related to #14. Do not close #14: it stays parked. This MR builds the measurement tool its revisit gate condition 2 asks for, "per-reader CER on the owner's real cursive".

Why

On #14 the owner says the plugin's readings of their handwriting are "mostly garbage". They asked for training/verification templates: prepared text they copy by hand on the reMarkable, with the result stored in the vault in a format that keeps the ink strokes, so they can attach it to a forge issue. Today an agent can't measure anything on the real hand. The only CER we have (21.6%, WritePad) comes from synthetic bundled-font strokes (src/__tests__/writepad.test.ts). The owner's real ink only exists in reMarkable/.sync/samples.json and per-remark ink/*.json, and neither has ground-truth text.

The repo already has nearly all the pieces. This MR adds a third sheet kind alongside the letter sheet and the word sheet.

What to build

1. A test sheet layout: src/ink/handsheet.ts

  • Add SheetKind = "letters" | "words" | "test". In buildSheet, add a test branch:
    • one column, full content width;
    • per box: the prompt printed in black above the guides, the same way the word sheet prints its word above the guides. Use the existing write and wrap helpers, and wrap long prompts;
    • below the prompt, one writing line with the same v2 guides (cap dashed, x-height dotted, baseline solid, descender dotted);
    • a box height that fits a sentence in normal writing size, e.g. ≈ 230 * unit. The implementer picks the value; Paper Pro Move must still get ≥ 2 rows per page (rowsPerPage already guarantees ≥ 2).
    • Header title Handwriting test ${p}/${n}. Instructions: "Copy each line once, below it, the way you normally write — joined-up if that is how you write. Don't correct the text to make it neater. Skip any you like."
  • Add export const TEST_SHEET_PROMPTS: Record<number, string[]> with TEST_SHEET_VERSION = 1. Keep the old versions buildable, under the same rule as SHEET_VERSION: a sheet waiting on the tablet is read back with the layout it was sent with. Starting set for v1. The implementer may tweak it, but should keep the mix of DE/EN, digits, punctuation and jargon, and keep it ≤ ~3 pages on rm2:
    Franz jagt im komplett verwahrlosten Taxi quer durch Bayern
    The quick brown fox jumps over the lazy dog
    Zwölf Boxkämpfer jagen Viktor quer über den großen Sylter Deich
    Pack my box with five dozen liquor jugs
    Morgen noch prüfen, ob der Termin passt
    Check this again tomorrow before the meeting
    Das ist wichtig, aber nicht dringend
    Maybe we should think about it next week
    Bitte bis Freitag erledigen
    Ask about the budget and the timeline
    Termin am 14.03. um 9:30 Uhr
    Call back at 4 pm, room 217
    Größe: 42 cm × 18 cm, Preis 12,50 €
    Frage: ja oder nein? (siehe #14)
    later → done ✓, not now ✗
    Obsidian, reMarkable, WritePad, Ollama, GitLab
    Besprechung mit Anna und Jörg im Büro
    e-mail: [email protected]
    
  • Refactor the "which box does each user stroke fall in" loop that learnWords and learnHand both inline. Turn it into an exported boxInk(pending, pages, kind): Map<SheetBox, Stroke[]>, where Stroke is incoming/rmlines.ts, so the full stroke is kept: tool, color, thicknessScale, and per-point x/y/width/pressure. learnWords keeps its current output by mapping down to {x,y}. learnHand may use the helper, or be left alone if the refactor gets awkward. This is the only behaviour-preserving refactor allowed; the existing ink.test.ts and samples.test.ts sheet tests must stay green.
  • PendingSheet.kind becomes "words" | "test" | undefined.

2. The capture format: new pure module src/ocr/testcapture.ts

This module has no Obsidian imports, so it can be unit-tested and replayed headless.

export const CAPTURE_FORMAT = "remarkable-ink-handwriting-test";
export interface HandwritingCapture {
  format: typeof CAPTURE_FORMAT; version: 1;
  createdAt: string; pluginVersion: string;
  sheet: { kind: "test"; version: number };   // TEST_SHEET_VERSION it was sent with
  profile: DeviceProfile;                     // canvas, so boxes can be rebuilt
  hand: HandLibrary | null;                   // learned letters at capture time → built-in reader is replayable
  items: CaptureItem[];
}
export interface CaptureItem {
  id: string;            // "p1-03" (page-row)
  prompt: string;        // ground truth
  box: SheetBox;
  strokes: { tool: number; color: number; thicknessScale: number; points: number[] }[]; // flat x,y,width,pressure — device units
  readings: Partial<Record<"builtin" | "writepad" | "endpoint", { text: string; confidence?: number; accepted: boolean } | null>>;
}
export function cer(got: string, want: string): number   // move the copy from writepad.test.ts here and import it there
export function summarize(capture): { reader; items; meanCer; accepted; }[]
export function captureMarkdown(capture, imageNames: Map<string,string>): string
  • Store points flat (points: number[], 4 numbers per point), like InkStrokes.points in ink/inkimage.ts stores its flat per-point values. A full sheet then stays at a few hundred KB, which is attachable.
  • Privacy constraint: do not include the vault lexicon, samples.json word samples, note paths or remark text. The file holds only the prepared prompts, the owner's ink of those prompts, the hand library (their letter shapes, which are already shared in .sync) and readings. Note this in the docs.

3. Commands + wiring: src/inkedge.ts, src/main.ts, settings

Mirror the word sheet everywhere it appears. grep -n pendingWordSheet src lists every touch point:

  • settingsmodel.ts:87/159: add pendingTestSheet: PendingSheet | null (default null).
  • sync/syncstate.ts:59-63 SharedHandwriting and main.ts handwritingOf (≈2446) / adoptHandwriting (≈538-550): share it across vault copies. Read it tolerantly: a handwriting.json written before this change has no field, so map undefined to null.
  • main.ts:809: add it to known, so the sheet is never imported as a new notebook.
  • inkedge.ts:405 detectPageSize: add it to ours.
  • inkedge.ts: add sendTestSheet(api, parentId), a copy of sendWordSheet with title "Handwriting test".
  • main.ts commands, next to the word-sheet ones at ≈308-315:
    • send-handwriting-test-sheet: Send handwriting test sheet to reMarkable, a copy of sendWordSheet() (≈2100);
    • save-handwriting-test-sheet: Save handwriting test sheet to the vault. This command:
      1. reads the pending notebook (readNotebook) and calls boxInk. If no box has ink, show a notice like the word sheet's: "Nothing written on the test sheet yet…";
      2. reads each box with the current readers, without learning. Call recognize(strokes, await this.ocrTemplates(), lexicon, await this.wordReader()) and, if ocrUseWritePad, (await this.writePad())?.read(strokes, lexicon) directly. Do NOT call recognizeLocally / rememberReading / addSample. The sheet must stay held-out, or it contaminates the sample store and every later evaluation. Record accepted against OCR_TRUST / WRITEPAD_TRUST, as recognizeLocally (≈1737) decides it. Also record the raw text when it was not accepted: that is the whole point. If ocrMode === "endpoint", also call this.ink.ocr(http, png, undefined, false, []) with no worked examples and record that reading. An endpoint failure records null and must not abort the save;
      3. renders one PNG per box with strokesToPng (≈1715);
      4. writes, under <handwritingFolder>/test-sheets/<YYYY-MM-DD-HHmmss>/:
        • handwriting-test-<stamp>.json: the HandwritingCapture. This is the one file to attach to an issue;
        • p1-03.png …;
        • Handwriting test <stamp>.md: a table of prompt | built-in | WritePad | (endpoint) with per-row CER, the mean CER per reader, the embedded PNGs, and a line saying "Attach handwriting-test-<stamp>.json to the forge issue.";
      5. opens the note and keeps the sheet pending, so it can be saved again after more writing. Each save makes a new stamped folder.
  • settings.ts ≈165-180: next to the word sheet, add a "Handwriting test" row with Send and Save buttons and a description, and "A test sheet is waiting on the tablet." when one is pending.

4. Offline replay: what an agent runs on an attached file

src/__tests__/testcapture.test.ts:

  • unit-test cer, summarize and captureMarkdown on a small synthetic capture. Build the capture from buildSheet(DEVICE_PROFILES.rm2, "test") boxes, with prompt strokes written in the allure font via the write() helper pattern from writepad.test.ts;
  • round-trip: build a fake filled-in test sheet .rm. Take the sheet's pages plus user strokes inside boxes, the same way the existing sheet tests in ink.test.ts / samples.test.ts fake a filled sheet. Then check that boxInk puts each box's strokes under the right prompt and that the JSON survives JSON.parse(JSON.stringify());
  • a replay block gated on process.env.HANDWRITING_CAPTURE (describe.skipIf(!process.env.HANDWRITING_CAPTURE)). It loads that file, re-reads every item with today's recognize (templates from capture.hand if present, else readability, as ocrTemplates does) and WritePad, without a lexicon, and prints a per-reader table of captured CER vs replayed CER plus the mean. It asserts only that the file parses and format/version match. It must not assert accuracy.

5. Docs

  • README "Your own handwriting" / "Reading handwriting": add a short "Test how well your handwriting is read" subsection covering send, write, sync, save, and attach the .json.
  • docs/ocr-corrections.md: document the capture format, the "held-out, never learned from" rule, and the replay command.
  • Add a CHANGELOG-INK.md entry.

Constraints and decisions

  • Held-out by design. Saving a test sheet teaches the readers nothing. Training stays with the existing letter sheet, word sheet and corrections. "Learn from this test sheet" is out of scope. If it's ever added, it must say that a sheet you trained on can no longer verify.
  • One capture JSON file rather than raw .rm pages. The .rm also contains the sheet's own strokes, needs the diff to separate them, and isn't human-viewable. The JSON carries the full stroke data (x, y, width, pressure) and the box geometry, which is everything a reader consumes.
  • Readings are recorded as the plugin would read today, lexicon included (the "what the owner experiences" number). Replay runs without the lexicon, because the vault isn't in the file. Both numbers are reported, labelled.
  • Interfaces that must not change: learnWords output shape, PendingSheet for existing kinds, the SHEET_VERSION letter and word layouts (a pending sheet on someone's tablet must still read back).

Out of scope

  • Any change to how readings are made (thresholds, WritePad, word reader). This MR only measures.
  • Revisiting #14's gate. That happens on #14 once the owner attaches a capture.

Validation contract

  • npm ci && npx vitest run src/__tests__/testcapture.test.ts src/__tests__/ink.test.ts src/__tests__/samples.test.ts src/__tests__/writepad.test.ts → all PASS. The repo has no flake.nix, so use plain npx, not nix develop.
  • npx vitest run (full suite) → PASS; npm run build (includes scripts/check-bundle.mjs) → exit 0.
  • Red gate: before the change, buildSheet(profile, "test"), boxInk and src/ocr/testcapture.ts don't exist, so the new test file fails to compile and import. After it, the round-trip test must find every prompt's strokes under its own prompt. Assert the per-box stroke counts equal what the fixture wrote, not just "> 0", so the test is not vacuous.
  • Replay positive control: write the synthetic capture from the round-trip test to a temp file, then run HANDWRITING_CAPTURE=<tmp> npx vitest run src/__tests__/testcapture.test.ts. It must print a non-empty table with one row per reader. Paste that output into the MR.

🤖 Generated with Claude Code

Related to #14. Do **not** close #14: it stays parked. This MR builds the measurement tool its revisit gate condition 2 asks for, "per-reader CER on the owner's *real* cursive". ## Why On #14 the owner says the plugin's readings of their handwriting are "mostly garbage". They asked for **training/verification templates**: prepared text they copy by hand on the reMarkable, with the result stored **in the vault in a format that keeps the ink strokes**, so they can attach it to a forge issue. Today an agent can't measure anything on the real hand. The only CER we have (21.6%, WritePad) comes from synthetic bundled-font strokes (`src/__tests__/writepad.test.ts`). The owner's real ink only exists in `reMarkable/.sync/samples.json` and per-remark `ink/*.json`, and neither has ground-truth text. The repo already has nearly all the pieces. This MR adds a third sheet kind alongside the letter sheet and the word sheet. ## What to build ### 1. A test sheet layout: `src/ink/handsheet.ts` - Add `SheetKind = "letters" | "words" | "test"`. In `buildSheet`, add a `test` branch: - **one column**, full content width; - per box: the **prompt printed in black** above the guides, the same way the word sheet prints its word above the guides. Use the existing `write` and `wrap` helpers, and wrap long prompts; - below the prompt, **one writing line** with the same v2 guides (cap dashed, x-height dotted, baseline solid, descender dotted); - a box height that fits a sentence in normal writing size, e.g. `≈ 230 * unit`. The implementer picks the value; Paper Pro Move must still get ≥ 2 rows per page (`rowsPerPage` already guarantees ≥ 2). - Header title `Handwriting test ${p}/${n}`. Instructions: *"Copy each line once, below it, the way you normally write — joined-up if that is how you write. Don't correct the text to make it neater. Skip any you like."* - Add `export const TEST_SHEET_PROMPTS: Record<number, string[]>` with `TEST_SHEET_VERSION = 1`. Keep the old versions buildable, under the same rule as `SHEET_VERSION`: a sheet waiting on the tablet is read back with the layout it was sent with. Starting set for v1. The implementer may tweak it, but should keep the mix of DE/EN, digits, punctuation and jargon, and keep it ≤ ~3 pages on rm2: ``` Franz jagt im komplett verwahrlosten Taxi quer durch Bayern The quick brown fox jumps over the lazy dog Zwölf Boxkämpfer jagen Viktor quer über den großen Sylter Deich Pack my box with five dozen liquor jugs Morgen noch prüfen, ob der Termin passt Check this again tomorrow before the meeting Das ist wichtig, aber nicht dringend Maybe we should think about it next week Bitte bis Freitag erledigen Ask about the budget and the timeline Termin am 14.03. um 9:30 Uhr Call back at 4 pm, room 217 Größe: 42 cm × 18 cm, Preis 12,50 € Frage: ja oder nein? (siehe #14) later → done ✓, not now ✗ Obsidian, reMarkable, WritePad, Ollama, GitLab Besprechung mit Anna und Jörg im Büro e-mail: [email protected] ``` - Refactor the "which box does each user stroke fall in" loop that `learnWords` and `learnHand` both inline. Turn it into an exported `boxInk(pending, pages, kind): Map<SheetBox, Stroke[]>`, where `Stroke` is `incoming/rmlines.ts`, so the **full** stroke is kept: tool, color, thicknessScale, and per-point x/y/width/pressure. `learnWords` keeps its current output by mapping down to `{x,y}`. `learnHand` may use the helper, or be left alone if the refactor gets awkward. This is the only behaviour-preserving refactor allowed; the existing `ink.test.ts` and `samples.test.ts` sheet tests must stay green. - `PendingSheet.kind` becomes `"words" | "test" | undefined`. ### 2. The capture format: new pure module `src/ocr/testcapture.ts` This module has no Obsidian imports, so it can be unit-tested and replayed headless. ```ts export const CAPTURE_FORMAT = "remarkable-ink-handwriting-test"; export interface HandwritingCapture { format: typeof CAPTURE_FORMAT; version: 1; createdAt: string; pluginVersion: string; sheet: { kind: "test"; version: number }; // TEST_SHEET_VERSION it was sent with profile: DeviceProfile; // canvas, so boxes can be rebuilt hand: HandLibrary | null; // learned letters at capture time → built-in reader is replayable items: CaptureItem[]; } export interface CaptureItem { id: string; // "p1-03" (page-row) prompt: string; // ground truth box: SheetBox; strokes: { tool: number; color: number; thicknessScale: number; points: number[] }[]; // flat x,y,width,pressure — device units readings: Partial<Record<"builtin" | "writepad" | "endpoint", { text: string; confidence?: number; accepted: boolean } | null>>; } export function cer(got: string, want: string): number // move the copy from writepad.test.ts here and import it there export function summarize(capture): { reader; items; meanCer; accepted; }[] export function captureMarkdown(capture, imageNames: Map<string,string>): string ``` - Store points flat (`points: number[]`, 4 numbers per point), like `InkStrokes.points` in `ink/inkimage.ts` stores its flat per-point values. A full sheet then stays at a few hundred KB, which is attachable. - **Privacy constraint:** do **not** include the vault lexicon, `samples.json` word samples, note paths or remark text. The file holds only the prepared prompts, the owner's ink of those prompts, the hand library (their letter shapes, which are already shared in `.sync`) and readings. Note this in the docs. ### 3. Commands + wiring: `src/inkedge.ts`, `src/main.ts`, settings Mirror the word sheet everywhere it appears. `grep -n pendingWordSheet src` lists every touch point: - `settingsmodel.ts:87/159`: add `pendingTestSheet: PendingSheet | null` (default `null`). - `sync/syncstate.ts:59-63` `SharedHandwriting` and `main.ts` `handwritingOf` (≈2446) / `adoptHandwriting` (≈538-550): share it across vault copies. **Read it tolerantly**: a `handwriting.json` written before this change has no field, so map `undefined` to `null`. - `main.ts:809`: add it to `known`, so the sheet is never imported as a new notebook. - `inkedge.ts:405` `detectPageSize`: add it to `ours`. - `inkedge.ts`: add `sendTestSheet(api, parentId)`, a copy of `sendWordSheet` with title `"Handwriting test"`. - `main.ts` commands, next to the word-sheet ones at ≈308-315: - `send-handwriting-test-sheet`: **Send handwriting test sheet to reMarkable**, a copy of `sendWordSheet()` (≈2100); - `save-handwriting-test-sheet`: **Save handwriting test sheet to the vault**. This command: 1. reads the pending notebook (`readNotebook`) and calls `boxInk`. If no box has ink, show a notice like the word sheet's: "Nothing written on the test sheet yet…"; 2. reads each box with the **current readers, without learning**. Call `recognize(strokes, await this.ocrTemplates(), lexicon, await this.wordReader())` and, if `ocrUseWritePad`, `(await this.writePad())?.read(strokes, lexicon)` directly. Do **NOT** call `recognizeLocally` / `rememberReading` / `addSample`. The sheet must stay held-out, or it contaminates the sample store and every later evaluation. Record `accepted` against `OCR_TRUST` / `WRITEPAD_TRUST`, as `recognizeLocally` (≈1737) decides it. Also record the raw text when it was not accepted: that is the whole point. If `ocrMode === "endpoint"`, also call `this.ink.ocr(http, png, undefined, false, [])` with **no worked examples** and record that reading. An endpoint failure records `null` and must not abort the save; 3. renders one PNG per box with `strokesToPng` (≈1715); 4. writes, under `<handwritingFolder>/test-sheets/<YYYY-MM-DD-HHmmss>/`: - `handwriting-test-<stamp>.json`: the `HandwritingCapture`. **This is the one file to attach to an issue;** - `p1-03.png` …; - `Handwriting test <stamp>.md`: a table of prompt | built-in | WritePad | (endpoint) with per-row CER, the mean CER per reader, the embedded PNGs, and a line saying *"Attach `handwriting-test-<stamp>.json` to the forge issue."*; 5. opens the note and keeps the sheet pending, so it can be saved again after more writing. Each save makes a new stamped folder. - `settings.ts` ≈165-180: next to the word sheet, add a "Handwriting test" row with **Send** and **Save** buttons and a description, and "A test sheet is waiting on the tablet." when one is pending. ### 4. Offline replay: what an agent runs on an attached file `src/__tests__/testcapture.test.ts`: - unit-test `cer`, `summarize` and `captureMarkdown` on a small synthetic capture. Build the capture from `buildSheet(DEVICE_PROFILES.rm2, "test")` boxes, with prompt strokes written in the `allure` font via the `write()` helper pattern from `writepad.test.ts`; - **round-trip**: build a fake filled-in test sheet `.rm`. Take the sheet's pages plus user strokes inside boxes, the same way the existing sheet tests in `ink.test.ts` / `samples.test.ts` fake a filled sheet. Then check that `boxInk` puts each box's strokes under the right prompt and that the JSON survives `JSON.parse(JSON.stringify())`; - a **replay** block gated on `process.env.HANDWRITING_CAPTURE` (`describe.skipIf(!process.env.HANDWRITING_CAPTURE)`). It loads that file, re-reads every item with today's `recognize` (templates from `capture.hand` if present, else readability, as `ocrTemplates` does) and `WritePad`, **without a lexicon**, and prints a per-reader table of captured CER vs replayed CER plus the mean. It asserts only that the file parses and `format`/`version` match. It must not assert accuracy. ### 5. Docs - README "Your own handwriting" / "Reading handwriting": add a short "Test how well your handwriting is read" subsection covering send, write, sync, save, and attach the `.json`. - `docs/ocr-corrections.md`: document the capture format, the "held-out, never learned from" rule, and the replay command. - Add a `CHANGELOG-INK.md` entry. ## Constraints and decisions - **Held-out by design.** Saving a test sheet teaches the readers nothing. Training stays with the existing letter sheet, word sheet and corrections. "Learn from this test sheet" is out of scope. If it's ever added, it must say that a sheet you trained on can no longer verify. - **One capture JSON file** rather than raw `.rm` pages. The `.rm` also contains the sheet's own strokes, needs the diff to separate them, and isn't human-viewable. The JSON carries the full stroke data (x, y, width, pressure) and the box geometry, which is everything a reader consumes. - Readings are recorded **as the plugin would read today, lexicon included** (the "what the owner experiences" number). Replay runs **without the lexicon**, because the vault isn't in the file. Both numbers are reported, labelled. - Interfaces that must not change: `learnWords` output shape, `PendingSheet` for existing kinds, the `SHEET_VERSION` letter and word layouts (a pending sheet on someone's tablet must still read back). ## Out of scope - Any change to how readings are made (thresholds, WritePad, word reader). This MR only measures. - Revisiting #14's gate. That happens on #14 once the owner attaches a capture. ## Validation contract - `npm ci && npx vitest run src/__tests__/testcapture.test.ts src/__tests__/ink.test.ts src/__tests__/samples.test.ts src/__tests__/writepad.test.ts` → all PASS. The repo has no flake.nix, so use plain `npx`, not `nix develop`. - `npx vitest run` (full suite) → PASS; `npm run build` (includes `scripts/check-bundle.mjs`) → exit 0. - **Red gate**: before the change, `buildSheet(profile, "test")`, `boxInk` and `src/ocr/testcapture.ts` don't exist, so the new test file fails to compile and import. After it, the round-trip test must find **every** prompt's strokes under its own prompt. Assert the per-box stroke counts equal what the fixture wrote, not just "> 0", so the test is not vacuous. - **Replay positive control**: write the synthetic capture from the round-trip test to a temp file, then run `HANDWRITING_CAPTURE=<tmp> npx vitest run src/__tests__/testcapture.test.ts`. It must print a non-empty table with one row per reader. Paste that output into the MR. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
claude self-assigned this 2026-10-20 23:27:16 +02:00
Author
Collaborator

✅ Agent run completed

15m56s · 76 turns · $3.47 · pushed agent/handwriting-test-sheet · session saved — new comments resume this run

⚠️ Verification gate not satisfied: awaiting intent review. Outcome parked — not marked done.

✅ **Agent run completed** _15m56s · 76 turns · $3.47 · pushed `agent/handwriting-test-sheet` · session saved — new comments resume this run_ ⚠️ _Verification gate not satisfied: awaiting intent review. Outcome parked — not marked done._
Author
Collaborator

What changed

  • Test sheet layout (src/ink/handsheet.ts): new SheetKind "test" with TEST_SHEET_PROMPTS / TEST_SHEET_VERSION = 1, using the 18 prompts as given. Each row prints the prompt in black (wrapped when it's long) above one writing line with the v2 guides. Boxes are 230 units high: 3 pages on rm2 and Paper Pro, 4 on Move, 5 rows per page on Move. New exported boxInk(pending, pages, kind) keeps the full Stroke. learnWords and learnHand both use it now and keep the same output and key order. The existing sheet tests pass unchanged.
  • Capture format (src/ocr/testcapture.ts, no Obsidian imports): the HandwritingCapture / CaptureItem types with flat x,y,width,pressure points, plus cer (moved here; writepad.test.ts imports it), summarize, captureMarkdown, captureItems, replayTemplates and parseCapture.
  • Wiring: pendingTestSheet added to settings, SharedHandwriting (an old handwriting.json without the field reads as null, with a test), adoptHandwriting, handwritingOf, known and detectPageSize. Also sendTestSheet / readTestSheet in inkedge.ts, two commands, and a settings row with Send / Save buttons. Save never calls recognizeLocally, rememberReading or addSample, and the endpoint gets [] worked examples. Docs added to the README, docs/ocr-corrections.md and CHANGELOG-INK.md.

Decisions you may want to redirect:

  1. WritePad's known-word gate makes read() return null. When that happens, the save stores the reading made without the lexicon, marked accepted: false. Otherwise the text that was rejected would be lost.
  2. Per-item CER collapses whitespace (itemCer). A sentence that runs onto a second row comes back with a \n, which shouldn't count as a misreading. Long prompts like the 63-character "Zwölf Boxkämpfer…" may not fit one line at normal writing size, so the box leaves room below the line. cer itself is unchanged.
  3. Built-in and WritePad are both recorded for every line. The import only asks WritePad when the built-in reader isn't trusted; here each reader gets its own number.
  4. summarize has an extra failed count for endpoint null readings, which are left out of meanCer.

Verify

  • Run locally; no pipeline has run for this branch yet:
    • The four contract test files pass: 76 tests, plus the replay test, which is skipped without a capture file.
    • The full npx vitest run passes: 775 tests, 1 skipped.
    • npm run build exits 0, including check-bundle, and eslint src/ is clean.
  • Red gate: on origin/main, the new test file fails with Cannot find module '../ocr/testcapture'. The round-trip test checks that every prompt's stroke count equals what the fixture wrote. As a control, I narrowed the box hit-test in boxInk, and it failed (expected 25 to be 26).
  • Replay positive control: I wrote the synthetic capture with HANDWRITING_CAPTURE_OUT=/tmp/hw-capture.json npx vitest run src/__tests__/testcapture.test.ts (488 KB for 17 lines), then ran HANDWRITING_CAPTURE=/tmp/hw-capture.json npx vitest run src/__tests__/testcapture.test.ts. The output, cut after the first 6 of 34 item rows:
Handwriting capture 2026-10-20T21:30:00.000Z (plugin test, test sheet v1, 17 items, hand none)
Captured: as the plugin read it, with the vault's words. Replayed: today's readers, no vault words.

reader    items  captured  replayed
Built-in     17   81.8%   81.8%
WritePad     17   34.1%   34.1%

item    reader    captured  replayed  replayed text
p1-01  Built-in   84.7%   84.7%  "@m@ m @% m%%mw @ ym @% @mw"
p1-01  WritePad   35.6%   35.6%  "Fronsrjogt I'm konqelett iverwohrlgorten Seffriquer darch Boyern"
p1-02  Built-in   88.4%   88.4%  "@ m@%m@mzm % @ @"
p1-02  WritePad   37.2%   37.2%  "(tic quick brown patr junqlf aver the lorry dog"
p1-03  Built-in   87.3%   87.3%  "@%@@mew @m ym @m @wm@w @@ @@"
p1-03  WritePad   42.9%   42.9%  "Frwod28gfrksonqiSerjTgen Xiktor quee iiloer den grobcn Sylter soeich"

Captured and replayed match because the synthetic file was read without a lexicon too. These numbers come from bundled-font strokes, not a measurement of real handwriting.

  • Not exercised: the Obsidian commands themselves (PNG rendering, vault writes, opening the note). There's no Obsidian runtime here; they type-check and build.

Follow-ups

  • None. #14 stays parked until the owner attaches a capture.
## What changed - **Test sheet layout** (`src/ink/handsheet.ts`): new `SheetKind` `"test"` with `TEST_SHEET_PROMPTS` / `TEST_SHEET_VERSION = 1`, using the 18 prompts as given. Each row prints the prompt in black (wrapped when it's long) above one writing line with the v2 guides. Boxes are 230 units high: 3 pages on rm2 and Paper Pro, 4 on Move, 5 rows per page on Move. New exported `boxInk(pending, pages, kind)` keeps the full `Stroke`. `learnWords` and `learnHand` both use it now and keep the same output and key order. The existing sheet tests pass unchanged. - **Capture format** (`src/ocr/testcapture.ts`, no Obsidian imports): the `HandwritingCapture` / `CaptureItem` types with flat x,y,width,pressure points, plus `cer` (moved here; `writepad.test.ts` imports it), `summarize`, `captureMarkdown`, `captureItems`, `replayTemplates` and `parseCapture`. - **Wiring**: `pendingTestSheet` added to settings, `SharedHandwriting` (an old `handwriting.json` without the field reads as `null`, with a test), `adoptHandwriting`, `handwritingOf`, `known` and `detectPageSize`. Also `sendTestSheet` / `readTestSheet` in `inkedge.ts`, two commands, and a settings row with **Send** / **Save** buttons. Save never calls `recognizeLocally`, `rememberReading` or `addSample`, and the endpoint gets `[]` worked examples. Docs added to the README, `docs/ocr-corrections.md` and `CHANGELOG-INK.md`. Decisions you may want to redirect: 1. **WritePad's known-word gate makes `read()` return `null`.** When that happens, the save stores the reading made *without* the lexicon, marked `accepted: false`. Otherwise the text that was rejected would be lost. 2. **Per-item CER collapses whitespace** (`itemCer`). A sentence that runs onto a second row comes back with a `\n`, which shouldn't count as a misreading. Long prompts like the 63-character "Zwölf Boxkämpfer…" may not fit one line at normal writing size, so the box leaves room below the line. `cer` itself is unchanged. 3. **Built-in and WritePad are both recorded for every line.** The import only asks WritePad when the built-in reader isn't trusted; here each reader gets its own number. 4. **`summarize` has an extra `failed` count** for endpoint `null` readings, which are left out of `meanCer`. ## Verify - Run locally; no pipeline has run for this branch yet: - The four contract test files pass: 76 tests, plus the replay test, which is skipped without a capture file. - The full `npx vitest run` passes: 775 tests, 1 skipped. - `npm run build` exits 0, including `check-bundle`, and `eslint src/` is clean. - **Red gate:** on `origin/main`, the new test file fails with `Cannot find module '../ocr/testcapture'`. The round-trip test checks that every prompt's stroke count equals what the fixture wrote. As a control, I narrowed the box hit-test in `boxInk`, and it failed (`expected 25 to be 26`). - **Replay positive control:** I wrote the synthetic capture with `HANDWRITING_CAPTURE_OUT=/tmp/hw-capture.json npx vitest run src/__tests__/testcapture.test.ts` (488 KB for 17 lines), then ran `HANDWRITING_CAPTURE=/tmp/hw-capture.json npx vitest run src/__tests__/testcapture.test.ts`. The output, cut after the first 6 of 34 item rows: ``` Handwriting capture 2026-10-20T21:30:00.000Z (plugin test, test sheet v1, 17 items, hand none) Captured: as the plugin read it, with the vault's words. Replayed: today's readers, no vault words. reader items captured replayed Built-in 17 81.8% 81.8% WritePad 17 34.1% 34.1% item reader captured replayed replayed text p1-01 Built-in 84.7% 84.7% "@m@ m @% m%%mw @ ym @% @mw" p1-01 WritePad 35.6% 35.6% "Fronsrjogt I'm konqelett iverwohrlgorten Seffriquer darch Boyern" p1-02 Built-in 88.4% 88.4% "@ m@%m@mzm % @ @" p1-02 WritePad 37.2% 37.2% "(tic quick brown patr junqlf aver the lorry dog" p1-03 Built-in 87.3% 87.3% "@%@@mew @m ym @m @wm@w @@ @@" p1-03 WritePad 42.9% 42.9% "Frwod28gfrksonqiSerjTgen Xiktor quee iiloer den grobcn Sylter soeich" ``` Captured and replayed match because the synthetic file was read without a lexicon too. These numbers come from bundled-font strokes, not a measurement of real handwriting. - **Not exercised:** the Obsidian commands themselves (PNG rendering, vault writes, opening the note). There's no Obsidian runtime here; they type-check and build. ## Follow-ups - None. #14 stays parked until the owner attaches a capture.
Author
Collaborator

✅ Agent run completed

11m7s · 19 turns · $1.00

✅ **Agent run completed** _11m7s · 19 turns · $1.00_
daniel merged commit 6955948169 into main 2026-10-20 23:53:51 +02:00
Author
Collaborator

Acceptance-intent assessment

This MR closes no issue: it is "Related to #14", which stays parked. So I checked the diff against two things: the owner's request on #14 (note 45305: prepared text to copy by hand, stored in the vault in a format that keeps the ink strokes, attachable to a forge issue) and every item of this MR's own spec. The diff delivers all five sections of the spec. I re-ran the four contract test files plus syncstate.test.ts locally: 84 passed, 0 failed, 1 skipped (the replay block, which is gated on HANDWRITING_CAPTURE). npm run build exits 0, including check-bundle.

Criteria

Owner request (#14)

  • PASS: prepared text to copy. There are 18 DE/EN prompts on a sendable sheet.
  • PASS: stored in the vault in a format that keeps the strokes. The capture JSON holds flat x,y,width,pressure, tool, color and thicknessScale.
  • PASS: easy to attach. The note and the save notice both say "Attach handwriting-test-<stamp>.json to the forge issue."

§1 Sheet layout (handsheet.ts)

  • PASS: SheetKind now includes "test". The test branch has one full-width column, the prompt in black wrapped via write/wrap above one writing line, and the v2 guides (dashed cap line, dotted x-height, solid baseline, dotted descender). The box is 230·unit high. Header and instructions match the spec.
  • PASS: TEST_SHEET_PROMPTS / TEST_SHEET_VERSION = 1 use the spec's list verbatim. Older versions stay buildable through pending.version. The test asserts ≥ 2 rows on every device and ≤ 3 pages on rm2.
  • PASS: boxInk(pending, pages, kind) is exported and keeps the full Stroke. learnWords maps it down to {x,y} with the same output shape. learnHand uses it too. The existing ink/samples tests stay green.
  • PASS: PendingSheet.kind is now "words" | "test" | undefined. Letter and word layouts are untouched; the test asserts buildSheet(profile) equals the v2 letters.

§2 Capture format (ocr/testcapture.ts)

  • PASS: a pure module with no Obsidian imports. CAPTURE_FORMAT, HandwritingCapture and CaptureItem match the spec's shape. Points are stored flat, 4 per point.
  • PASS: cer moved here, and writepad.test.ts now imports it. summarize and captureMarkdown are present; extras such as the failed count and an optional jsonName parameter are additive.
  • PASS: privacy. The capture holds only the prompts, the ink, hand (a HandLibrary) and the readings: no lexicon, samples, paths or remark text. The README, docs/ocr-corrections.md and the CHANGELOG all say so.

§3 Commands + wiring

  • PASS: pendingTestSheet is added to the settings model and defaults, SharedHandwriting, handwritingOf and adoptHandwriting. It is read tolerantly (undefined → null), and a test covers that.
  • PASS: it is added to known in main.ts and to ours in detectPageSize. sendTestSheet in inkedge.ts uses the title "Handwriting test".
  • PASS: the send-handwriting-test-sheet and save-handwriting-test-sheet commands exist. Save does the following:
    • It shows a "Nothing written on the test sheet yet…" notice when there is no ink.
    • It calls recognize(strokes, ocrTemplates(), lexicon, wordReader()) and writePad.read(strokes, lexicon) directly. It never calls recognizeLocally, rememberReading or addSample.
    • It records accepted against OCR_TRUST / WRITEPAD_TRUST, and keeps the raw text even when the reading was not accepted.
    • For the endpoint, it calls ink.ocr(http, png, undefined, false, []). A failure is caught and stored as null.
    • It writes one PNG per box via strokesToPng, plus the JSON and the note under <handwritingFolder>/test-sheets/<stamp>/.
    • It opens the note and leaves the sheet pending.
  • PASS: the settings row "Handwriting test" has Send/Save buttons and shows "A test sheet is waiting on the tablet." while one is pending.

§4 Offline replay tests

  • PASS: unit tests cover cer, summarize and captureMarkdown on a synthetic capture built from buildSheet(rm2, "test") with prompts written in the allure font.
  • PASS: the round-trip fakes a filled .rm, then asserts the per-box stroke count equals what the fixture wrote and that the prompt order is exact, with a skipped box included. It also checks that the JSON survives JSON.parse(JSON.stringify()). This is not vacuous: the MR note reports that narrowing the box hit-test in boxInk makes it fail.
  • PASS: the replay block is describe.skipIf(!HANDWRITING_CAPTURE). It reads with no lexicon and builds templates from capture.hand, falling back to readability/hershey (the same base set as ocrTemplates). It prints captured vs replayed CER per reader and per item, and asserts only the format and version. The positive-control output is pasted in MR note 45325.

§5 Docs

  • PASS: the README has a "Test how well your handwriting is read" subsection (send, write, sync, save, attach the .json) and a row in the command table.
  • PASS: docs/ocr-corrections.md covers the capture format, the "held out, never learned from" rule, the privacy rule and the replay command. There is a CHANGELOG-INK.md entry.

Constraints / out of scope

  • PASS: the readers' thresholds and logic are unchanged; this MR only measures. #14 is not closed.

Notes for the owner, none of which fails a requirement:

  1. When WritePad's lexicon gate returns null, the stored reading is the one made without the lexicon, marked not accepted.
  2. itemCer collapses whitespace before comparing.
  3. An endpoint reading counts as accepted whenever its text is non-empty.
  4. The Obsidian commands themselves (PNG rendering, vault writes, opening the note) were not exercised end to end: there is no Obsidian runtime here. They type-check and build.

Verdict

Approved

## Acceptance-intent assessment This MR closes no issue: it is "Related to #14", which stays parked. So I checked the diff against two things: the owner's request on #14 (note 45305: prepared text to copy by hand, stored in the vault in a format that keeps the ink strokes, attachable to a forge issue) and every item of this MR's own spec. The diff delivers all five sections of the spec. I re-ran the four contract test files plus `syncstate.test.ts` locally: 84 passed, 0 failed, 1 skipped (the replay block, which is gated on `HANDWRITING_CAPTURE`). `npm run build` exits 0, including `check-bundle`. ## Criteria **Owner request (#14)** - PASS: prepared text to copy. There are 18 DE/EN prompts on a sendable sheet. - PASS: stored in the vault in a format that keeps the strokes. The capture JSON holds flat x,y,width,pressure, tool, color and thicknessScale. - PASS: easy to attach. The note and the save notice both say "Attach `handwriting-test-<stamp>.json` to the forge issue." **§1 Sheet layout (`handsheet.ts`)** - PASS: `SheetKind` now includes `"test"`. The test branch has one full-width column, the prompt in black wrapped via `write`/`wrap` above one writing line, and the v2 guides (dashed cap line, dotted x-height, solid baseline, dotted descender). The box is 230·unit high. Header and instructions match the spec. - PASS: `TEST_SHEET_PROMPTS` / `TEST_SHEET_VERSION = 1` use the spec's list verbatim. Older versions stay buildable through `pending.version`. The test asserts ≥ 2 rows on every device and ≤ 3 pages on rm2. - PASS: `boxInk(pending, pages, kind)` is exported and keeps the full `Stroke`. `learnWords` maps it down to `{x,y}` with the same output shape. `learnHand` uses it too. The existing `ink`/`samples` tests stay green. - PASS: `PendingSheet.kind` is now `"words" | "test" | undefined`. Letter and word layouts are untouched; the test asserts `buildSheet(profile)` equals the v2 letters. **§2 Capture format (`ocr/testcapture.ts`)** - PASS: a pure module with no Obsidian imports. `CAPTURE_FORMAT`, `HandwritingCapture` and `CaptureItem` match the spec's shape. Points are stored flat, 4 per point. - PASS: `cer` moved here, and `writepad.test.ts` now imports it. `summarize` and `captureMarkdown` are present; extras such as the `failed` count and an optional `jsonName` parameter are additive. - PASS: privacy. The capture holds only the prompts, the ink, `hand` (a `HandLibrary`) and the readings: no lexicon, samples, paths or remark text. The README, `docs/ocr-corrections.md` and the CHANGELOG all say so. **§3 Commands + wiring** - PASS: `pendingTestSheet` is added to the settings model and defaults, `SharedHandwriting`, `handwritingOf` and `adoptHandwriting`. It is read tolerantly (`undefined → null`), and a test covers that. - PASS: it is added to `known` in `main.ts` and to `ours` in `detectPageSize`. `sendTestSheet` in `inkedge.ts` uses the title "Handwriting test". - PASS: the `send-handwriting-test-sheet` and `save-handwriting-test-sheet` commands exist. Save does the following: - It shows a "Nothing written on the test sheet yet…" notice when there is no ink. - It calls `recognize(strokes, ocrTemplates(), lexicon, wordReader())` and `writePad.read(strokes, lexicon)` directly. It never calls `recognizeLocally`, `rememberReading` or `addSample`. - It records `accepted` against `OCR_TRUST` / `WRITEPAD_TRUST`, and keeps the raw text even when the reading was not accepted. - For the endpoint, it calls `ink.ocr(http, png, undefined, false, [])`. A failure is caught and stored as `null`. - It writes one PNG per box via `strokesToPng`, plus the JSON and the note under `<handwritingFolder>/test-sheets/<stamp>/`. - It opens the note and leaves the sheet pending. - PASS: the settings row "Handwriting test" has Send/Save buttons and shows "A test sheet is waiting on the tablet." while one is pending. **§4 Offline replay tests** - PASS: unit tests cover `cer`, `summarize` and `captureMarkdown` on a synthetic capture built from `buildSheet(rm2, "test")` with prompts written in the `allure` font. - PASS: the round-trip fakes a filled `.rm`, then asserts the per-box stroke count equals what the fixture wrote and that the prompt order is exact, with a skipped box included. It also checks that the JSON survives `JSON.parse(JSON.stringify())`. This is not vacuous: the MR note reports that narrowing the box hit-test in `boxInk` makes it fail. - PASS: the replay block is `describe.skipIf(!HANDWRITING_CAPTURE)`. It reads with no lexicon and builds templates from `capture.hand`, falling back to readability/hershey (the same base set as `ocrTemplates`). It prints captured vs replayed CER per reader and per item, and asserts only the format and version. The positive-control output is pasted in MR note 45325. **§5 Docs** - PASS: the README has a "Test how well your handwriting is read" subsection (send, write, sync, save, attach the `.json`) and a row in the command table. - PASS: `docs/ocr-corrections.md` covers the capture format, the "held out, never learned from" rule, the privacy rule and the replay command. There is a `CHANGELOG-INK.md` entry. **Constraints / out of scope** - PASS: the readers' thresholds and logic are unchanged; this MR only measures. #14 is not closed. Notes for the owner, none of which fails a requirement: 1. When WritePad's lexicon gate returns `null`, the stored reading is the one made without the lexicon, marked not accepted. 2. `itemCer` collapses whitespace before comparing. 3. An endpoint reading counts as `accepted` whenever its text is non-empty. 4. The Obsidian commands themselves (PNG rendering, vault writes, opening the note) were not exercised end to end: there is no Obsidian runtime here. They type-check and build. ## Verdict Approved
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
daniel/obsidian-remarkable-round-trip!34
No description provided.