refactor(ocr): versioned migrate() seam for the sample store #19

Merged
daniel merged 2 commits from agent/ocr-store-migrate into main 2026-10-20 17:18:02 +02:00
Collaborator

Part of #17 (OCR Direction 2 — richer persisted store), Part of #13.

Context

Today loadSamples (src/inkedge.ts:315) casts raw JSON straight to SampleStore from either host.state.readSamples() or the on-disk ocr/samples.json, and the format: 1 literal (src/ocr/samples.ts:118; emptyStore, :137) is never read or checked. Direction 2 keeps adding optional fields (letters? landed in !12; joins? is coming in the join-synthesis work). Optional fields are backward-compatible, but the store has no seam for the first change that isn't, and no validation that a loaded blob is actually a store. This MR adds that seam — the first real format version transition — so later shape changes have a home. This is issue #17 part 3.

What to change

src/ocr/samples.ts

  1. Add export const CURRENT_FORMAT = 2. Change SampleStore.format from 1 to 2, and emptyStore() to return format: CURRENT_FORMAT.
  2. Add export function migrate(raw: unknown): SampleStore, a single linear version ladder so the next bump is "add a case":
    • Non-object / null / wrong-typed input → return emptyStore() (defends the raw cast that exists today).
    • Read raw.format. Missing or 1 ⇒ a format-1 store: upgrade losslessly to 2 — default absent required containers (words, remarks → {}), leave optional fields (letters, and future joins) exactly as-is, set format: 2. A format-1 store carries the same information, so the upgrade adds no data, only the version.
    • format === 2 ⇒ return as-is after the same defensive container defaulting.
    • Unknown / newer format ⇒ be conservative: default containers and return it rather than throwing or wiping, so a store written by a newer plugin build survives a downgrade.

src/inkedge.ts

  1. In loadSamples (:315) route both sources through migrate(...) instead of the raw cast:
    this.samples ??= migrate((await this.host.state.readSamples()) ?? (await this.readJson<unknown>(\${this.dir}/ocr/samples.json`)) ?? emptyStore()) (emptyStore()is already current, somigrate` is a no-op on it.)

Notes for the implementer

  • The format: 1 literals in src/ink/handsheet.ts:330,477 are a different, unrelated interface (the hand sheet). Do not touch them.
  • No existing test constructs a SampleStore literal with format: 1 — they all go through emptyStore(). So the type change to 2 should not break test literals; if the typechecker surfaces any, fix them to use emptyStore()/CURRENT_FORMAT.
  • This is pure plumbing + a version bump: no new fields, no change to reads/writes or retention. The letter-retention change (MR on agent/ocr-letter-diversity) and the join work (child issue) are separate.

Validation contract

npm install then npx vitest run src/__tests__ → PASS (no flake.nix).

Add a test (new src/__tests__/migrate.test.ts, or extend samples.test.ts):

  • Format-1 fixture upgrades losslessly. Hand-write a literal { format: 1, words: { und: [[...one stroke...]] }, remarks: { r1: {...a RemarkRecord...} } } with no letters. Assert migrate(fixture) returns format === 2, with words and remarks deep-equal to the input, and letters absent-or-empty.
    • Red gate (must fail on main): on main there is no migrate; the raw cast leaves format: 1, so an assertion that the loaded store is format: 2 fails before this change. State that you watched it fail on main first.
  • migrate(emptyStore()) is idempotent and stays format: 2.
  • Round-trip: migrate(JSON.parse(JSON.stringify(store))) deep-equals a current store.
  • Garbage input: migrate(null) and migrate(42) each return a valid empty current store (not a throw).
  • Keep the full existing suite green (notably any store-serialisation test).
Part of #17 (OCR Direction 2 — richer persisted store), Part of #13. ## Context Today `loadSamples` (`src/inkedge.ts:315`) casts raw JSON straight to `SampleStore` from either `host.state.readSamples()` or the on-disk `ocr/samples.json`, and the `format: 1` literal (`src/ocr/samples.ts:118`; `emptyStore`, :137) is **never read or checked**. Direction 2 keeps adding optional fields (`letters?` landed in !12; `joins?` is coming in the join-synthesis work). Optional fields are backward-compatible, but the store has **no seam** for the first change that isn't, and no validation that a loaded blob is actually a store. This MR adds that seam — the first real format version transition — so later shape changes have a home. This is issue #17 part 3. ## What to change ### `src/ocr/samples.ts` 1. Add `export const CURRENT_FORMAT = 2`. Change `SampleStore.format` from `1` to `2`, and `emptyStore()` to return `format: CURRENT_FORMAT`. 2. Add `export function migrate(raw: unknown): SampleStore`, a single linear version ladder so the next bump is "add a case": - Non-object / `null` / wrong-typed input → return `emptyStore()` (defends the raw cast that exists today). - Read `raw.format`. Missing or `1` ⇒ a format-1 store: upgrade **losslessly** to 2 — default absent required containers (`words`, `remarks` → `{}`), leave optional fields (`letters`, and future `joins`) exactly as-is, set `format: 2`. A format-1 store carries the same information, so the upgrade adds no data, only the version. - `format === 2` ⇒ return as-is after the same defensive container defaulting. - Unknown / newer format ⇒ be conservative: default containers and return it rather than throwing or wiping, so a store written by a newer plugin build survives a downgrade. ### `src/inkedge.ts` 3. In `loadSamples` (:315) route **both** sources through `migrate(...)` instead of the raw cast: `this.samples ??= migrate((await this.host.state.readSamples()) ?? (await this.readJson<unknown>(\`${this.dir}/ocr/samples.json\`)) ?? emptyStore())` (`emptyStore()` is already current, so `migrate` is a no-op on it.) ## Notes for the implementer - The `format: 1` literals in `src/ink/handsheet.ts:330,477` are a **different, unrelated interface** (the hand sheet). Do **not** touch them. - No existing test constructs a `SampleStore` literal with `format: 1` — they all go through `emptyStore()`. So the type change to `2` should not break test literals; if the typechecker surfaces any, fix them to use `emptyStore()`/`CURRENT_FORMAT`. - This is **pure plumbing + a version bump**: no new fields, no change to reads/writes or retention. The letter-retention change (MR on `agent/ocr-letter-diversity`) and the join work (child issue) are separate. ## Validation contract `npm install` then `npx vitest run src/__tests__` → **PASS** (no `flake.nix`). Add a test (new `src/__tests__/migrate.test.ts`, or extend `samples.test.ts`): - **Format-1 fixture upgrades losslessly.** Hand-write a literal `{ format: 1, words: { und: [[...one stroke...]] }, remarks: { r1: {...a RemarkRecord...} } }` with **no** `letters`. Assert `migrate(fixture)` returns `format === 2`, with `words` and `remarks` **deep-equal** to the input, and `letters` absent-or-empty. - **Red gate (must fail on `main`):** on `main` there is no `migrate`; the raw cast leaves `format: 1`, so an assertion that the loaded store is `format: 2` fails before this change. State that you watched it fail on `main` first. - `migrate(emptyStore())` is idempotent and stays `format: 2`. - Round-trip: `migrate(JSON.parse(JSON.stringify(store)))` deep-equals a current `store`. - Garbage input: `migrate(null)` and `migrate(42)` each return a valid empty current store (not a throw). - Keep the full existing suite green (notably any store-serialisation test).
claude self-assigned this 2026-10-20 17:13:00 +02:00
Author
Collaborator

✅ Agent run completed

3m41s · 28 turns · $1.49 · pushed agent/ocr-store-migrate · session saved — new comments resume this run

✅ **Agent run completed** _3m41s · 28 turns · $1.49 · pushed `agent/ocr-store-migrate` · session saved — new comments resume this run_
Bump SampleStore.format 1 -> 2 (CURRENT_FORMAT) and route both loadSamples
sources through migrate(), a linear version ladder: non-objects become an
empty store, format-1 upgrades losslessly to 2 (defaulting absent containers,
leaving optional fields as-is), current format is a no-op, and an unknown/newer
format is preserved so a newer build's store survives a downgrade.

Part of #17
Author
Collaborator

What changed

  • src/ocr/samples.ts: added export const CURRENT_FORMAT = 2, bumped SampleStore.format 1 → 2, and emptyStore() now returns format: CURRENT_FORMAT. Added export function migrate(raw: unknown): SampleStore — a linear version ladder: non-object/null/array → emptyStore(); missing/1 → lossless upgrade to 2 (default absent words/remarks, leave letters/future joins as-is); 2 → no-op after the same container defaulting; unknown/newer format → preserved as-is (downgrade-safe).
  • src/inkedge.ts: loadSamples now routes both sources (host.state.readSamples() and on-disk ocr/samples.json) through migrate(...) instead of the raw cast.
  • Left the unrelated format: 1 hand-sheet literals in src/ink/handsheet.ts untouched.

Verify

npx tsc --noEmit clean; npx vitest run src/__tests__ → 683 pass / 0 fail. New src/__tests__/migrate.test.ts (6 tests): format-1 fixture upgrades losslessly (format===2, words/remarks deep-equal, letters absent/empty), idempotence on emptyStore(), JSON round-trip, newer-format downgrade preservation, and garbage input (null/42/…) → valid empty store. Red gate: watched migrate.test.ts fail on the pre-change tree with migrate is not a function before implementing.

Follow-ups

None. (Pure plumbing + version bump; the letter-retention and join-synthesis work remain separate per the MR description.)

## What changed - `src/ocr/samples.ts`: added `export const CURRENT_FORMAT = 2`, bumped `SampleStore.format` `1 → 2`, and `emptyStore()` now returns `format: CURRENT_FORMAT`. Added `export function migrate(raw: unknown): SampleStore` — a linear version ladder: non-object/`null`/array → `emptyStore()`; missing/`1` → lossless upgrade to 2 (default absent `words`/`remarks`, leave `letters`/future `joins` as-is); `2` → no-op after the same container defaulting; unknown/newer format → preserved as-is (downgrade-safe). - `src/inkedge.ts`: `loadSamples` now routes **both** sources (`host.state.readSamples()` and on-disk `ocr/samples.json`) through `migrate(...)` instead of the raw cast. - Left the unrelated `format: 1` hand-sheet literals in `src/ink/handsheet.ts` untouched. ## Verify `npx tsc --noEmit` clean; `npx vitest run src/__tests__` → **683 pass / 0 fail**. New `src/__tests__/migrate.test.ts` (6 tests): format-1 fixture upgrades losslessly (`format===2`, `words`/`remarks` deep-equal, `letters` absent/empty), idempotence on `emptyStore()`, JSON round-trip, newer-format downgrade preservation, and garbage input (`null`/`42`/…) → valid empty store. **Red gate:** watched `migrate.test.ts` fail on the pre-change tree with `migrate is not a function` before implementing. ## Follow-ups None. (Pure plumbing + version bump; the letter-retention and join-synthesis work remain separate per the MR description.)
daniel merged commit fca18d0aa6 into main 2026-10-20 17:18:02 +02:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
daniel/obsidian-remarkable-round-trip!19
No description provided.