feat(ocr): Test-connection action + actionable local-endpoint errors (Ollama/LM Studio) #7

Merged
daniel merged 2 commits from agent/ocr-endpoint-test-connection into main 2026-10-20 15:30:52 +02:00
Collaborator

Part of #3 (Part of #2). The code half of #3: make the vision-endpoint path diagnosable so a misconfigured local server tells the user exactly what is wrong instead of silently falling back to image-only.

Red gate (reproduce first)

Today, at import, src/main.ts:803 does await this.transcribeStrokes(request).catch(() => null) — every endpoint failure is swallowed and the remark silently comes back image-only, with zero guidance. There is no way for a user to find out why their endpoint produced no text. Confirm this is the current behaviour before changing anything; the Test-connection action is what makes the failure visible.

What to build (one focused change)

1. Pure helpers in src/transport/ocr.ts (testable over the injected HttpClient)

  • listModels(http, config): Promise<string[]> — GET {endpoint}/models (OpenAI route; Ollama/LM Studio both serve /v1/models). Parse the data[].id array. Map failures to TransportError with the same actionable messages as below.
  • checkConnection(http, config): Promise<{ models: string[]; probe: string | null }> (or a small result struct) — orchestrates the Test-connection flow: (a) listModels, (b) assert config.model is present (TransportError "Model "X" is not served by this endpoint — pull it (ollama pull X) or pick one of: …" listing what /models returned), (c) send a tiny probe PNG through transcribe() and return its result. Keep a 1×1 (or small) PNG as a const base64 → ArrayBuffer in the module.

2. Extend error mapping — status codes and network errors

Current transcribe() only maps HTTP status (401/403→"refused the API key", 404, generic). Two gaps:

  • Network-level failures never reach the status checks. requestUrl({throw:false}) suppresses non-2xx status throws but still rejects on connection refused / DNS / timeout — so a stopped server throws a raw error out of transcribe (today swallowed by the import .catch). Wrap the http(...) call: on a thrown/rejected request, raise TransportError("Could not reach the OCR endpoint at {endpoint} — is the server running? (connection refused)"). Detect refused/timeout by message where possible but fall back to a generic "could not reach" so no failure is lost.
  • 403 for a local server is almost always the Ollama origin/CORS allowlist, not a bad key. Branch the 401/403 handling on whether config.apiKey is set:
    • apiKey empty (local server) + 403 → TransportError("The endpoint refused the request's origin (app://obsidian.md). For Ollama, set OLLAMA_ORIGINS=app://obsidian.md (or *) and restart it. See docs/ocr-endpoint-setup.md.")
    • apiKey set + 401/403 → keep the existing "refused the API key" message.
  • 404 / model-not-pulled: keep the 404 message but, since checkConnection checks /models first, prefer the clearer "model not served / not pulled" message from step 1 when the model is simply absent.

Honesty / verification note for the implementer: the exact failure signature of an Ollama origin rejection via Obsidian's requestUrl (desktop requestUrl is a Node-side request and may send no Origin header at all, in which case Ollama does not 403) is not yet empirically confirmed — that uncertainty is the "untested because of setup difficulties" gap itself. So: always surface the raw HTTP status and a short snippet of the response body in the failure message (e.g. … (HTTP 403: <first 200 chars of body>)), and layer the specific hints above on top of known signatures. The raw passthrough guarantees the action is useful even for a signature we did not anticipate; do not rely solely on guessed status codes. Record what you actually observe against a live Ollama in an MR comment.

3. "Test connection" action in OCR settings

Add it in src/settings.ts, not as a new schema control.type. The existing action: + visible: pattern (see handwritingActions(), L125–182, and the ButtonComponent helper) already supports exactly this and is the lowest-risk path:

  • A SettingDefinition with name: "Test connection", desc explaining it pings the endpoint and sends a probe image, visible: () => this.plugin.settings.ocrMode === "endpoint", and an action that builds a ButtonComponent whose onClick runs checkConnection(obsidianHttp, { endpoint, model, apiKey }) and reports via notify(...): success → "Connected to {model} at {endpoint} — probe read OK." / failure → the TransportError.message.
  • Insert it the same way handwritingActions() is spliced in getSettingDefinitions() (the section.heading === "Handwriting" branch at L117) — add a parallel branch for the OCR section heading "Reading handwriting (OCR)" returning an ocrActions() list.
  • obsidianHttp is currently a module-private const in main.ts (L2336). Export it (or expose a small plugin.testOcrConnection() method on the plugin that settings.ts calls, mirroring how handwriting actions call this.plugin.*). Prefer a plugin.testOcrConnection() method — it keeps the obsidianHttp edge and the {endpoint,model,apiKey} assembly (already done in inkedge.ts:ocr) out of the settings UI.

4. Defaults — verify only

settingsmodel.ts L159–164: endpoint http://localhost:11434/v1, model qwen2.5vl:7b, examples 3. These are current/good (confirmed). No change expected; only correct if you find the Ollama tag has changed.

5. Tests (required by the validation contract)

Add src/__tests__/ocr.test.ts with a fake HttpClient:

  • listModels parses data[].id.
  • checkConnection throws the model-not-served message when config.model is absent from /models.
  • 403 with empty apiKey → message contains OLLAMA_ORIGINS; 403 with apiKey set → "API key" message.
  • a rejecting/throwing http (simulated connection refused) → message contains "connection refused" / "could not reach".
  • success path → returns the probe result.

Scope boundary (do NOT do here)

  • The setup guide docs/ocr-endpoint-setup.md, the README link, and the settings-note mention are the sibling MR (agent/ocr-endpoint-setup-docs, MR !6). This MR only references that doc path in error strings — it does not create it. No ordering dependency: both land independently (a referenced-but-not-yet-merged doc path in an error string is harmless).
  • Do not change the built-in reader / stroke recognizer (that is child #A) or add an in-process JS model (child #C). Do not touch WritePad.
  • Do not change the import-time .catch(() => null) fallback behaviour (silent image-only on failure is intentional for unattended imports); the Test-connection action is the surfacing path. If you want import-time failures logged, log to console only — do not block the import.

Validation contract

  • nix develop -c npx vitest run src/__tests__ → PASS, including the new ocr.test.ts. Red first: write the 403-origin and connection-refused assertions, watch them fail against the current transcribe, then implement the mapping.
  • Manual (document results in an MR comment): Ollama running + qwen2.5vl pulled → Test connection reports success; Ollama stopped → "connection refused / could not reach"; model not pulled → "not served / pull it"; wrong endpoint (no /v1) → 404 route message.
Part of #3 (Part of #2). The code half of #3: make the vision-endpoint path **diagnosable** so a misconfigured local server tells the user exactly what is wrong instead of silently falling back to image-only. ## Red gate (reproduce first) Today, at import, `src/main.ts:803` does `await this.transcribeStrokes(request).catch(() => null)` — **every** endpoint failure is swallowed and the remark silently comes back image-only, with zero guidance. There is no way for a user to find out *why* their endpoint produced no text. Confirm this is the current behaviour before changing anything; the Test-connection action is what makes the failure visible. ## What to build (one focused change) ### 1. Pure helpers in `src/transport/ocr.ts` (testable over the injected `HttpClient`) - `listModels(http, config): Promise<string[]>` — `GET {endpoint}/models` (OpenAI route; Ollama/LM Studio both serve `/v1/models`). Parse the `data[].id` array. Map failures to `TransportError` with the same actionable messages as below. - `checkConnection(http, config): Promise<{ models: string[]; probe: string | null }>` (or a small result struct) — orchestrates the Test-connection flow: (a) `listModels`, (b) assert `config.model` is present (TransportError "Model \"X\" is not served by this endpoint — pull it (ollama pull X) or pick one of: …" listing what `/models` returned), (c) send a **tiny probe PNG** through `transcribe()` and return its result. Keep a 1×1 (or small) PNG as a `const` base64 → ArrayBuffer in the module. ### 2. Extend error mapping — status codes **and** network errors Current `transcribe()` only maps HTTP status (401/403→"refused the API key", 404, generic). Two gaps: - **Network-level failures never reach the status checks.** `requestUrl({throw:false})` suppresses non-2xx *status* throws but still **rejects** on connection refused / DNS / timeout — so a stopped server throws a raw error out of `transcribe` (today swallowed by the import `.catch`). Wrap the `http(...)` call: on a thrown/rejected request, raise `TransportError("Could not reach the OCR endpoint at {endpoint} — is the server running? (connection refused)")`. Detect refused/timeout by message where possible but fall back to a generic "could not reach" so no failure is lost. - **403 for a local server is almost always the Ollama origin/CORS allowlist, not a bad key.** Branch the 401/403 handling on whether `config.apiKey` is set: - apiKey empty (local server) + 403 → `TransportError("The endpoint refused the request's origin (app://obsidian.md). For Ollama, set OLLAMA_ORIGINS=app://obsidian.md (or *) and restart it. See docs/ocr-endpoint-setup.md.")` - apiKey set + 401/403 → keep the existing "refused the API key" message. - **404 / model-not-pulled:** keep the 404 message but, since `checkConnection` checks `/models` first, prefer the clearer "model not served / not pulled" message from step 1 when the model is simply absent. **Honesty / verification note for the implementer:** the *exact* failure signature of an Ollama origin rejection via Obsidian's `requestUrl` (desktop `requestUrl` is a Node-side request and may send no `Origin` header at all, in which case Ollama does *not* 403) is **not yet empirically confirmed** — that uncertainty is the "untested because of setup difficulties" gap itself. So: **always surface the raw HTTP status and a short snippet of the response body in the failure message** (e.g. `… (HTTP 403: <first 200 chars of body>)`), and layer the specific hints above on top of known signatures. The raw passthrough guarantees the action is useful even for a signature we did not anticipate; do not rely solely on guessed status codes. Record what you actually observe against a live Ollama in an MR comment. ### 3. "Test connection" action in OCR settings Add it in `src/settings.ts`, **not** as a new schema `control.type`. The existing `action:` + `visible:` pattern (see `handwritingActions()`, L125–182, and the `ButtonComponent` helper) already supports exactly this and is the lowest-risk path: - A `SettingDefinition` with `name: "Test connection"`, `desc` explaining it pings the endpoint and sends a probe image, `visible: () => this.plugin.settings.ocrMode === "endpoint"`, and an `action` that builds a `ButtonComponent` whose `onClick` runs `checkConnection(obsidianHttp, { endpoint, model, apiKey })` and reports via `notify(...)`: success → "Connected to {model} at {endpoint} — probe read OK." / failure → the `TransportError.message`. - Insert it the same way `handwritingActions()` is spliced in `getSettingDefinitions()` (the `section.heading === "Handwriting"` branch at L117) — add a parallel branch for the OCR section heading `"Reading handwriting (OCR)"` returning an `ocrActions()` list. - `obsidianHttp` is currently a module-private const in `main.ts` (L2336). Export it (or expose a small `plugin.testOcrConnection()` method on the plugin that settings.ts calls, mirroring how handwriting actions call `this.plugin.*`). Prefer a `plugin.testOcrConnection()` method — it keeps the `obsidianHttp` edge and the `{endpoint,model,apiKey}` assembly (already done in `inkedge.ts:ocr`) out of the settings UI. ### 4. Defaults — verify only `settingsmodel.ts` L159–164: endpoint `http://localhost:11434/v1`, model `qwen2.5vl:7b`, examples 3. These are current/good (confirmed). No change expected; only correct if you find the Ollama tag has changed. ### 5. Tests (required by the validation contract) Add `src/__tests__/ocr.test.ts` with a fake `HttpClient`: - `listModels` parses `data[].id`. - `checkConnection` throws the model-not-served message when `config.model` is absent from `/models`. - 403 with empty apiKey → message contains `OLLAMA_ORIGINS`; 403 with apiKey set → "API key" message. - a rejecting/throwing http (simulated connection refused) → message contains "connection refused" / "could not reach". - success path → returns the probe result. ## Scope boundary (do NOT do here) - The setup guide `docs/ocr-endpoint-setup.md`, the README link, and the settings-note mention are the **sibling MR** (`agent/ocr-endpoint-setup-docs`, MR !6). This MR only *references* that doc path in error strings — it does not create it. No ordering dependency: both land independently (a referenced-but-not-yet-merged doc path in an error string is harmless). - Do not change the built-in reader / stroke recognizer (that is child #A) or add an in-process JS model (child #C). Do not touch WritePad. - Do not change the import-time `.catch(() => null)` fallback behaviour (silent image-only on failure is intentional for unattended imports); the Test-connection action is the surfacing path. If you want import-time failures logged, log to console only — do not block the import. ## Validation contract - `nix develop -c npx vitest run src/__tests__` → PASS, including the new `ocr.test.ts`. Red first: write the 403-origin and connection-refused assertions, watch them fail against the current `transcribe`, then implement the mapping. - Manual (document results in an MR comment): Ollama running + `qwen2.5vl` pulled → Test connection reports success; Ollama stopped → "connection refused / could not reach"; model not pulled → "not served / pull it"; wrong endpoint (no `/v1`) → 404 route message.
claude self-assigned this 2026-10-20 15:17:46 +02:00
Hand-off stub for issue #3 (code MR). See MR description.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Author
Collaborator

✅ Agent run completed

8m27s · 71 turns · $3.56 · pushed agent/ocr-endpoint-test-connection · session saved — new comments resume this run

✅ **Agent run completed** _8m27s · 71 turns · $3.56 · pushed `agent/ocr-endpoint-test-connection` · session saved — new comments resume this run_
Make the endpoint OCR path tell the user what is wrong instead of
silently falling back to image-only.

- transport/ocr.ts: add listModels() and checkConnection() (probe a 1×1
  PNG through the real transcribe path). Extend transcribe() mapping:
  wrap the http call so a rejected request (stopped server) becomes a
  reachability error; branch 401/403 on whether an API key is set so a
  local 403 points at OLLAMA_ORIGINS; always append the raw HTTP status
  and a body snippet so unanticipated signatures stay useful.
- main.ts: expose plugin.testOcrConnection() over obsidianHttp.
- settings.ts: "Test connection" action in the OCR section, reporting
  success or the TransportError message via notify().
- src/__tests__/ocr.test.ts: parse, model-not-served, 403 both ways,
  connection-refused, and success-path coverage.

Part of #3 (Part of #2).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Author
Collaborator

What changed

  • src/transport/ocr.ts — added listModels() (GET /models, parses data[].id) and checkConnection() (lists models → asserts config.model is served → sends a 1×1 probe PNG through the real transcribe() path). Extended transcribe() mapping: a rejected request (stopped server / DNS / timeout) now becomes Could not reach … (connection refused) instead of propagating raw; 401/403 branches on whether an API key is set so a local 403 points at OLLAMA_ORIGINS=app://obsidian.md while a keyed endpoint keeps the "refused the API key" message; every non-2xx now appends the raw (HTTP <status>: <first 200 chars of body>) so an unanticipated signature is still actionable.
  • src/main.ts — exposed plugin.testOcrConnection() over the module-private obsidianHttp (keeps the edge + {endpoint,model,apiKey} assembly out of the UI).
  • src/settings.ts — "Test connection" action spliced into the Reading handwriting (OCR) section via a parallel ocrActions() branch (same action:/visible: pattern as handwritingActions()); reports success (Connected to {model} at {endpoint} — probe read OK.) or the TransportError.message via notify().
  • Import-time .catch(() => null) at main.ts:803 left untouched (silent image-only fallback is intentional; Test connection is the surfacing path). Defaults verified unchanged.

Verify

  • Red gate first: the 403-origin and connection-refused assertions were written and watched fail against the current transcribe (403→"refused the API key"; raw rejection propagated), then the mapping was implemented.
  • npx vitest run src/__tests__/ocr.test.ts → 10 PASS (parse, model-not-served, 403 both ways, raw-status passthrough, connection-refused, opaque-reject, success + NO_TEXT probe).
  • Full suite npx vitest run src/__tests__ → 663 PASS; tsc -noEmit -skipLibCheck → clean; eslint on changed files → clean; production esbuild + check-bundle.mjs → clean.
  • No CI pipeline runs on MR branch pushes in this repo (only upstream-sync scheduled + release on tag), so there is no pipeline to gate on — verification rests on the local runs above.

Honesty / live-Ollama note

The manual matrix (Ollama running→success, stopped→reachability, model-not-pulled→"not served", no-/v1→404 route) could not be run — there is no live Ollama in this sandbox. Per the hand-off's verification note, the exact failure signature of an Ollama origin rejection via Obsidian's desktop requestUrl (which may send no Origin header, in which case Ollama does not 403) remains empirically unconfirmed. The implementation does not rely on the guessed status alone: it always passes through the raw HTTP status + body snippet, and layers the OLLAMA_ORIGINS / "model not served" / 404-route hints on top, so the action stays useful even for a signature we did not anticipate. A reviewer with a live Ollama should confirm the 403-origin path end-to-end.

Follow-ups

  • Setup guide docs/ocr-endpoint-setup.md (referenced by the error strings) is the sibling MR !6 (agent/ocr-endpoint-setup-docs) — no ordering dependency; a referenced-but-unmerged doc path is harmless.
## What changed - **`src/transport/ocr.ts`** — added `listModels()` (GET `/models`, parses `data[].id`) and `checkConnection()` (lists models → asserts `config.model` is served → sends a 1×1 probe PNG through the real `transcribe()` path). Extended `transcribe()` mapping: a **rejected request** (stopped server / DNS / timeout) now becomes `Could not reach … (connection refused)` instead of propagating raw; **401/403 branches on whether an API key is set** so a local `403` points at `OLLAMA_ORIGINS=app://obsidian.md` while a keyed endpoint keeps the "refused the API key" message; every non-2xx now appends the **raw `(HTTP <status>: <first 200 chars of body>)`** so an unanticipated signature is still actionable. - **`src/main.ts`** — exposed `plugin.testOcrConnection()` over the module-private `obsidianHttp` (keeps the edge + `{endpoint,model,apiKey}` assembly out of the UI). - **`src/settings.ts`** — "Test connection" action spliced into the `Reading handwriting (OCR)` section via a parallel `ocrActions()` branch (same `action:`/`visible:` pattern as `handwritingActions()`); reports success (`Connected to {model} at {endpoint} — probe read OK.`) or the `TransportError.message` via `notify()`. - Import-time `.catch(() => null)` at `main.ts:803` left untouched (silent image-only fallback is intentional; Test connection is the surfacing path). Defaults verified unchanged. ## Verify - Red gate first: the 403-origin and connection-refused assertions were written and watched **fail** against the current `transcribe` (403→"refused the API key"; raw rejection propagated), then the mapping was implemented. - `npx vitest run src/__tests__/ocr.test.ts` → **10 PASS** (parse, model-not-served, 403 both ways, raw-status passthrough, connection-refused, opaque-reject, success + NO_TEXT probe). - Full suite `npx vitest run src/__tests__` → **663 PASS**; `tsc -noEmit -skipLibCheck` → clean; `eslint` on changed files → clean; production `esbuild` + `check-bundle.mjs` → clean. - No CI pipeline runs on MR branch pushes in this repo (only `upstream-sync` scheduled + `release` on tag), so there is no pipeline to gate on — verification rests on the local runs above. ### Honesty / live-Ollama note The manual matrix (Ollama running→success, stopped→reachability, model-not-pulled→"not served", no-`/v1`→404 route) **could not be run** — there is no live Ollama in this sandbox. Per the hand-off's verification note, the exact failure signature of an Ollama origin rejection via Obsidian's desktop `requestUrl` (which may send **no** `Origin` header, in which case Ollama does *not* 403) remains **empirically unconfirmed**. The implementation does not rely on the guessed status alone: it **always** passes through the raw HTTP status + body snippet, and layers the `OLLAMA_ORIGINS` / "model not served" / 404-route hints on top, so the action stays useful even for a signature we did not anticipate. A reviewer with a live Ollama should confirm the 403-origin path end-to-end. ## Follow-ups - Setup guide `docs/ocr-endpoint-setup.md` (referenced by the error strings) is the sibling MR !6 (`agent/ocr-endpoint-setup-docs`) — no ordering dependency; a referenced-but-unmerged doc path is harmless.
daniel merged commit 176156cd0d into main 2026-10-20 15:30:52 +02:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
daniel/obsidian-remarkable-round-trip!7
No description provided.