feat(ocr): LM Studio detect-and-guide in "Set up local reading for me" (desktop) #32
No reviewers
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
daniel/obsidian-remarkable-round-trip!32
Loading…
Reference in a new issue
No description provided.
Delete branch "agent/lmstudio-detect-guide"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Part of #28 (item 3). When Ollama is not running but LM Studio is, offer to point the OCR endpoint at LM Studio, pick a vision model it already has, and run Test connection. This MR adds no download or pull of LM Studio models.
Why / context
v1 (
src/ollamasetup.ts, merged in !27) handles Ollama only. When Ollama does not answer,setUpLocalOllama()goes straight toOllamaMissingModal, even if the user already runs LM Studio's server onhttp://localhost:1234.LM Studio needs no origin allowlist.
requestUrlsends noOriginheader (see !31), so pointing the endpoint at it is enough.LM Studio's APIs (researched on #28, from lmstudio.ai/docs/developer/rest)
Try these in order, falling through on 404 or an unparseable body:
GET /api/v1/models(LM Studio 0.4+) →{ models: [{ type: "llm"|"embedding"|"decision", key, display_name, capabilities?: { vision: boolean, … }, loaded_instances: [{ id, … }], … }] }.vision = capabilities?.vision === true,loaded = loaded_instances.length > 0,id = key.type === "llm".GET /api/v0/models(0.3.6+) →{ data: [{ id, type: "llm"|"vlm"|"embeddings", state: "loaded"|"not-loaded", … }] }.vision = type === "vlm",loaded = state === "loaded". Dropembeddings.GET /v1/models(OpenAI-compatible, any version) →{ data: [{ id }] }.vision = nullandloaded = null(unknown).If the server has Require API token on, it answers 401. Report that as running-but-needs-a-key, not as "not running".
What to change
src/transport/lmstudio.ts. Pure over an injectedHttpClient, in the same style assrc/transport/ollama.ts:export const LMSTUDIO_LOCAL = "http://localhost:1234";export interface LmStudioModel { id: string; vision: boolean | null; loaded: boolean | null }export type LmStudioStatus = { running: true; models: LmStudioModel[]; needsApiKey: boolean } | { running: false; reason: string }export async function detectLmStudio(http, base): Promise<LmStudioStatus>. It never throws:http()on the first call (connection refused) →running: false, with a reason worded likeunreachable()inollama.ts;{ running: true, models: [], needsApiKey: true };export function pickVisionModel(models): LmStudioModel | null. Prefer a loaded vision model, then any vision model, else null. Never pick a model withvision === false.src/lmstudiosetup.ts. The dialogue, followingOllamaMissingModal/OllamaPullModalinsrc/ollamasetup.ts(aModalwith achoice()promise andmodal-button-containerbuttons). Title: "LM Studio is running".DropdownComponentlists them, with loaded ones first and marked "(loaded)"; the default ispickVisionModel(). CTA "Use LM Studio" resolves{ kind: "use", model }./v1/modelsfallback): list them all, with a note: "pick a vision model (e.g. Qwen2.5-VL); text-only models cannot read handwriting".needsApiKey: explain that LM Studio requires an API token, and that the user should paste it into API key and set the endpoint tohttp://localhost:1234/v1by hand.{ kind: "retry" }, "Use Ollama instead" →{ kind: "ollama" }, and "Close" →{ kind: "close" }.src/ollamasetup.ts:setUpLocalOllama()(fromconst settings = host.settings();through the test notifies) into an exported helper, e.g.applyAndTest(host, endpoint, model, refresh). v1 then calls it withLOCAL_ENDPOINT, model; behaviour and wording stay unchanged.if (!found.running)branch, probedetectLmStudio(host.http, LMSTUDIO_LOCAL)beforeOllamaMissingModal. If it is running, show the LM Studio modal:use→applyAndTest(host, \${LMSTUDIO_LOCAL}/v1`, model, refresh)`;retry→ recurse intosetUpLocalOllama;ollama→ fall through to the existingOllamaMissingModal;close→ return.src/settings.tsocrActions(): in the "Set up local reading for me" description, mention that it also finds LM Studio.docs/ocr-endpoint-setup.md→ One-click setup (desktop): one bullet saying that if Ollama is not running but LM Studio's server is, it offers LM Studio and the vision models already downloaded there (no download).CHANGELOG-INK.md→ Unreleased: one bullet.Constraints and decisions
POST /api/v1/models/downloadand/loadexist, but #28 scoped this as detect-and-guide. Leave a follow-up note in the MR if you think it is worth it./v1/chat/completionsaccepts the modelkey(JIT-loads it when JIT loading is on), so storeid/keyasocrModel.host.http(requestUrl). No Node and nowindow.require, so the change stays mobile-safe. The action is still desktop-only through the existingvisible: () => Platform.isDesktopApp.OllamaPullModal,pullModel) ortransport/ollama.ts.agent/ollama-installer-download, which also editsOllamaMissingModaland the not-running branch) and theOLLAMA_ORIGINSdocs (!31). Mergeorigin/mainbefore finishing and resolve any overlap insrc/ollamasetup.tsand the docs.Validation contract
src/__tests__/lmstudio.test.tsfirst. Use a route-keyed fakeHttpClient, likefixed()insrc/__tests__/ollama.test.tsbut answering per URL. Beforesrc/transport/lmstudio.tsexists,npx vitest run src/__tests__/lmstudio.test.tsFAILS.vlm/llm/embeddings;/v1/modelsids withvision: null;httpthrowsnew Error("net::ERR_CONNECTION_REFUSED")→running: false;needsApiKey: true;pickVisionModelordering, and null when onlyvision: falsemodels exist.npm ci && npx vitest run src/__tests__/lmstudio.test.ts→ PASS. Thennpx vitest run→ whole suite PASS,npm run build→ exit 0 with check-bundle's "bundle clean" line, andnpm run lint→ exit 0.flake.nixin this repo; use plainnpm/npx.)🤖 Generated with Claude Code
✅ Agent run completed
3m0s · 19 turns · $1.82 · pushed
agent/lmstudio-detect-guide· session saved — new comments resume this runclaude referenced this pull request2026-10-20 20:06:02 +02:00
claude referenced this pull request2026-10-20 20:06:02 +02:00
What changed
src/transport/lmstudio.ts, which providesdetectLmStudioandpickVisionModel.detectLmStudiotries/api/v1/models, then/api/v0/models, then/v1/models. It never throws: a 401 means the server needs an API key, and a refused connection means not running.pickVisionModelprefers a loaded vision model, then any vision model. It never picks a text-only one.src/lmstudiosetup.ts, the "LM Studio is running" modal. It has:src/ollamasetup.ts: the end of setup is now a shared helper,applyAndTest(host, endpoint, model, refresh).OllamaMissingModal.ocrModelwith endpointhttp://localhost:1234/v1.docs/ocr-endpoint-setup.mdandCHANGELOG-INK.md.origin/main(!31). It conflicted inollamasetup.tsand the docs. I took main's "→ Troubleshooting" hint and dropped my own per-endpoint hint.Verify
npx vitest run src/__tests__/lmstudio.test.ts src/__tests__/lmstudiosetup.test.ts→ 24/24 pass.npx vitest run→ 786/786 pass.npm run build→ exit 0, "bundle clean".npm run lint→ exit 0.Follow-ups
POST /api/v1/models/downloadand/load, mirroring Ollama's pull-with-progress. That would remove the manual "open Discover" step.src/__tests__/helpers/obsidian.tsnow does more:ModalcallsonOpen/onClose, and records opened modals inModal.opened;Noticerecords shown messages inNotice.shown;DropdownComponentandappendText.claude referenced this pull request2026-10-20 20:22:55 +02:00
claude referenced this pull request2026-10-21 00:45:47 +02:00