From ad7e24e40d4e05ce31ba644b01eb8f536fc882f3 Mon Sep 17 00:00:00 2001 From: Hibryda Date: Wed, 11 Mar 2026 03:56:05 +0100 Subject: [PATCH] docs: update meta files for Codex/Ollama provider runners --- .claude/CLAUDE.md | 6 +++--- CHANGELOG.md | 3 +++ CLAUDE.md | 8 ++++++++ TODO.md | 3 +-- docs/provider-adapter/progress.md | 32 +++++++++++++++++++++++++++++++ docs/v3-progress.md | 21 ++++++++++++++++++++ 6 files changed, 68 insertions(+), 5 deletions(-) diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index ff3588d..07c9e99 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -5,7 +5,7 @@ - v1 is a single-file Python app (`bterminal.py`). Changes are localized. - v2 docs are in `docs/`. Architecture decisions are in `docs/task_plan.md`. - v2 Phases 1-7 + multi-machine (A-D) + profiles/skills complete. Extras: SSH, ctx, themes, detached mode, auto-updater, shiki, copy/paste, session resume, drag-resize, session groups, Deno sidecar, Claude profiles, skill discovery. -- v3 Mission Control (All Phases 1-10 Complete + S-1 Phase 1/1.5/2 Conflict Detection + Provider Adapter Pattern): project groups, workspace store, 12 Workspace components, session continuity, workspace teardown, file overlap conflict detection, inotify-based external write detection, multi-provider adapter pattern (3 phases: core abstraction, settings UI, sidecar routing), dead v2 component cleanup. 202 vitest + 42 cargo tests. +- v3 Mission Control (All Phases 1-10 Complete + S-1 Phase 1/1.5/2/3 + S-2 Session Anchors + Provider Adapter Pattern + Provider Runners): project groups, workspace store, 12 Workspace components, session continuity, workspace teardown, file overlap conflict detection, inotify-based external write detection, multi-provider adapter pattern (3 phases + Codex/Ollama runners), worktree isolation, session anchors, dead v2 component cleanup. 256 vitest + 42 cargo tests. - v3 docs: `docs/v3-task_plan.md`, `docs/v3-findings.md`, `docs/v3-progress.md`. - Consult Memora (tag: `bterminal`) before making architectural changes. @@ -31,7 +31,7 @@ ## Key Technical Constraints - WebKit2GTK has no WebGL — xterm.js must use Canvas addon explicitly. -- Agent sessions use `@anthropic-ai/claude-agent-sdk` query() function (migrated from raw CLI spawning due to piped stdio hang bug). SDK handles subprocess management internally. All output goes through the adapter layer (`src/lib/adapters/claude-messages.ts` via `message-adapters.ts` registry) — SDK message format matches CLI stream-json. Multi-provider support: message-adapters.ts routes by ProviderId to provider-specific parsers (currently only Claude registered). +- Agent sessions use `@anthropic-ai/claude-agent-sdk` query() function (migrated from raw CLI spawning due to piped stdio hang bug). SDK handles subprocess management internally. All output goes through the adapter layer (`src/lib/adapters/claude-messages.ts` via `message-adapters.ts` registry) — SDK message format matches CLI stream-json. Multi-provider support: message-adapters.ts routes by ProviderId to provider-specific parsers (claude-messages.ts, codex-messages.ts, ollama-messages.ts — all 3 registered). - Sidecar uses per-provider runner bundles (`sidecar/dist/{provider}-runner.mjs`). Currently only `claude-runner.mjs` exists. SidecarManager.resolve_sidecar_for_provider(provider) finds the right runner file. Deno preferred (faster startup), Node.js fallback. Communicates with Rust via stdio NDJSON. Claude CLI auto-detected at startup via `findClaudeCli()` — checks ~/.local/bin/claude, ~/.claude/local/claude, /usr/local/bin/claude, /usr/bin/claude, then `which claude`. Path passed to SDK via `pathToClaudeCodeExecutable` option. Agents error immediately if CLI not found. Provider env var stripping: strip_provider_env_var() strips CLAUDE*/CODEX*/OLLAMA* vars (whitelists CLAUDE_CODE_EXPERIMENTAL_*). Dual-layer: (1) Rust env_clear() + clean_env, (2) JS runner SDK `env` option. Session stop uses AbortController.abort(). `agent-runner-deno.ts` exists as standalone alternative runner but is NOT used by SidecarManager. - AgentPane does NOT stop agents in onDestroy — onDestroy fires on layout remounts, not just explicit close. Stop-on-close is handled externally (was TilingGrid in v2, now workspace teardown in v3). - Agent dispatcher (`src/lib/agent-dispatcher.ts`) is a singleton that routes sidecar events to the agent store. Provider-aware: sessionProviderMap routes messages through adaptMessage(provider, event) from message-adapters.ts. Handles subagent routing (project-scoped sessions skip layout pane, render in TeamAgentsPanel; detached mode creates layout pane). Session persistence via registerSessionProject(sessionId, projectId, provider) + persistSessionForProject() (saves state + messages to SQLite on complete). @@ -43,7 +43,7 @@ - Skill discovery: claude_list_skills() reads ~/.claude/skills/ (dirs with SKILL.md or .md files). claude_read_skill() reads content. AgentPane `/` prefix triggers autocomplete menu. Skill content injected as prompt via expandSkillPrompt(). - claude-bridge.ts adapter wraps profile/skill Tauri commands (ClaudeProfile, ClaudeSkill interfaces). provider-bridge.ts wraps claude-bridge as generic provider bridge (delegates by ProviderId). - Provider adapter pattern: ProviderId = 'claude' | 'codex' | 'ollama'. ProviderCapabilities flags gate UI (hasProfiles, hasSkills, hasModelSelection, hasSandbox, supportsSubagents, supportsCost, supportsResume). ProviderMeta registered via registerProvider() in App.svelte onMount. AgentPane receives provider + capabilities props. SettingsTab has Providers section with collapsible per-provider config panels. ProjectConfig.provider field for per-project selection. Settings persisted as `provider_settings` JSON blob. -- Sidecar build: `npm run build:sidecar` bundles SDK into claude-runner.mjs via esbuild (no --external, SDK included in bundle). Future providers will have separate runner bundles. +- Sidecar build: `npm run build:sidecar` builds all 3 runners via esbuild (claude-runner.mjs, codex-runner.mjs, ollama-runner.mjs). Each is a standalone ESM bundle. Codex runner dynamically imports @openai/codex-sdk (graceful failure if not installed). Ollama runner uses native fetch (zero deps). - Agent preview terminal: `AgentPreviewPane.svelte` is a read-only xterm.js terminal (disableStdin:true) that subscribes to an agent session's messages via `$derived(getAgentSession(sessionId))` and renders tool calls/results in real-time. Bash commands shown as cyan `❯ cmd`, file ops as yellow `[Read] path`, results as plain text (80-line truncation), errors in red. Spawned via 👁 button in TerminalTabs (appears when agentSessionId prop is set). TerminalTab type: `'agent-preview'` with `agentSessionId` field. Deduplicates — won't create two previews for the same session. ProjectBox passes mainSessionId to TerminalTabs. - Maximum 4 active xterm.js instances to avoid WebKit2GTK memory issues. Agent preview uses disableStdin and no PTY so is lighter, but still counts. - Store files using Svelte 5 runes (`$state`, `$derived`) MUST have `.svelte.ts` extension (not `.ts`). Import with `.svelte` suffix. Plain `.ts` compiles but fails at runtime with "rune_outside_svelte". diff --git a/CHANGELOG.md b/CHANGELOG.md index 2111c4a..7530fb3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - `claude_read_skill` path traversal: added `canonicalize()` + `starts_with()` validation to prevent reading arbitrary files via crafted skill paths (lib.rs) ### Added +- **Codex provider runner** — `sidecar/codex-runner.ts` wraps `@openai/codex-sdk` (dynamic import, graceful failure if not installed). Maps Codex ThreadEvents (agent_message, reasoning, command_execution, file_change, mcp_tool_call, web_search) to common AgentMessage format via `codex-messages.ts` adapter. Sandbox/approval mode mapping from BTerminal permission modes. Session resume via thread ID. `providers/codex.ts` ProviderMeta (gpt-5.4 default, hasSandbox, supportsResume). 19 adapter tests +- **Ollama provider runner** — `sidecar/ollama-runner.ts` uses direct HTTP to `localhost:11434/api/chat` with NDJSON streaming (zero external dependencies). Health check before session start. Configurable host/model/num_ctx/think via providerConfig. Supports Qwen3 extended thinking. `ollama-messages.ts` adapter maps streaming chunks to AgentMessage (text, thinking, cost with token counts). `providers/ollama.ts` ProviderMeta (qwen3:8b default, modelSelection only). 11 adapter tests +- All 3 providers registered in App.svelte onMount + message-adapters.ts. `build:sidecar` builds all 3 runners - **S-1 Phase 3: Worktree isolation per project** — per-project `useWorktrees` toggle in SettingsTab. When enabled, agents run in git worktrees at `/.claude/worktrees//` via SDK `extraArgs: { worktree: sessionId }`. CWD-based worktree detection in agent-dispatcher (`detectWorktreeFromCwd()`) matches `.claude/`, `.codex/`, `.cursor/` worktree patterns on init events. Dual detection: CWD-based (primary) + tool_call-based (subagent fallback). 8 files, +125 lines, 7 new tests. 226 vitest + 42 cargo tests - **S-2 Session Anchors** — preserves important conversation turns through context compaction chains. Auto-anchors first 3 turns with observation masking (reasoning preserved in full per research). Manual pin button on AgentPane text messages. Three anchor types: auto (re-injectable), pinned (display-only), promoted (user-promoted, re-injectable). Re-injection via `system_prompt` field. ContextTab anchor section with budget meter bar, per-anchor promote/demote/remove actions. SQLite `session_anchors` table with 5 CRUD commands. 5 new files, 7 modified. 219 vitest + 42 cargo tests - **Configurable anchor budget scale** — `AnchorBudgetScale` type with 4 presets: Small (2K), Medium (6K, default), Large (12K), Full (20K). Per-project 4-stop range slider in SettingsTab. `ProjectConfig.anchorBudgetScale` persisted in groups.json. ContextTab budget meter derives from project setting. agent-dispatcher resolves scale on auto-anchor diff --git a/CLAUDE.md b/CLAUDE.md index e302381..a1e3f09 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -58,6 +58,10 @@ Terminal emulator with SSH and Claude Code session management. v1 (GTK3+VTE Pyth | `v2/src/lib/providers/types.ts` | Provider abstraction types (ProviderId, ProviderCapabilities, ProviderMeta, ProviderSettings) | | `v2/src/lib/providers/registry.svelte.ts` | Svelte 5 rune-based provider registry (registerProvider, getProviders) | | `v2/src/lib/providers/claude.ts` | Claude provider metadata constant (CLAUDE_PROVIDER) | +| `v2/src/lib/providers/codex.ts` | Codex provider metadata constant (CODEX_PROVIDER, gpt-5.4 default) | +| `v2/src/lib/providers/ollama.ts` | Ollama provider metadata constant (OLLAMA_PROVIDER, qwen3:8b default) | +| `v2/src/lib/adapters/codex-messages.ts` | Codex message adapter (ThreadEvent parser) | +| `v2/src/lib/adapters/ollama-messages.ts` | Ollama message adapter (streaming chunk parser) | | `v2/src/lib/agent-dispatcher.ts` | Routes sidecar events to agent store + subagent routing + session persistence + toast notifications + CWD-based worktree detection | | `v2/src/lib/adapters/file-bridge.ts` | File watcher IPC wrapper | | `v2/src/lib/adapters/settings-bridge.ts` | Settings IPC wrapper (get/set/list) | @@ -100,9 +104,13 @@ Terminal emulator with SSH and Claude Code session management. v1 (GTK3+VTE Pyth | `v2/src/lib/adapters/session-bridge.ts` | Session/layout/group persistence IPC wrapper | | `v2/src/lib/components/Markdown/MarkdownPane.svelte` | Markdown file viewer (marked.js + shiki, live reload) | | `v2/sidecar/claude-runner.ts` | Claude sidecar source (compiled to .mjs by esbuild, includes findClaudeCli()) | +| `v2/sidecar/codex-runner.ts` | Codex sidecar source (@openai/codex-sdk dynamic import, sandbox/approval mapping) | +| `v2/sidecar/ollama-runner.ts` | Ollama sidecar source (direct HTTP to localhost:11434, zero external deps) | | `v2/sidecar/agent-runner-deno.ts` | Standalone Deno sidecar runner (not used by SidecarManager, alternative) | | `v2/sidecar/dist/claude-runner.mjs` | Bundled Claude sidecar (runs on both Deno and Node.js) | | `v2/src/lib/adapters/claude-messages.test.ts` | Vitest tests for Claude message adapter (25 tests) | +| `v2/src/lib/adapters/codex-messages.test.ts` | Vitest tests for Codex message adapter (19 tests) | +| `v2/src/lib/adapters/ollama-messages.test.ts` | Vitest tests for Ollama message adapter (11 tests) | | `v2/src/lib/adapters/agent-bridge.test.ts` | Vitest tests for agent IPC bridge (11 tests) | | `v2/src/lib/agent-dispatcher.test.ts` | Vitest tests for agent dispatcher (29 tests) | | `v2/src/lib/stores/conflicts.test.ts` | Vitest tests for conflict detection (28 tests) | diff --git a/TODO.md b/TODO.md index 1906802..b6a7579 100644 --- a/TODO.md +++ b/TODO.md @@ -9,10 +9,9 @@ - [ ] **Multi-machine TLS/certificate pinning** -- TLS support for bterminal-relay + certificate pinning in RemoteManager. - [ ] **Agent Teams real-world testing** -- Env var whitelist fix done. 3 test sessions ran ($1.10, $0.69, $1.70) but model didn't spawn subagents — needs complex multi-part prompts to trigger delegation. Test with CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1. - [ ] **Configurable stall threshold** -- health.yaml per-project config for stall threshold (currently hardcoded 15 min). Adaptive suggestions after 50 sessions from session_metrics data. -- [ ] **Add Codex/Ollama provider runners** -- Provider adapter pattern implemented (all 3 phases). Need to create codex-runner.ts and ollama-runner.ts sidecar runners + corresponding message adapters (codex-messages.ts, ollama-messages.ts) + register providers in App.svelte. - ## Completed +- [x] **Add Codex/Ollama provider runners** -- Full provider stack for both: ProviderMeta constants, message adapters (codex-messages.ts, ollama-messages.ts), sidecar runners (codex-runner.ts uses @openai/codex-sdk dynamic import, ollama-runner.ts uses direct HTTP). 30 new tests, 256 vitest total. | Done: 2026-03-11 - [x] **Worktree isolation per project (S-1 Phase 3)** -- UI toggle in SettingsTab, spawn with --worktree via sidecar extraArgs, CWD-based worktree detection in agent-dispatcher (matches .claude/.codex/.cursor patterns). 8 files, +125 lines. 226 vitest + 42 cargo tests. | Done: 2026-03-11 - [x] **S-2 — Session Anchors + Configurable Budget** -- Preserves important turns through compaction chains. Auto-anchors first 3 turns (observation-masked — reasoning preserved in full per research). Configurable budget via AnchorBudgetScale slider (Small=2K, Medium=6K, Large=12K, Full=20K) in SettingsTab per-project. Manual pin, promote/demote in ContextTab. Re-injection via system_prompt. 219 vitest + 42 cargo tests. | Done: 2026-03-11 - [x] **Agent provider adapter pattern** -- Multi-provider support (Claude, Codex, Ollama) via 3-phase adapter pattern. Core abstraction, Settings UI, Sidecar routing. 5 new files, 4 renames, 20+ modified. 202 vitest + 42 cargo tests. | Done: 2026-03-11 diff --git a/docs/provider-adapter/progress.md b/docs/provider-adapter/progress.md index 8a63c66..a9d3a24 100644 --- a/docs/provider-adapter/progress.md +++ b/docs/provider-adapter/progress.md @@ -61,3 +61,35 @@ 3. Extracted strip_provider_env_var() — strips CLAUDE*/CODEX*/OLLAMA* env vars (whitelists CLAUDE_CODE_EXPERIMENTAL_*) **Status:** All 3 phases complete. 202 vitest + 42 cargo tests pass. Zero regression. + +### 2026-03-11 — Provider Runners (Codex + Ollama) + +**Duration:** ~45 min + +**What happened:** + +**Research:** +1. Researched OpenAI Codex CLI programmatic interface (SDK, NDJSON stream format, thread events, sandbox/approval modes, session resume) +2. Researched Ollama REST API (/api/chat, NDJSON streaming, tool calling, token counts, health check) + +**Codex Provider (3 files):** +1. Created providers/codex.ts — ProviderMeta (gpt-5.4 default, hasSandbox=true, supportsResume=true, no profiles/skills/cost) +2. Created adapters/codex-messages.ts — adaptCodexMessage() maps ThreadEvents to AgentMessage[] (agent_message→text, reasoning→thinking, command_execution→Bash tool pair, file_change→Write/Edit/Bash per change, mcp_tool_call→server:tool, web_search→WebSearch, turn.completed→cost with tokens) +3. Created sidecar/codex-runner.ts — @openai/codex-sdk wrapper (dynamic import, graceful failure, sandbox/approval mapping, CODEX_API_KEY auth, session resume via thread ID) + +**Ollama Provider (3 files):** +1. Created providers/ollama.ts — ProviderMeta (qwen3:8b default, hasModelSelection only, all other capabilities false) +2. Created adapters/ollama-messages.ts — adaptOllamaMessage() maps synthesized chunk events (text, thinking from Qwen3, done→cost with eval_duration/token counts, always $0) +3. Created sidecar/ollama-runner.ts — Direct HTTP to localhost:11434/api/chat (zero deps, health check, NDJSON stream parsing, configurable host/model/num_ctx/think) + +**Registration + Build:** +1. Registered CODEX_PROVIDER + OLLAMA_PROVIDER in App.svelte onMount +2. Registered adaptCodexMessage + adaptOllamaMessage in message-adapters.ts +3. Updated build:sidecar script to build all 3 runners via esbuild + +**Tests:** +- 19 new tests for codex-messages.ts (all event types) +- 11 new tests for ollama-messages.ts (all event types) +- 256 vitest + 42 cargo tests pass. Zero regression. + +**Status:** Provider runners complete. Both providers infrastructure-ready (will work when CLI/server installed). diff --git a/docs/v3-progress.md b/docs/v3-progress.md index c481bdd..a751242 100644 --- a/docs/v3-progress.md +++ b/docs/v3-progress.md @@ -662,3 +662,24 @@ All editor themes map to the same `--ctp-*` CSS custom property names (26 vars). - [x] Added 7 new tests to agent-dispatcher.test.ts (detectWorktreeFromCwd unit tests + init CWD integration) - [x] vitest: 226/226 tests pass - [x] cargo test: 42/42 pass + +### Session: 2026-03-11 — Provider Runners (Codex + Ollama) + +#### Codex Provider +- [x] providers/codex.ts — ProviderMeta (gpt-5.4, hasSandbox, supportsResume) +- [x] adapters/codex-messages.ts — adaptCodexMessage (ThreadEvents → AgentMessage[]) +- [x] sidecar/codex-runner.ts — @openai/codex-sdk wrapper (dynamic import, graceful failure) +- [x] adapters/codex-messages.test.ts — 19 tests + +#### Ollama Provider +- [x] providers/ollama.ts — ProviderMeta (qwen3:8b, modelSelection only) +- [x] adapters/ollama-messages.ts — adaptOllamaMessage (streaming chunks → AgentMessage[]) +- [x] sidecar/ollama-runner.ts — Direct HTTP to localhost:11434 (zero deps) +- [x] adapters/ollama-messages.test.ts — 11 tests + +#### Registration + Build +- [x] App.svelte: register CODEX_PROVIDER + OLLAMA_PROVIDER +- [x] message-adapters.ts: register codex + ollama adapters +- [x] package.json: build:sidecar builds all 3 runners +- [x] vitest: 256/256 tests pass +- [x] cargo test: 42/42 pass