docs: restructure documentation into multilevel directory layout
New structure: docs/ split into 11 subdirectories (getting-started/, agents/, providers/, sidecar/, multi-machine/, plugins/, config/, production/, architecture/, contributing/, pro/). New files: - docs/README.md: navigation index with audience table - docs/getting-started/quickstart.md: install, build, first session - docs/config/ref-settings.md: all env vars, config files, databases - docs/architecture/overview.md: split from architecture.md (>300 lines) - docs/pro/README.md: Pro edition overview - docs/pro/features/analytics.md: analytics dashboard docs - docs/pro/features/cost-intelligence.md: budget + router docs Remaining docs being written by background agents — will be committed in follow-up when complete.
This commit is contained in:
Hibryda
parent
738574b9f0
commit
8251321dac
7 changed files with 1268 additions and 59 deletions
177
docs/pro/features/analytics.md
Normal file
177
docs/pro/features/analytics.md
Normal file
|
|
@ -0,0 +1,177 @@
|
|||
# Analytics Dashboard
|
||||
|
||||
> This documentation covers Pro edition features available in the agents-orchestrator/agents-orchestrator private repository.
|
||||
|
||||
## Overview
|
||||
|
||||
The Analytics Dashboard provides historical cost and usage visibility across all projects. It reads from the community `session_metrics` table (no additional data collection required) and presents aggregated views through three commands and corresponding UI components.
|
||||
|
||||
## Data Source
|
||||
|
||||
All analytics are derived from the existing `session_metrics` table in `sessions.db`:
|
||||
|
||||
```sql
|
||||
-- Community table (read-only access from Pro plugin)
|
||||
CREATE TABLE session_metrics (
|
||||
id INTEGER PRIMARY KEY,
|
||||
project_id TEXT NOT NULL,
|
||||
session_id TEXT NOT NULL,
|
||||
started_at TEXT,
|
||||
ended_at TEXT,
|
||||
peak_tokens INTEGER,
|
||||
turn_count INTEGER,
|
||||
tool_call_count INTEGER,
|
||||
cost_usd REAL,
|
||||
model TEXT,
|
||||
status TEXT,
|
||||
error_message TEXT
|
||||
);
|
||||
```
|
||||
|
||||
The Pro plugin opens a read-only connection to `sessions.db` and queries this table. It never writes to community databases.
|
||||
|
||||
## Commands
|
||||
|
||||
### pro_analytics_summary
|
||||
|
||||
Returns an aggregated summary for the specified period.
|
||||
|
||||
**Arguments:**
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `period` | `u32` | Yes | Number of days to look back (7, 14, 30, or 90) |
|
||||
| `projectId` | `String` | No | Filter to a single project. Omit for all projects. |
|
||||
|
||||
**Response: `AnalyticsSummary`**
|
||||
|
||||
```typescript
|
||||
interface AnalyticsSummary {
|
||||
totalCostUsd: number;
|
||||
totalSessions: number;
|
||||
totalTurns: number;
|
||||
totalToolCalls: number;
|
||||
avgCostPerSession: number;
|
||||
avgTurnsPerSession: number;
|
||||
peakTokensMax: number;
|
||||
periodDays: number;
|
||||
projectCount: number;
|
||||
}
|
||||
```
|
||||
|
||||
### pro_analytics_daily
|
||||
|
||||
Returns per-day statistics for charting.
|
||||
|
||||
**Arguments:**
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `period` | `u32` | Yes | Number of days (7, 14, 30, or 90) |
|
||||
| `projectId` | `String` | No | Filter to a single project |
|
||||
|
||||
**Response: `Vec<DailyStats>`**
|
||||
|
||||
```typescript
|
||||
interface DailyStats {
|
||||
date: string; // ISO 8601 date (YYYY-MM-DD)
|
||||
costUsd: number;
|
||||
sessionCount: number;
|
||||
turnCount: number;
|
||||
toolCallCount: number;
|
||||
}
|
||||
```
|
||||
|
||||
Days with no sessions are included with zero values to ensure continuous chart data.
|
||||
|
||||
### pro_analytics_model_breakdown
|
||||
|
||||
Returns usage grouped by model.
|
||||
|
||||
**Arguments:**
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `period` | `u32` | Yes | Number of days (7, 14, 30, or 90) |
|
||||
| `projectId` | `String` | No | Filter to a single project |
|
||||
|
||||
**Response: `Vec<ModelBreakdown>`**
|
||||
|
||||
```typescript
|
||||
interface ModelBreakdown {
|
||||
model: string;
|
||||
sessionCount: number;
|
||||
totalCostUsd: number;
|
||||
avgCostPerSession: number;
|
||||
totalTurns: number;
|
||||
totalToolCalls: number;
|
||||
pctOfTotalCost: number; // 0.0 to 100.0
|
||||
}
|
||||
```
|
||||
|
||||
Results are sorted by `totalCostUsd` descending.
|
||||
|
||||
## Period Selection
|
||||
|
||||
The UI presents four period options:
|
||||
|
||||
| Period | Label | Use Case |
|
||||
|--------|-------|----------|
|
||||
| 7 | Last 7 days | Recent activity check |
|
||||
| 14 | Last 14 days | Sprint review |
|
||||
| 30 | Last 30 days | Monthly cost review |
|
||||
| 90 | Last 90 days | Quarterly trend analysis |
|
||||
|
||||
The selected period is stored in the component state and defaults to 30 days. Changing the period triggers a fresh query (no client-side caching).
|
||||
|
||||
## UI Components
|
||||
|
||||
### AnalyticsDashboard.svelte
|
||||
|
||||
Top-level Pro tab component. Contains:
|
||||
|
||||
1. **Period selector** -- segmented button group (7/14/30/90).
|
||||
2. **Summary cards** -- four stat cards showing total cost, sessions, avg cost/session, peak tokens.
|
||||
3. **Daily cost chart** -- SVG bar chart rendered from `DailyStats[]`.
|
||||
4. **Model breakdown table** -- sortable table from `ModelBreakdown[]`.
|
||||
|
||||
### Daily Cost Chart
|
||||
|
||||
The chart is a pure SVG element (no charting library). Implementation details:
|
||||
|
||||
- Bars are sized relative to the maximum daily cost in the period.
|
||||
- X-axis labels show abbreviated dates (e.g., "Mar 5").
|
||||
- Y-axis shows cost in USD with appropriate precision ($0.01 for small values, $1.00 for large).
|
||||
- Hover tooltip shows exact cost, session count, and turn count for the day.
|
||||
- Colors use `var(--ctp-blue)` for bars, `var(--ctp-surface1)` for gridlines.
|
||||
- Responsive: adapts bar width to container via `container-type: inline-size`.
|
||||
|
||||
### Model Breakdown Table
|
||||
|
||||
Standard sortable table with columns: Model, Sessions, Cost, Avg Cost, Turns, Tools, % of Total. Default sort by cost descending. The `% of Total` column includes a proportional bar using `var(--ctp-sapphire)`.
|
||||
|
||||
## IPC Examples
|
||||
|
||||
```typescript
|
||||
// Get 30-day summary across all projects
|
||||
const summary = await invoke('plugin:agor-pro|pro_analytics_summary', {
|
||||
period: 30,
|
||||
});
|
||||
|
||||
// Get daily stats for a specific project
|
||||
const daily = await invoke('plugin:agor-pro|pro_analytics_daily', {
|
||||
period: 14,
|
||||
projectId: 'my-project',
|
||||
});
|
||||
|
||||
// Get model breakdown
|
||||
const models = await invoke('plugin:agor-pro|pro_analytics_model_breakdown', {
|
||||
period: 90,
|
||||
});
|
||||
```
|
||||
|
||||
## Limitations
|
||||
|
||||
- Analytics are computed on-demand from raw session_metrics rows. For large datasets (10,000+ sessions), queries may take 100-200ms. No materialized views or pre-aggregation.
|
||||
- Historical data depends on community session_metrics persistence. If sessions.db is deleted or migrated, analytics history resets.
|
||||
- Cost values are only available for providers that report cost (Claude, Codex). Ollama sessions show $0.00.
|
||||
238
docs/pro/features/cost-intelligence.md
Normal file
238
docs/pro/features/cost-intelligence.md
Normal file
|
|
@ -0,0 +1,238 @@
|
|||
# Cost Intelligence
|
||||
|
||||
> This documentation covers Pro edition features available in the agents-orchestrator/agents-orchestrator private repository.
|
||||
|
||||
Cost Intelligence comprises two systems: the Budget Governor (hard cost controls) and the Smart Model Router (cost-aware model selection). Both operate at the per-project level and integrate with the agent launch pipeline.
|
||||
|
||||
---
|
||||
|
||||
## Budget Governor
|
||||
|
||||
### Purpose
|
||||
|
||||
The Budget Governor enforces per-project monthly token budgets. It prevents runaway costs by warning at a soft limit and blocking agent launches at a hard limit.
|
||||
|
||||
### Budget Configuration
|
||||
|
||||
Each project can have a monthly budget with two thresholds:
|
||||
|
||||
| Threshold | Default | Behavior |
|
||||
|-----------|---------|----------|
|
||||
| Soft limit | 80% of budget | Emits a warning notification. Agent proceeds. |
|
||||
| Hard limit | 100% of budget | Blocks agent launch. Returns error to UI. |
|
||||
|
||||
Budgets are denominated in USD. The governor tracks cumulative cost for the current calendar month by summing `cost_usd` from `session_metrics` and `pro_budget_log`.
|
||||
|
||||
### Emergency Override
|
||||
|
||||
When the hard limit is reached, the UI displays a confirmation dialog with the current spend and budget. The user can authorize an emergency override that allows the next session to proceed. Overrides are logged in `pro_budget_log` with `override = true`. Each override is single-use; subsequent launches require a new override.
|
||||
|
||||
### SQLite Tables
|
||||
|
||||
Both tables reside in `agor_pro.db`.
|
||||
|
||||
```sql
|
||||
CREATE TABLE pro_budgets (
|
||||
project_id TEXT PRIMARY KEY,
|
||||
monthly_budget_usd REAL NOT NULL,
|
||||
soft_limit_pct REAL NOT NULL DEFAULT 0.80,
|
||||
hard_limit_pct REAL NOT NULL DEFAULT 1.00,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE TABLE pro_budget_log (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
project_id TEXT NOT NULL,
|
||||
session_id TEXT NOT NULL,
|
||||
cost_usd REAL NOT NULL,
|
||||
cumulative_usd REAL NOT NULL,
|
||||
month TEXT NOT NULL, -- YYYY-MM format
|
||||
override INTEGER NOT NULL DEFAULT 0,
|
||||
logged_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
```
|
||||
|
||||
### Commands
|
||||
|
||||
#### pro_budget_set
|
||||
|
||||
Creates or updates a budget for a project.
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `projectId` | `String` | Yes | Project identifier |
|
||||
| `monthlyBudgetUsd` | `f64` | Yes | Monthly budget in USD |
|
||||
| `softLimitPct` | `f64` | No | Soft limit percentage (default 0.80) |
|
||||
| `hardLimitPct` | `f64` | No | Hard limit percentage (default 1.00) |
|
||||
|
||||
#### pro_budget_get
|
||||
|
||||
Returns the budget configuration for a project. Returns `null` if no budget is set.
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `projectId` | `String` | Yes | Project identifier |
|
||||
|
||||
#### pro_budget_check
|
||||
|
||||
Pre-dispatch hook. Checks whether the project can launch a new agent session.
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `projectId` | `String` | Yes | Project identifier |
|
||||
|
||||
**Response: `BudgetCheckResult`**
|
||||
|
||||
```typescript
|
||||
interface BudgetCheckResult {
|
||||
allowed: boolean;
|
||||
currentSpendUsd: number;
|
||||
budgetUsd: number;
|
||||
pctUsed: number; // 0.0 to 1.0
|
||||
softLimitReached: boolean;
|
||||
hardLimitReached: boolean;
|
||||
month: string; // YYYY-MM
|
||||
}
|
||||
```
|
||||
|
||||
#### pro_budget_log_usage
|
||||
|
||||
Records cost for a completed session. Called by the agent dispatcher on session end.
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `projectId` | `String` | Yes | Project identifier |
|
||||
| `sessionId` | `String` | Yes | Session identifier |
|
||||
| `costUsd` | `f64` | Yes | Session cost |
|
||||
| `override` | `bool` | No | Whether this was an emergency override |
|
||||
|
||||
#### pro_budget_reset
|
||||
|
||||
Resets the cumulative spend for a project's current month. Deletes all `pro_budget_log` entries for the current month.
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `projectId` | `String` | Yes | Project identifier |
|
||||
|
||||
#### pro_budget_list
|
||||
|
||||
Returns budgets and current spend for all projects.
|
||||
|
||||
**Response: `Vec<ProjectBudgetStatus>`**
|
||||
|
||||
```typescript
|
||||
interface ProjectBudgetStatus {
|
||||
projectId: string;
|
||||
monthlyBudgetUsd: number;
|
||||
currentSpendUsd: number;
|
||||
pctUsed: number;
|
||||
softLimitReached: boolean;
|
||||
hardLimitReached: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
### Pre-Dispatch Integration
|
||||
|
||||
The budget check is wired into the agent launch path:
|
||||
|
||||
1. User triggers agent start in AgentPane.
|
||||
2. `AgentSession.startQuery()` calls `pro_budget_check` before invoking the sidecar.
|
||||
3. If `softLimitReached`: toast warning, agent proceeds.
|
||||
4. If `hardLimitReached`: blocks launch, shows override dialog.
|
||||
5. On override confirmation: calls `pro_budget_log_usage` with `override: true`, then proceeds.
|
||||
6. On session completion: `agent-dispatcher` calls `pro_budget_log_usage` with actual cost.
|
||||
|
||||
---
|
||||
|
||||
## Smart Model Router
|
||||
|
||||
### Purpose
|
||||
|
||||
The Smart Model Router recommends which model to use for a given agent session based on the project's routing profile and the agent's role. It does not force model selection -- it provides a recommendation that the user can accept or override.
|
||||
|
||||
### Routing Profiles
|
||||
|
||||
Three built-in profiles define model preferences:
|
||||
|
||||
#### CostSaver
|
||||
|
||||
Minimizes cost by preferring smaller models for routine tasks.
|
||||
|
||||
| Role | Recommended Model |
|
||||
|------|-------------------|
|
||||
| Manager | claude-sonnet-4-5-20250514 |
|
||||
| Architect | claude-sonnet-4-5-20250514 |
|
||||
| Tester | claude-haiku-4-5-20250514 |
|
||||
| Reviewer | claude-haiku-4-5-20250514 |
|
||||
| Default (no role) | claude-sonnet-4-5-20250514 |
|
||||
|
||||
#### QualityFirst
|
||||
|
||||
Maximizes output quality by preferring the most capable model.
|
||||
|
||||
| Role | Recommended Model |
|
||||
|------|-------------------|
|
||||
| Manager | claude-opus-4-5-20250514 |
|
||||
| Architect | claude-opus-4-5-20250514 |
|
||||
| Tester | claude-sonnet-4-5-20250514 |
|
||||
| Reviewer | claude-sonnet-4-5-20250514 |
|
||||
| Default (no role) | claude-opus-4-5-20250514 |
|
||||
|
||||
#### Balanced
|
||||
|
||||
Default profile. Balances cost and quality per role.
|
||||
|
||||
| Role | Recommended Model |
|
||||
|------|-------------------|
|
||||
| Manager | claude-sonnet-4-5-20250514 |
|
||||
| Architect | claude-opus-4-5-20250514 |
|
||||
| Tester | claude-haiku-4-5-20250514 |
|
||||
| Reviewer | claude-sonnet-4-5-20250514 |
|
||||
| Default (no role) | claude-sonnet-4-5-20250514 |
|
||||
|
||||
### Commands
|
||||
|
||||
#### pro_router_recommend
|
||||
|
||||
Returns the recommended model for a given project and role.
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `projectId` | `String` | Yes | Project identifier |
|
||||
| `role` | `String` | No | Agent role (manager/architect/tester/reviewer) |
|
||||
|
||||
**Response:**
|
||||
|
||||
```typescript
|
||||
interface RouterRecommendation {
|
||||
model: string;
|
||||
profile: string; // CostSaver | QualityFirst | Balanced
|
||||
reason: string; // Human-readable explanation
|
||||
}
|
||||
```
|
||||
|
||||
#### pro_router_set_profile
|
||||
|
||||
Sets the routing profile for a project.
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `projectId` | `String` | Yes | Project identifier |
|
||||
| `profile` | `String` | Yes | Profile name: CostSaver, QualityFirst, or Balanced |
|
||||
|
||||
#### pro_router_get_profile
|
||||
|
||||
Returns the current routing profile for a project. Defaults to Balanced.
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `projectId` | `String` | Yes | Project identifier |
|
||||
|
||||
#### pro_router_list_profiles
|
||||
|
||||
Returns all available routing profiles with their role-to-model mappings. Takes no arguments.
|
||||
|
||||
### Integration with Agent Launch
|
||||
|
||||
The router recommendation is surfaced in the AgentPane model selector. When a project has a routing profile set, the recommended model appears as a highlighted option in the dropdown. The user retains full control to select any available model.
|
||||
Loading…
Add table
Add a link
Reference in a new issue