Skip to content

ADR-0009 — Deeper Research: LiteLLM proxy instead of llm CLI

Date: 2026-06-05 Status: Accepted Vikunja: #906–910 (Deeper Research deployment tasks)

Context

The Deeper Research implementation manual assumed llm CLI (Simon Willison's tool) as the AI backend, with Ollama for local/CRITICAL sensitivity subjects. The VPS has neither installed.

  • llm CLI: not installed; would require pip + plugin management per model provider
  • Ollama: removed 2026-04-18 (VPS memory pressure, 500 errors; freed ~8GB RAM)

The VPS already runs a LiteLLM proxy at http://litellm:4000 (OpenAI-compatible API) with 25 models pre-configured across Anthropic, Mistral, DeepSeek, xAI, Gemini, and 7 OpenRouter free models. All API keys are already managed there.

Decision

All AI calls in Deeper Research use the OpenAI Python client pointed at the LiteLLM proxy:

from openai import AsyncOpenAI
client = AsyncOpenAI(
    base_url=os.environ["LITELLM_BASE_URL"],   # http://litellm:4000
    api_key=os.environ["LITELLM_API_KEY"],
)

Model aliases (e.g. brain-query, standard-mistral, free_tier) are resolved by LiteLLM — Deeper Research never references provider-specific model IDs.

For CRITICAL sensitivity (no Ollama): Claude Haiku (brain-query alias) is the sovereign-safe default. A --force-critical flag is available for explicit acknowledgement.

Consequences

Positive: - Zero new dependencies: no llm CLI install, no plugin management, no Ollama memory overhead - All 25 LiteLLM models available to DR immediately, including future additions - Cost tracking, rate limiting, and provider failover handled by existing LiteLLM config - CRITICAL sensitivity handled by Anthropic (US, paid) — acceptable given Ollama unavailability

Negative: - CRITICAL subjects no longer truly local (data leaves VPS to Anthropic US) - If LiteLLM proxy is down, all DR AI calls fail (single point of failure — same as all other services)

Neutral: - deeper-research CLI script at /usr/local/bin/deeper-research provides identical UX to the manual's llm deeper-research plugin command