History

Every saved version of decisions/architecture-pivot.md, newest first. Each row shows what changed compared to the version before it.

• v1currentCreatedby agent · 18d ago
  Initial content

  # Architecture pivot: platform stores, agent reasons
  
  ## Problem
  
  The original Hive scaffold included a workers package (BullMQ + Anthropic SDK) that called LLMs server-side to ingest sources, generate page summaries, and answer queries. Each platform request paid the latency and dollar cost of an LLM call. Worse, it positioned Hive as another "we run the LLM for you" SaaS — competing with NotebookLM, ChatGPT file uploads, etc.
  
  [[karpathy-llm-wiki|Karpathy's gist]] argued for the opposite shape: the platform stores, indexes, searches, and serves; the user's agent (via MCP) does all the reasoning and writing.
  
  ## Alternatives
  
  1. **Keep server-side LLM** — managed experience, but expensive and limits which model the user can choose.
  2. **Hybrid** — platform makes "easy" LLM calls (titles, summaries) but offloads heavy work to the agent. Complicates the data model; agent and platform writes race.
  3. **Agent-driven (chosen)** — platform never calls a generative LLM. Agent owns all writes via MCP tools. Platform's only AI-adjacent call is embedding (Voyage / OpenAI) for pgvector similarity search.
  
  ## What we chose
  
  Removed `packages/workers` entirely. Dropped `ingest_jobs` table and `sources.status` column. Kept embedding generation server-side (it runs on every source/page write, blocking; ~1s per chunk against Voyage). All page/source writes flow through MCP tools the agent calls — see [[decisions/remote-mcp]].
  
  `CLAUDE.md` codifies the rule:
  
  > The platform does NOT call generative LLMs. It stores, indexes, searches, and serves. The user's agent (via MCP) does all reasoning and writing.
  
  ## Why
  
  - **Cost** — Hive runs as cheap infrastructure (Postgres + Redis + a Hono process). The agent's tokens are the user's problem.
  - **Model choice** — the user picks Claude / GPT / local; we don't constrain.
  - **Quality** — page-curation prompts evolve faster in agent context than in a server codebase. Skill files ([[guides/installing-hive-mcp]] mentions the `hive-wiki-builder` skill we ship) can be iterated independently.
  - **Honest positioning** — Hive is plumbing, not magic. The user keeps control of the loop.
  
  ## What we'd revisit
  
  - Embeddings are still a synchronous AI-adjacent call inside `addTextSource` (`packages/api/src/services/sourceService.ts:92`). If Voyage is down, source writes 5xx. Worth making this best-effort with retry-later semantics.
  - Some MCP tool descriptions are still written assuming an LLM caller. They could be tightened to be more "tool surface" and less "API documentation". See [[decisions/remote-mcp]].