Agent systems that are operated rather than demonstrated

Capability statement. This is a working estate, not a prototype: more than sixteen pipelines in production, six providers behind a single gateway, retrieval that filters on metadata before it searches, and a voice front-end used daily by people who did not build it.

What is actually run

Routing. Every model call goes through one gateway — OmniRoute. Six providers sit behind it: DeepSeek, MiniMax, OpenRouter, OpenCode-Go, Gemini and OpenAI. Requests are classified before they are sent, so the cheapest model that can do the job gets the job. No project in the estate calls a provider directly; that is a rule, not a preference.

Pipelines. Ten or more are maintained on n8n, Prefect and LangGraph — email scanners, school-portal scanners, calendar scanners, project agents — plus six Paperclip pipelines that run work through a bench of narrow specialist agents behind human approval gates. The estate is registered in a dashboard built to keep track of it.

Harnesses. JEV is a custom harness with its own routing, tool set and memory rather than a wrapper around a vendor SDK. Work is executed as sequences of narrow agents instead of one large prompt; capability is loaded per task through a skill-selection step, so the agent gets what this job needs and not an enormous system prompt.

Memory. Two deliberate layers. Engram keeps what should persist — curated summaries, decisions, prompts and session handoffs in local SQLite, integrated with JEV for fast compaction. Headroom keeps what should not persist out of the window, stripping and compressing prompt context, tool output and logs before they reach the model. Storage and token control are separate problems and are solved separately.

Retrieval. ChromaDB first, then replaced with Qdrant for payload-filtered retrieval and a smaller footprint. LangGraph query graphs on top; Prefect and n8n for ingestion. The corpus is metadata-tagged at ingest, which means retrieval filters before it searches — the difference between a device answer and a work-history answer is a filter, not a better prompt.

Voice. Local Whisper for speech to text with a fallback, Piper for text to speech, MiniMax for a cloned voice with emotion and non-verbal cues. A custom wake word on an ESP32 with LED confirmation. Voice is a system front-end: it controls lighting, music, calendars, shopping, email, SMS and alerts, and it answers through eight Snapcast endpoints with music ducked underneath.

Guardrails. Custom hooks live in the model path and block injection and constrain tool use, built because the house voice channel is shared and unauthenticated and there was no vendor control to borrow. Consequential actions sit behind approval gates, and no model sits in the path of a fast or dangerous command. The public site applies the same reasoning from the other end: the assistant it exposes has no tools at all, so the attack surface is a conversation rather than a shell.

Agent operations. Agent sessions are standardised on TUIOS across four hosts — sessions survive a disconnect, agents report their own state, and approvals land in one inbox. TUIOS is a third-party tool: the contribution is operations and packaging, not authorship. It is pinned at 0.10.1 in a hand-written Nix derivation because nixpkgs lagged at 0.7.0, and its daemon is declared as a systemd user service so the fleet is reproducible rather than hand-installed.

Evidence

Claim	Evidence	Artefact
Sixteen or more pipelines in production	Ten or more on n8n, Prefect and LangGraph plus six on Paperclip	Paperclip and n8n pipelines
Six providers behind one gateway	DeepSeek, MiniMax, OpenRouter, OpenCode-Go, Gemini, OpenAI	Model routing
Retrieval filters before it searches	Corpus tagged at ingest; Qdrant payload filters; LangGraph query nodes	Corpus and query graph
Spoken output reaches eight endpoints	Eight Snapcast clients with music ducking	Voice pipeline
Guardrails are code in the path	Injection-blocking hooks and constrained tool use	Guardrails
Memory is two layers by design	Engram for persistence, Headroom for context control	Memory layers
Agent operations are standardised	TUIOS across four hosts with one approval inbox	House AI hub

Trade-offs and limits

• Local inference was measured, then demoted. The estate started local on Ollama. Hardware constraints made local inference both slower and more expensive than partitioned remote calls, so heavy flows moved remote while sensitive and low-latency work stayed local. That is a documented comparison, not a taste in hardware.
• The retired stack is in the record. Airflow DAGs were abandoned for RAM, ChromaDB was replaced by Qdrant, and a visual-builder layer was dropped. Each replacement is stated rather than hidden, because the reasoning is the transferable part.
• Not everything is automated. Parts of the house are automated end to end and parts still need a person to press the button. The estate is described as it is run, not as it was imagined.
• Cost is bounded by routing, not by hope. Caps and a refusal path exist so a runaway session cannot become a bill; the routing layer keeps the median request on a cheap model.

Drill-downs

• House AI hub — service topology — level 3
• Model routing and cost control — level 3
• Paperclip and n8n pipelines in production — level 3
• Retrieval corpus and query graph — level 3
• Voice pipeline — speech in and speech out — level 3
• Guardrails — hooks in the model path — level 3
• Memory in two layers — level 3

Period and confidence

period: 2020–. The start is anchored on the homelab year in the evidence notes, because the AI estate and the homelab have the same origin. Confidence is running: this is the current arrangement, verified 2026-10-26 against the evidence notes.